Two hundred tabs exposed the wrong broadcast shape

In a shared Reedapt project, you can see who else is there and which line they are working on. Moving to another line changes a small piece of presence state: this tab is now looking here.

Our earlier approach answered that small change with the whole list of people in the project. It sent the list to everyone.

That is easy to reason about. Each recipient replaces its old list with a fresh one. Nobody has to work out which entry changed or whether an earlier update went missing.

It became expensive in a load test with two hundred editor tabs. Those were test connections, not two hundred independent people collaborating on a film, and the test is not a supported-participant limit. It gave us a way to see what happened when a lot of tabs kept producing small updates.

The shape of the broadcast mattered more than the size of any single update.

There were two kinds of repetition

Imagine a project with twenty tabs, each changing its presence once a second. That gives the server twenty updates to handle. If each update is broadcast to all twenty tabs, there are four hundred client deliveries in that second.

Now make every delivery contain all twenty entries. We haven't just repeated the delivery. We've repeated mostly unchanged state inside it.

Under those assumptions, delivered messages grow with the square of the tab count. The number of presence entries carried across those deliveries grows with its cube. Payload bytes depend on the actual serialized entries and envelopes, so they are a separate measurement again.

That distinction is worth making. A single call to emit into a room isn't a single delivery to a browser. And replacing a full list with a patch doesn't, by itself, reduce the number of recipients or how often we broadcast.

The example uses synthetic presence entries over a ten-second window. It counts server emits, client deliveries, and uncompressed application JSON bytes separately. It is not a network-throughput benchmark or a reproduction of the old production test. Initial joins, recovery snapshots, transport framing, compression, and browser rendering are excluded.

You can also separate the equation from the payload accounting. If N tabs each produce r updates a second for T seconds, and every update reaches all N tabs, deliveries are N × N × r × T. Doubling N changes both the number of sources and the number of recipients. Changing only r changes one factor.

This is a count under the stated assumptions, not an equation for how fast the server will run. Message size, processing cost, connection behavior, and recovery traffic still matter.

Patches reduced what we repeated

The service now keeps track of the entries that arrived, changed, or left since the last flush. A patch contains complete changed entries in an upsert list and the keys of departed entries in a remove list.

That is a patch to the presence list, not a field-by-field description of every action. If a tab moves between several lines before a flush, the latest whereabouts can replace its earlier whereabouts in the pending update.

Presence makes that possible. If someone is now on line twelve, showing every intermediate line they crossed is usually not useful. We are not applying the same rule to someone's transcript edits. Those are document changes, with different correctness requirements.

Entries also represent tabs, not just people. One person can have more than one tab open. The patch needs to identify the entry being replaced or removed without confusing it with that person's other connection.

Coalescing reduced how often we repeated it

The browser already limits how often it reports changed whereabouts. The server also coalesces changes by project, with flushes spaced at least a quarter of a second apart.

Several updates can therefore become one patch broadcast. If two tabs change during that interval, both can appear in the same patch. If one tab changes repeatedly, the pending state can carry its latest entry rather than a history of every intermediate position.

There is a trade-off. A collaborator's marker can wait for the next flush instead of moving immediately. That is acceptable only if the marker remains useful. Batching is not a reason to hide latency or postpone important document changes.

The model uses aligned quarter-second windows to make the counts reproducible. The real server schedules against the time of its last flush, so the first flush and exact event boundaries can differ.

Would a different queue order have solved it?

FIFO starts the oldest waiting item first. LIFO starts the newest waiting item first. Either can be useful in the right system, but neither makes a server process more work per second by itself.

The queue below keeps the arrival rate and processing rate separate. Start it with arrivals faster than processing and the waiting work grows. Switch to LIFO and newer jobs can pass older ones, but the extra work hasn't disappeared. A job already being processed is allowed to finish in either mode.

For a simpler view of the pressure, the next equation treats work as a continuous quantity: excess work grows at the positive difference between arrival and processing rates. It starts empty, assumes constant rates, and ignores the first-arrival delay and individual-job boundaries in the queue above. It is a fluid approximation, not an exact queue-length formula or a waiting-time prediction.

Presence coalescing does something different from either queue discipline. It can replace an older, unsent whereabouts entry with the newest entry for that same tab. We don't need to replay every line a marker crossed before showing where it ended up. That replacement is what makes skipping intermediate presence states reasonable. It would not justify dropping transcript edits or treating every kind of work as interchangeable.

A patch needs a recovery path

A full list can replace whatever a client currently knows. A patch assumes the client has a suitable starting point.

On joining, a client gets a complete presence list and a sequence number. Subsequent patches are numbered. The editor applies the next expected patch and ignores older or duplicate numbers.

If it sees a gap, it doesn't continue applying later patches and hope for the best. It holds the existing list, asks for a fresh snapshot, and resumes from the sequence number returned with it. Further patches don't start a new synchronization request while that recovery is already in progress.

If the service can't answer immediately, the client retries with backoff. A reconnect starts from a fresh join. The snapshot never stopped being useful; we moved it to the places where rebuilding the client's view is actually the task.

The recovery example is deliberately small: it drops one numbered patch and shows the client waiting for a snapshot. It isn't a simulation of network timing or every reconnection case.

The next example adds a clock and lets you change the connection. Its losses are deliberately predictable, and recovery snapshots always arrive if the connection is up. It shows the difference between a patch that has not arrived and a sequence gap the client can detect; it does not reproduce the production retry logic.

Older clients still needed the full list

We couldn't assume every open editor tab would understand the new event as soon as the backend changed.

Newer clients advertise patch support when they join. The service routes them into the patch audience. Older clients receive complete lists in a separate audience, but those lists also benefit from coalescing.

A flush can therefore produce two server emits: a patch for one audience and a snapshot for the other. It doesn't send both formats to every client. The compatibility path still costs more payload for older tabs, which is why the example lets you vary that share separately.

This made the protocol change less dependent on the frontend and backend arriving at exactly the same moment. It didn't make supporting old clients free.

The optimization wasn't “never send a snapshot”

We still send snapshots when a tab joins or needs to recover. We still send them to clients that don't understand patches. For ordinary movement between lines, we send the changed presence entries, and we group nearby changes before broadcasting them.

The client tests we ran for this draft cover ordered patches, duplicates, missing sequence numbers, reconnects, sync retries, and the older full-state path. They check protocol behavior, not how many people a production project should admit.

The two-hundred-tab investigation helped expose a repeated-work problem. The useful fix was to separate three decisions we had previously bundled together: who needs the update, what they need in it, and how often they need to receive it.