Wire protocol
The protocol between client and server is a small set of typed events. Application code never touches them directly.
All events travel over an event bus — WebSocket frames from a Durable Object or a Node process, function calls in-process. The vocabulary is identical either way.
Client → server
Section titled “Client → server”| Event | Meaning |
|---|---|
doc:subscribe |
Start receiving changes for a document — carrying the last seen sequence and generation (if any) and per-Y.Doc state vectors, so each lane answers with only what’s missing. That cursor only counts for the generation it names. Server replies with doc:init or doc:resume (or doc:notfound). |
doc:unsubscribe |
Stop receiving changes. |
doc:create |
Create a document — id, type, initial data, an optional name, the generation the client minted for its optimistic copy (which the server adopts; a create naming a purged generation is rejected notFound), the schema sequence the data was authored against, optionally with embedded Yjs documents. |
doc:update |
Change a document — a JSON Patch and/or Yjs updates, an optional guard (sequence or patch), the generation and schema sequence it was written against, and the client’s event id for optimistic confirmation. A write naming a generation the document no longer has — its id was purged and reused — is rejected notFound; one naming an earlier schema sequence is brought forward through the migrations stamped since. |
doc:command |
Run a domain command — the command’s name and arguments instead of a patch; the server runs the registered mutator over the document it holds and broadcasts the resulting diff as a doc:patch. Carries the base sequence, the generation and, for an exact contract, a "sequence" guard. A refusal comes back as a doc:error of category commandRefused with the mutator’s code. |
doc:delete |
Soft-delete a document — it leaves the index and the read path, but its history is retained for restore. Takes an optional sequence guard (“delete only if unchanged since I looked”) and, like doc:update, names the generation it was written against. |
doc:restore |
Undo a soft delete — the document re-enters the index at the exact sequence it left, and subscribers receive a fresh doc:init. Like doc:delete, it names the generation it was aimed at — the one the client saw deleted, from its doc:deleted or the document’s sys:trash entry — and a restore naming a generation the document no longer has is rejected notFound. |
doc:purge |
PERMANENTLY destroy a batch of soft-deleted documents (up to 100) and release their ids — a purged id afterwards answers doc:notfound like one that never existed, and may be created again. All-or-nothing — one bad or unauthorized target rejects the whole batch and destroys nothing. Requires the docType’s explicit access.purge rule (the default is nobody, admins included); success is one sys:trash removal patch at one sequence. |
doc:rename |
Set a document’s name (the name lives in sys:index, not the document body). Patches the document’s sys:index entry; never touches its data, sequence or event log. Last-writer-wins, but — like doc:update — it names the generation it was written against when the client holds a copy, and a rename naming a generation the document no longer has is rejected notFound. |
doc:get-events |
Fetch one page of a document’s event logs — both lanes: the JSON patch events and the Yjs update events, after a per-lane cursor (afterSequence, afterYjsSequence), at most limit rows (500 at most). The answer’s hasMore says whether rows remain, and its generation which incarnation of the document the page came from; getAllDocumentEvents walks every page, starting over if the document is purged and re-created mid-walk. |
doc:get-processed-writes |
Ask which of the client’s own writes on a document the server has processed — the answer, from the write de-duplication store, is what lets a client re-send safely behind its own later writes (see reconnect and replay). |
clock:ping |
Ask for the server’s clock — sent only by a client that started its connection clock. Carries nothing; the client times the round trip itself. |
Server → client
Section titled “Server → client”| Event | Meaning |
|---|---|
doc:init |
Full JSON snapshot at subscribe time — sequence, generation, type, data; a client holding a copy of another generation of the id replaces it, embedded Y.Docs included. The Yjs lane rides along as exact per-Y.Doc diffs against the subscribe’s state vectors (full states when none were sent), plus tombstone notices and the server’s own vectors. A create/restore commit’s init also carries the document’s listing entry, so it is never readable-but-unlisted while its sys:index patch is in flight. |
doc:patch |
An incremental change — patch and/or Yjs updates, the new sequence, and the originating client event id (so the originator can retire its optimistic entry). May also carry yjsDeleted — ids of embedded Y.Docs the orphan GC reclaimed on this write, so subscribers drop those sub-docs immediately. |
doc:error |
A rejected event — categorized (validation failure, malformed request, undeclared type, failed guard, limit, a refused command with its code), tied back to the client event id. Guard failures and command refusals are expected signals, not faults. |
doc:notfound |
The subscribed document doesn’t exist. |
doc:deleted |
The document was soft-deleted — pushed to live subscribers when a delete commits, and the answer to subscribing to an already-deleted document (deliberately distinct from doc:notfound). Names the deleted document’s type and generation, which the client keeps with the tombstone: they type a later restore or rename and aim it at the incarnation that was deleted. |
doc:resume |
The client’s JSON state is current and the document is valid — no data re-transferred. The Yjs lane rides along independently: diffs and tombstones for whatever the client’s vectors were missing, plus the server’s own vectors. |
doc:processed-writes |
The answer to doc:get-processed-writes: the asked-about event ids the server holds as processed, each with the sequence its write was answered at. |
doc:resync |
Presence-only cold-start nudge — carries just the docId, asking a presence document’s subscribers to republish their cells. See below. |
clock:pong |
The answer to clock:ping, to the asking connection only: the ping’s id and the server’s time (Unix epoch milliseconds) as it handled it. The one pair of events that is about the connection rather than a document. |
A subscribe is the one exchange where the answer branches, and where the two lanes visibly answer independently:
sequenceDiagram
accTitle: What a doc:subscribe earns back
accDescr {
A client subscribes carrying the last sequence and generation it saw and a
state vector per embedded Y.Doc. If the document does not exist the server
answers notfound, and if it is soft-deleted it answers deleted. Otherwise
the JSON lane answers resume when the client's copy is the document's
generation at its current sequence, transferring no data, or init with a
full snapshot when the client is behind, sent no sequence, or holds another
generation — a copy of a purged document whose id was reused, whose state
vectors are then ignored. Either answer carries the Yjs lane alongside it
as exact diffs against the client's vectors, plus the server's own vectors
so the client can push back anything the server lacks.
}
participant c as Client
participant s as Server
c->>s: doc:subscribe — last seen sequence and generation<br/>+ a state vector per embedded Y.Doc
alt no such document
s-->>c: doc:notfound
else soft-deleted
s-->>c: doc:deleted
else client's copy is current — same generation, same sequence
s-->>c: doc:resume — no JSON transferred
else client is behind, sent no sequence, or holds another generation
s-->>c: doc:init — full JSON snapshot
end
Note over c,s: either answer carries the Yjs lane alongside it:<br/>exact diffs against the client's vectors, tombstones,<br/>and the server's own vectors to push back against
Presence rides the same events
Section titled “Presence rides the same events”Presence adds just one event of its own — the
doc:resync nudge. Otherwise a sys:presence:<presenceType>:<docId> document is
subscribed, initialized, patched, and updated through exactly the doc:*
vocabulary above — a cell write is a doc:update, peers learn of it via
doc:patch, and the per-session sys:presence-view:<presenceType>:<docId> is a
client-side projection the wire never sees. What differs is server-side
lifetime, not the protocol: presence state lives only in memory, so it never
appends to an event log (doc:get-events answers an explicit error — no history
exists, ever) and is dropped when the publishing connection goes.
That ephemerality surfaces on the wire in four ways:
- Liveness is a patch, not an event. When a connection closes, the server
doesn’t invent a disconnect event: it removes that connection’s cells, and
peers receive the removal as an ordinary
doc:patch. A reconnecting client simply republishes its cells. Thestaleflag in the view is client-side, with no wire vocabulary of its own: a client marks its peers stale while its own view is unverified (after it reconnects, or after adoc:resync) and hides any that aren’t confirmed within the grace window. - Ephemeral Y.Docs re-send in full. A cell’s
streaming Y.Doc
rides the normal Yjs lane inside
doc:update/doc:patch, but because it is never logged it is the one place catch-up isn’t a state-vector diff: on the owner’s reconnect the server can’t reconstruct it, so the owner re-sends the Y.Doc’s full state. - Cold starts nudge, they don’t reset. When a transport host wakes from
hibernation or restarts, its in-memory cell registry is gone, so it sends each
still-subscribed presence document a
doc:resync, asking its subscribers to republish. It is deliberately not an emptydoc:init, which would clear the peers a client is still showing; instead the client arms its staleness ledger and republishes its own cell, so the roster refreshes without flickering empty. - Sequences count in epochs. A presence document’s sequence lives in memory
too, and the server lets it go when the last cell leaves, so the next publish
starts again at 1. Every presence
doc:initanddoc:patchtherefore names the epoch its sequence counts in. A client that sees a frame that doesn’t follow its copy (a skipped sequence, or a new epoch arriving while it still holds cells confirmed by the server since its last wake) knows a frame went missing, applies what arrived, and resubscribes for a freshdoc:init. Cells retained afterdoc:resyncare excluded from that check until a server patch reconfirms or removes them. The new epoch can therefore start at 1 without cutting short the grace period for peers still republishing. A missing frame within the new epoch still triggers a fresh snapshot.
Design notes
Section titled “Design notes”- Binary on the wire. Every event is one frame: a short header, a JSON envelope, and a table of byte payloads that the Yjs lanes index into, so a Yjs update is read in place from the frame the transport already holds. Both ends speak one frame version and deploy together.
- Every collection is bounded. A frame is capped at 16 MiB, and the
server checks the size of each collection in a client event — patch
operations, Yjs lanes, update bytes, purge ids — before parsing it, so an
oversized payload is refused up front with
sizeLimitExceeded. A small patch can still ask for a lot (acopythat doubles a value, forty times over), so applying one is bounded too, and refused with the same category. - Deltas dominate. After
doc:init, a subscriber receives patches. The exceptions are existence transitions, not content: adoc:restorepushes a freshdoc:initto live subscribers (as does the presence re-init that rides along with it), because the document’s re-entry can’t be expressed as a patch against a tombstone. - One update, two payloads — one sequence. A single
doc:updatecarries structured patches and Yjs binary updates together, so a mixed edit (retitle + type in the body) is one event. But the lanes version independently: only the JSON half advancessequence; a Yjs-only update repeats it unchanged on itsdoc:patchbroadcast. - Errors are addressed, not broadcast. Rejections return to the sender with its event id; other subscribers never see them.
- Orphaned Y.Docs are reclaimed live. When a write orphans an embedded
Y.Doc — a removed
yjsRef, or a discarded session copy — the reclaimed ids ride the samedoc:patchasyjsDeleted, so live subscribers tear those sub-docs down at once rather than waiting for their next reconnect’s init/resume. - Deletion is lifecycle, not content. A delete never appends to the
document’s own event log — it’s recorded in the folder’s membership log, so
a delete → restore round-trip returns the document at the exact sequence it
left, history intact. Deleted documents move from
sys:indexinto the synthesizedsys:trashdocument; subscribe to it and a trash UI updates live, with no dedicated listing request. The move is broadcast add-before-remove, so a row switches lists without ever vanishing in between. - Sequences make hydration cheap. Document state can be fetched outside
the socket — say, server-rendered over plain HTTP — and handed to the
client as preloaded state. When the client later subscribes, it sends the
sequence and generation it already holds: the server answers
doc:resume(no data) if nothing changed, or a freshdoc:initif the client is behind — or holds another generation of the id, so a document purged and recreated under the same id is never mistaken for the old copy. A document the server’s read flags invalid is also answereddoc:init, whatever the sequence: the flag travels ondoc:initalone, so adoc:resumealways means current and valid. JSON catch-up is deliberately a full snapshot, not incremental patches — simple over clever, at the cost of re-sending a document that moved one event. The same mechanism makes reconnects cheap — resubscribing with the cached sequence transfers nothing when nothing moved. - Yjs catch-up is exact, either way. The CRDT lane never falls back to
snapshot-resending: whether the JSON lane earns an init or a resume, the
Yjs lane answers the subscribe’s state vectors with precisely the missing
diffs. So a reconnect after pure typing is a
doc:resumeplus a small diff — not a re-send of the document. A subscribe without vectors gets full Yjs states ondoc:init(and, when the document holds Y.Docs, a sequence-matched subscribe without vectors falls back to a full init, since Yjs currency is unknowable without them). - Catch-up is bidirectional.
doc:initanddoc:resumecarry the server’s own state vectors; a client holding Yjs content the server lacks — offline edits, or a write-behind crash window — pushes exactly the missing diffs back as an ordinarydoc:update. See Two sync lanes.