Skip to content
datadata

Two sync lanes

The hybrid model puts two kinds of change in one document: guarded JSON patches for structure, Yjs deltas for rich text. They have different merge semantics on purpose — and it turns out they want different sync mechanics too. datadata syncs them as two independent lanes.

One version each

Section titled “One version each”

A document’s sequence versions the JSON snapshot only: it advances by one per confirmed JSON change, and it is what resume checks, sequence guards, and the staged session’s conflict gate compare against. The Yjs lane has its own per-document version, yjs_sequence, and its own stored log. A Yjs-only update advances yjs_sequence, leaves the current sequence unchanged, appends nothing to the JSON event log, and never rewrites the snapshot. The two versions never read each other.

yjs_sequence never leaves the server. Clients are sent no Yjs version at all, because they have nothing to do with one: the lane is a CRDT, so updates merge in any order. The single place ordering matters — deleting an embedded Y.Doc — is ordered by sequence, since a Y.Doc lives exactly as long as a yjsRef field names it, and removing that reference is a JSON change.

flowchart TB
  accTitle: One update, two independently versioned lanes
  accDescr {
    A single doc:update carries JSON patches and Yjs deltas together. The JSON
    half is validated against guards, advances the document sequence by one,
    and is written to the snapshot and the JSON event log. The Yjs half merges
    as a CRDT in any order, advances a separate yjs_sequence, and is written to
    the Yjs state and its own update log. The two versions never read each
    other, and yjs_sequence never leaves the server. Both halves leave as one
    doc:patch.
  }
  update["one doc:update<br/><small>JSON patch and/or Yjs deltas</small>"]

  subgraph jsonLane["JSON lane — order-sensitive"]
    direction TB
    j1["validate · check guards"]
    j2["sequence + 1"]
    j3[("snapshot + JSON event log")]
    j1 --> j2 --> j3
  end

  subgraph yjsLane["Yjs lane — CRDT, merges in any order"]
    direction TB
    y1["merge delta"]
    y2["yjs_sequence + 1<br/><small>never sent to clients</small>"]
    y3[("Yjs state + update log")]
    y1 --> y2 --> y3
  end

  out["one doc:patch<br/><small>a Yjs-only update repeats sequence unchanged</small>"]

  update --> j1
  update --> y1
  j3 --> out
  y3 --> out

Why bother? A single sequence number shared by both lanes makes the CRDT lane trip machinery built for the other one:

  • a sequence guard fails because someone typed — a concurrent change that, being CRDT, conflicts with nothing;
  • a reconnecting client whose JSON is current gets a full snapshot re-send because only rich text has moved;
  • a session watching for head movement surfaces conflicts for edits that merge unconditionally.

All three are the same category error: describing an order-insensitive lane with the snapshot’s version number. Splitting the versions fixes them structurally instead of special-casing each one.

What rides neither lane

Section titled “What rides neither lane”

Blobs — files referenced from document JSON — are not a third sync lane. Their bytes are immutable, so there is nothing to sync: the handle travels over the JSON lane as an ordinary field, and the bytes move over plain HTTP, exactly once per blob, without a version, a log, or a replay of their own.

State-vector catch-up

Section titled “State-vector catch-up”

The Yjs lane doesn’t catch up with snapshots. A subscribe carries a state vector per embedded Y.Doc — Yjs’s compact summary of “what I already hold” — and the server answers with exact diffs: precisely the missing content, nothing for a Y.Doc that’s current, a tombstone notice for one that’s been deleted. The diffs ride whichever answer the JSON lane earns — doc:init or doc:resume, see the wire protocol — so a reconnect after an hour of pure typing is a tiny resume plus a diff, never a full re-send. A subscribe without vectors falls back to full Yjs states.

Catch-up runs both ways

Section titled “Catch-up runs both ways”

The exchange is symmetric. The server’s answer also carries its own state vectors, and a client holding Yjs content the server lacks pushes exactly the missing diffs back as one ordinary update. Three situations produce that divergence: edits made while disconnected (CRDT deltas merge cleanly later, unlike guarded patches), content held across a soft delete and released by the restore that undid it, and the write-behind window — the bounded period where a streamed Yjs burst has been broadcast but not yet durably stored. If the server crashes inside that window, the next reconnect of any client that saw the broadcasts restores the burst.