Skip to content

Open questions

These aren’t missing features — they’re design problems where we don’t yet trust any answer, including our own. If you’ve faced one of these in your engine, we’d like to compare notes.

Compaction without losing history’s value

Section titled “Compaction without losing history’s value”

The event log is an asset — audit, provenance, future undo/diff features — until it’s a liability. What’s the right compaction contract? Snapshot + truncate loses blame; rolling windows lose old provenance; never compacting loses the disk. Yjs has the same question with different math. And staged sessions have quietly come to depend on the log: a cold client reconstructs a stage’s base for the three-way preview by replaying it, so truncation would degrade those previews (conflict detection and commit never need the log).

For the CRDT half, the answer turned out to be structural: the Yjs lane self-heals — a reconnecting client pushes back exactly the content the server lacks, because CRDT deltas merge cleanly however late they arrive. The JSON half is answered too: writes buffered while disconnected replay on reconnect, last-writer-wins (a guarded write is rejected if the server moved under it), re-sends de-duplicated by client event id and bounded by the replay horizon (full replay rules) — and with the opt-in persisted write queue that buffer now survives a page reload, closing the durable-queue half of this question. What stays open is the semantics of long offline periods: a persisted unguarded queue replayed days later lands silently last-writer-wins (protocol-legal, humanly surprising), and a guarded one mostly means rejections, which pushes everything into session-style three-way resolution. Is offline-then-review (a reconnect producing a changeset to review rather than silent writes) the right model for longer offline periods on server-authoritative engines? We suspect yes — staged sessions’ changesets are already server-durable, they compose with the persisted queue (work staged offline survives a reload and resumes; the flow is integration-tested), and commit is deliberately an online act — one that is now also crash-safe: a durable commit intent lets a commit that died with its page finish its bookkeeping on resume instead of double-applying on retry. What hasn’t been proven is the experience: whether a changeset review on reconnect is what people actually want after days offline, or an interruption they’ll resent.

A second sub-question: who arbitrates while offline? Tabs in one browser profile share their offline work over a cache bus, but each tab is still a full client with its own optimistic queue, so two tabs editing the same field keep showing their own value until a reconnect lets the server settle the order (offline and multi-tab). The end-state we know of is a single arbiter per browser profile — a leader tab or SharedWorker owning one queue for all tabs — which would make renders converge offline, at the cost of a single point of failure and a large architecture change for a divergence that is render-only: nothing either tab journals or replays differs. We haven’t taken that trade. What would move it is evidence that people hit same-field divergence across their own tabs in practice, rather than the disjoint edits that already render the same everywhere.

How far should declarative authorization stretch?

Section titled “How far should declarative authorization stretch?”

Authorization moved into the engine (July 2026): declarative folder-role + docType rules in schema documents, read filtering, scope caps — see Authorization. What stays open is the boundary. Per-document and per-field rules currently live in the app policy callbacks — synchronous code the capability prediction deliberately doesn’t consult. Making them declarative too would keep prediction accurate for them, but each step re-runs the original trade-off: declarative-in-schema-documents is elegant and a big correctness surface; app callbacks are explicit and push complexity onto every app.

The three-way preview provides the data, but what do users actually want — field-level pick-and-choose? Restage-from-head and re-apply? Let the agent propose the merge and review that? The apps are the laboratory; no pattern has won yet.

A concrete sub-question: partial accept. Per-document accept already falls out of the model (each stage commits independently), but accepting some staged operations within one document while keeping the rest staged needs a selective drain plus a rebase of the remainder. Per-item triage of AI suggestions is proven UX elsewhere — is it worth the machinery, or is whole-stage commit the right simplicity?

A document’s event stream replays anywhere, but its schema version stamps are folder-local coordinates — so import only works into a fresh or cloned folder that carries the schema document’s history. Importing into a folder whose schema evolved independently is a remap problem: translate sequence stamps by matching reconstructed schema content. Nothing extra needs to be stored to make that possible — but the remap tool doesn’t exist, and whether it’s ever needed (versus “clone whole folders” being the only real use case) is open.

The projection layer reshapes a fully synced document into a more ergonomic view — picks, renames, sorts, groupings. A subset of those operations inverts cleanly, which makes writable lenses possible: edit the view, and the edit translates back into path-disjoint physical operations that merge cleanly. The mechanics exist at a very alpha level — but we genuinely don’t know whether they’re worth the effort, what the right shape is, or whether projections should simply stay read-only and writes always go through the physical document. If you’ve built (or abandoned) a lens layer over a sync engine, this is the question we’d most like to compare notes on.

A changeset occupies one field of its host document, and every session opened over that field shares it. What’s missing is named changesets: addressing several by id, with an API to create and drop them. Parallel agent proposals and draft-vs-review lanes want exactly that, and it is repeatedly tempting and repeatedly deferred — is the added model complexity worth it, or is “one changeset, host documents are cheap” the right discipline?

Should anything speak the protocol but TypeScript?

Section titled “Should anything speak the protocol but TypeScript?”

datadata is TypeScript on both sides of the wire, and nobody has yet asked for otherwise — which is a fact about our sample, not evidence of anything. The design problem hiding behind the missing feature is what a second language would cost: the wire protocol is documented today but not promised, and it evolves whenever the two lanes need it to. A client we don’t compile is a client we can’t migrate in the same commit, so supporting one means versioning the protocol as a public contract and holding it stable for implementers we can’t see. That is the trade — not “write a Rust client” but “freeze the wire protocol.” Whether the reach is worth the loss of velocity is the part we don’t know.

How far does “schemas as documents” stretch?

Section titled “How far does “schemas as documents” stretch?”

Migration happens per document on its next read, with the result written back — so cold documents accumulate pending migrations until someone loads them. “Is the migration done?” now has a clean answer: a budgeted validation sweep forces discovery over cold documents, and when it finishes, the invalid-document enumeration is complete for the current schema. What remains open is the half before and the half after: should a schema write warn when it will strand existing documents as flagged-invalid (the sweep machinery could power a dry-run preview, but nothing does yet), and what do good repair workflows for flagged documents look like? The enumeration carries per-field violation detail for exactly that tool, which nobody has built. Nobody has hit these walls yet, which is not the same as the walls not existing.