Portable event streams

The event log is more than audit. It’s designed so that a folder’s history can be consumed somewhere else — a replica, a pipeline, another system — with guarantees that are deliberately narrow and therefore easy to honor.

The guarantees

Per-document total order. A document’s events form an append-only stream, ordered by its sequence numbers. Replaying the JSON patches in order reconstructs every state the document has ever had.

No cross-document ordering — on purpose. datadata does not guarantee any event ordering between documents. That sounds like a weakness; it’s the property that makes the streams portable. Each document’s stream is self-contained: you can fetch it, ship it, and apply it without knowing anything about any other document in the folder, and replicas can consume documents independently, at different paces, without coordination.

Every event is tagged with its schema version. Each event records the sequence of the governing sys:schema:<type> document in effect when it was written. And since schemas are documents, the schema’s own history is also an event stream. Sync a document’s stream plus its schema document’s stream, and a remote location can interpret every historic version of the document with the exact schema it conformed to — a complete, semantically meaningful picture of the data over time, reconstructed with no access to the origin.

Migration events are ordinary events. When schema evolution rewrites a document on read, the rewrite is appended to the stream as a normal patch event. A consumer replays it like any other change — it doesn’t need to implement the migration engine to follow along.

The folder boundary

The schema version recorded on an event is the schema document’s sequence within its folder — a folder-local coordinate, not a global schema identity. There is no global schema timeline that every folder re-coordinatizes: a schema may evolve through versions A → B → C in source code, but a folder whose first boot upserted C directly has no record of A or B at all — each folder holds only the slice of the evolution it actually lived through, and two folders can hold different, incomplete slices.

Within a folder this is fully consistent: every event’s stamp resolves against that folder’s own schema history, so a folder’s streams are always coherent with each other. Replicating or cloning a whole folder (document streams plus schema-document streams) therefore reconstructs everything faithfully. What doesn’t work is importing a single document into a folder whose schema evolved independently — even a logically identical one — because the sequence spaces don’t align and the target may never have held the versions the document’s history depends on. Cross-folder transfer is a remap problem (match reconstructed schema content, not sequence numbers), and no remap tool exists yet.

What this enables

Read replicas and mirrors — feed a folder’s documents into another system, region, or store by tailing per-document streams.
Analytics and pipelines — events are inspectable JSON patches with schema context, not opaque blobs.
Point-in-time reconstruction — any historic version of any document, with the schema that governed it.

What this is not

This is replication, not offline sync: a one-way replay of server-confirmed history. Authority stays with the folder’s server — a replica can’t write events of its own and reconcile later. The no-offline-writes limitation stands; this page describes how confirmed history travels, not how disconnected writers merge.