Documents & folders

Documents

A document is the unit of data:

docId — a string id, unique within its folder: 1–128 letters, digits, - and _, which fits UUIDs, nanoids and prefixed ids like project_42. The one exclusion is the name of an Object.prototype member (__proto__, constructor, toString, …): ids end up as keys of other documents, such as the folder’s index, and those names can’t safely be one. The same rule covers the ids of embedded Yjs documents, blobs and presence cells, and creating or importing a document under any other id is rejected.
type — a string naming the document type. Every type has a schema document (sys:schema:<type>) that writes are validated against.
sequence — a monotonically increasing number assigned by the server on every accepted change. It orders the document’s log and is what a sequence guard anchors to. It counts JSON-lane changes only — edits to embedded Yjs documents travel on a separate lane and don’t advance it.
generation — an opaque token telling this document apart from anything else that has ever carried its docId. It is minted when the document is created — by the creating client, so its not-yet-confirmed copy already has one — or imported, and never changes afterwards — not on edits, delete or restore. Only purge followed by a new create under the same id produces a new one.
data — the JSON payload. Rich text fields hold references to embedded Yjs documents rather than raw text.

A document can also have a user-facing name — an optional display label that is index-level metadata, not part of data. Set it at create time or change it later with renameDocument; renaming never touches the document’s data, sequence or event log. A name is capped at 1024 characters; an empty or whitespace-only name is treated as unnamed (null). By convention a / in the name nests documents into display-only subfolders — a UI grouping over the flat label, not an engine-level hierarchy (the folder below is still one flat namespace).

Folders

A folder is the unit of synchronization and authority: one server instance owns one folder, orders all of its events, and broadcasts to all of its subscribers. In production a folder maps to one Cloudflare Durable Object. On the Node backend one process and one Postgres database serve many folders, each still with its own server instance and its own ordering.

Clients subscribe per document, not per folder — a client only receives changes for documents it has subscribed to.

Schemas are per folder, too: a document type is defined by the sys:schema:<type> document living inside the folder, and a document’s schema_sequence is a folder-local coordinate. Two folders can carry the same type name at independently evolved schemas.

Admission is folder-granular — a connection presents a host-minted token to enter the folder — but inside it the engine authorizes every write and filters every read across three declarative grains: folder roles, access rules per document type, and per-document entries in the folder’s sys:access document — with hard scope caps on the principal above them all. See Authorization.

Deleting documents

Deletion is soft: a deleted document leaves the index and every read path but is listed in sys:trash, with its row, event log, and embedded Yjs state retained so it can be restored. Delete is a lifecycle change, recorded in the folder’s membership log rather than the document’s own event log — restoring a document returns it at the exact sequence it left, history intact.

Deletion is also observable. Live subscribers are pushed a doc:deleted event the moment a delete commits (an open editor can show “this document was deleted” instead of silently going stale), and subscribing to an already-deleted document answers doc:deleted too — deliberately distinct from “never existed”. Subscriptions survive the deletion, so a restore pushes the document straight back to everyone who was watching.

A deleted document still owns its id: creating a new document under a deleted docId is rejected with a distinct deleted error (restore it, or pick a new id) rather than the benign already-exists signal.

Deletes are optimistic and stackable: the document reads as gone locally the moment you call deleteDocument, and you can delete a document whose own create is still in flight — the pair goes to the server in author order rather than being cancelled against each other locally, so both writes report their real outcome. If the create is rejected, the trailing delete is dropped along with it and the failure surfaces once. If the create lost the id to another client’s create, the delete is refused too — it names the document this client created, never the one that won the id.

Purge is the second stage — hard deletion, for storage reclamation and right-to-erasure. It takes a deleted document, destroys its event logs, Yjs state and snapshot — and, once a later sweep finds them unreferenced for longer than the retention period, the bytes of any blob that no other document still references — and releases its id: afterwards the id behaves exactly like one that never existed — a subscribe answers notfound, restore has nothing to resurface, and a create under it simply starts a fresh document: a new generation, back at sequence 1. A client still holding a copy of the purged document — offline through the whole purge, say — is not fooled by a matching sequence: on reconnect its copy (embedded Yjs content included) is replaced by the new document, and edits, deletes and renames it made to the old one are rejected rather than applied. So is a restore it queued for the old one, even once the new document has been deleted too: a restore names the generation the client saw deleted, so it never brings back a different document. That includes the purged document’s own create: replayed by a client that never saw it acknowledged, it is rejected rather than bringing the purged generation back. That release is what makes purge safe for deterministic ids (one document per domain entity, a settings singleton): an id is never permanently burned. One erasure boundary to know: purge destroys a document’s content, never its identifier — the original id is kept in a server-side audit record, so don’t encode data in a docId that would itself need erasing.

Purge has two entry points, and both are batch-shaped and all-or-nothing: one storage transaction, one sys:trash sequence step for the whole batch, and a failing member anywhere destroys nothing. Clients purge over the wire with purgeDocuments (the “delete forever” button) — but only for document types whose schema explicitly opts in with an access.purge rule; the default is nobody, admins included (see Authorization). It is deliberately never optimistic and never queued offline: irreversible destruction doesn’t sit in a replay buffer. On the server, purgeDocuments (system authority) empties specific tombstones, and purgeDeletedDocuments sweeps everything deleted before a cutoff, oldest first and optionally bounded — the mechanism behind “trash empties after 30 days”. The retention window and the trigger (a Durable Object alarm, a cron, opportunistically on wake) belong to the application; datadata deliberately owns no scheduler.

One carve-out: sys: documents — including schema documents — can’t be deleted, and therefore can’t be purged; see Limitations.

System documents

The engine maintains a handful of documents itself — the folder index, the trash, schemas, staged-work views, presence — each readable and subscribable like any other document. System documents enumerates them.