JSON Patch RFC issues

datadata uses JSON Patch (RFC 6902) as its structured change format. The RFC was designed for HTTP PATCH — one writer, one round-trip — and applying it in a concurrent, multi-writer engine surfaces real problems. This page collects the ones we’ve hit.

Array operations address positions, not items

The issue. RFC 6902 paths address array elements by index (/items/3). An index is only meaningful against the exact array the patch was computed from: any concurrent insert or remove earlier in the array silently retargets every following operation. The patch still applies — to the wrong elements. The - append token has the mirror problem: two concurrent appends both “succeed” with no way to express ordering intent between them.

And it is worse than a retargeting risk, because a differ working from positions cannot see an insert at all. Inserting {id: "z"} at the head of [{id: "x"}, {id: "y"}] diffs to:

[
  { "op": "replace", "path": "/items/1/id", "value": "x" },
  { "op": "replace", "path": "/items/0/id", "value": "z" },
  { "op": "add", "path": "/items/2", "value": { "id": "y" } }
]

One logical insert became two in-place rewrites of existing elements’ ids plus an append. Nothing here says “insert” — so nothing downstream, from conflict detection to the event log, can recover what the user meant.

datadata’s workaround. Collections that matter aren’t arrays. They’re flat records keyed by stable ids with fractional-index ordering — so “insert between A and B” and “update item X” become single-key operations at disjoint leaf paths, which concurrent edits can’t retarget.

The identity and the ordering have to live somewhere, and JSON Patch has nowhere to put either. So they go in the data model. Fractional indexes work well for us; the price is that anything ordered must be modelled as a record set rather than an array. That’s a standing constraint on schema design, and we lean on it enough that every mutable collection the engine itself maintains (the index, schema migration logs, session changesets) is keyed by id.

Nested objects are spared, but only by an implementation detail: the differ recurses into them and emits ops at leaf paths, so two clients editing sibling keys of the same object touch /cfg/a and /cfg/b rather than both replacing /cfg. The format itself has no deep merge — a hand-authored replace of a container clobbers the whole thing.

test is the only concurrency tool, and it’s blunt

The issue. The RFC’s only precondition mechanism is the test op: assert a value, fail the whole patch otherwise. There is no way to express intent (“increment”, “insert after X”, “set if unset”), so any concurrent touch of a tested value rejects the entire patch — even when the writes were trivially compatible.

datadata’s workaround. Guard modes: auto-generated test ops scoped to the values actually changed (tolerating disjoint edits), or whole-document sequence guards when strictness is wanted. Rejection is treated as a benign, expected signal with a retry path — and the staged session turns it into reviewable three-way conflicts. But the underlying expressiveness gap is the RFC’s.

add has no precondition

The issue. Those auto-generated guards cover replace and remove — operations that have a prior value to assert. An add of a previously-absent member has nothing to test: the RFC offers no way to say “this key must not exist yet”, so the differ emits a bare add with no guard in front of it. Two clients concurrently adding the same new key both pass, and the later write silently overwrites the earlier one. Guarded writes are a complete same-path conflict detector for changes and deletions, but not for additions — and every layer built on guards (including session conflict detection) inherits that hole.

datadata’s position. Open. Candidate fixes: extend the patch profile with an explicit absence test, hash-based value preconditions, or telling strict callers to use the coarse sequence guard (which does catch it). For id-keyed records with generated ids the collision is improbable by construction, which is why this is a sharp edge rather than a daily wound.

Diff-generated patches lose intent

The issue. The array insert above is one instance of a general problem: patches produced by diffing two states (how datadata’s client produces every one of them) are not unique. A move is indistinguishable from a remove+add; a small edit inside a string is a whole-value replace. Whatever the differ guesses becomes the recorded “change”, which degrades both conflict detection (coarser test ops than the actual edit warranted) and the event log’s value as history. The whole-value replace also has a size cost: a one-character edit to a long text field ships and stores the entire new string on every change, so the wire and the event log bloat fast for frequently edited prose.

datadata’s position. Live where it’s tolerable (structured fields are small), escape where it isn’t (Yjs for text, where intent-per-keystroke is exactly what the CRDT captures).

---

If you’ve fought this RFC in your own engine, compare notes with us.