Skip to content
datadata

@repo/datadata/projection

Projections: declarative read models over a document's records (filter, sort, group, resolve), and which projected fields can be written back.

Functions

aggregate

aggregate: <const As extends string>(as: As, fn?: "count") => {
    op: "aggregate";
    as: As;
    fn: "count";
}

Replaces the rows (or each group's rows) with a single row holding the row count in field as. Read-only.

analyzeWritability

export declare function analyzeWritability(projection: Projection): WritabilityMap;

Computes which fields and axes of a projection's view can be written back, from the pipeline alone (no document needed). Use it to enable or disable editing affordances.

applyPhysicalOps

export declare function applyPhysicalOps<D extends object>(data: D, ops: readonly PhysicalOp<NoInfer<D>>[]): void;

Apply a sequence of physical ops to a document in place. Designed to run inside a single updateDocument draft callback: the mutated draft is then strict-validated (shape + references) as one atomic write, so a translation that would orphan a reference is rejected as a whole.

Generic over the document type so callers pass their typed draft (e.g. a ChatConversation) directly — no as Record<string, unknown> at the call site. Mutation is in place, so T is preserved; the one cast the path walk needs is localized here.

D is inferred from data alone (NoInfer on the ops), so the ops must carry the *same* document brand translate stamped them with — applying ops derived from one document to another is a compile error.

defineProjection

export declare function defineProjection<const P extends Projection>(projection: P): P;

Defines a projection, returning it unchanged with its literal source, pipeline and params kept in the type, so InferProjection and ProjectionParams can read them.

Throws when a token path (source, or a sortBy, resolve, nestBy or memberOf path) uses a $<name> parameter that params does not declare, so a typo like $tread fails at definition rather than reading as a missing path. $key is always available and is not declared.

filter

filter: <const F extends string>(field: F, equals: unknown) => {
    op: "filter";
    field: F;
    equals: unknown;
}

Keeps only rows whose field strictly equals equals. Field edits on the visible rows still translate; a create through a filtered view does not.

get

export declare function get<const P extends Projection, D = unknown>(projection: P, data: D, ctx: ProjectionParams<P>): InferProjection<P, D>;

Evaluates a projection against a physical document and returns its logical view.

A pipeline without a grouping primitive renders to an ordered array of LogicalRows; one with groupBy or nestBy renders to a record of such arrays keyed by group. A missing or non-object source record yields an empty view.

The return type is InferProjection, so a well-typed data yields a precisely typed view; a loosely typed data, or a projection widened to Projection, yields a looser type rather than an error.

Throws on a malformed pipeline: a rename onto an existing field, a value field named id, or a grouping primitive following another grouping.

Parameters

  • projection

    The projection to evaluate, usually from defineProjection.

  • data

    The physical document the projection reads.

  • ctx

    A value for each of the projection's declared params (null when absent, e.g. no current user).

groupBy

groupBy: <const F extends string>(field: F) => {
    op: "groupBy";
    field: F;
}

Partitions rows into named groups by a value field, which is removed from each row (the group key carries it). Moving a row between groups writes that field. Cannot follow another grouping.

keys

keys: <const F extends string>(field: F) => {
    op: "keys";
    field: F;
}

Renders a record-set value field (a record<id, …> used as a set) as the array of its keys. Read-only as a value; its membership is edited with addToSet / removeFromSet view ops.

memberOf

memberOf: <const S extends string>(set: S) => {
    op: "memberOf";
    set: S;
}

Keeps only rows whose key is present in the membership record at token path set (e.g. a thread's placement record). Paired with sortBy over the same record it yields the ordered members of one group. Writable only through the unlink view op.

nestBy

nestBy: <const M extends string>(membership: M) => {
    op: "nestBy";
    membership: M;
}

Groups rows by an external membership relation: membership is a token path to a record<groupId, record<entityId, …>>, and each entity appears under every group whose set contains it. Entities in no group are left out of the view. Edited with addToGroup / removeFromGroup view ops. Cannot follow another grouping.

pathsOverlap

export declare function pathsOverlap(a: readonly string[], b: readonly string[]): boolean;

Whether two physical paths overlap: one is equal to, or a prefix of, the other. A "*" segment matches any key at its position, so a wildcard path is treated as overlapping.

physicalOpsDisjoint

export declare function physicalOpsDisjoint(ops: readonly PhysicalOp[]): boolean;

Whether no two of the ops' paths overlap (see pathsOverlap). The ops one translate call emits are disjoint, which is what lets concurrent view edits merge.

pick

pick: <const F extends readonly string[]>(fields: F) => {
    op: "pick";
    fields: F;
}

Narrows each row to the named value fields. Writing a kept field writes the same physical field.

primitiveInverts

export declare function primitiveInverts(primitive: Primitive): boolean;

Whether a primitive inverts, i.e. a logical write through it translates back to a physical op: true for pick, rename, sortBy and groupBy, false for the rest.

This classifies the primitive alone. Whether a given field or axis of a view is writable is decided per target by analyzeWritability: a non-inverting filter, for one, still leaves the visible rows' fields writable.

rename

rename: <const From extends string, const To extends string>(from: From, to: To) => {
    op: "rename";
    from: From;
    to: To;
}

Renames a value field from from to to; a write to the logical name lands on the physical one. Evaluation throws when to already exists on a row.

resolve

resolve: <const F extends string, const Fr extends string>(field: F, from: Fr) => {
    op: "resolve";
    field: F;
    from: Fr;
}

Replaces a value field holding an id with the object of that id in the record at token path from, or null when there is none. Read-only: edit the resolved object through its own projection.

resolveTokenPath

export declare function resolveTokenPath(root: unknown, path: string, ctx: ProjectionContext, key: string): unknown;

Reads the value at a dot-separated token path from root. A leading $ segment marks the root, $key stands for key (the current row's key), any other $<name> is looked up in ctx, and other segments are literal keys: $.taskOrder.$me.$key.index with ctx = { me } reads taskOrder[me][key].index.

Returns

The value, or undefined when a segment is missing, passes through a non-object, or is a parameter whose value is null or absent.

sortBy

sortBy: <const P extends string>(path: P, direction?: "asc" | "desc") => {
    op: "sortBy";
    path: P;
    direction: "asc" | "desc";
}

Orders rows by the value at a token path (see resolveTokenPath), ascending by default. Rows with no value sort after the others when ascending (first when descending). The path may reach outside the source record, e.g. a per-user placement $.taskOrder.$me.$key.index; a reorder writes the value it points at.

tokenPathSegments

export declare function tokenPathSegments(path: string, ctx: ProjectionContext, key: string): string[] | null;

Substitutes a token path's parameter and $key tokens, with the same rules as resolveTokenPath, and returns the physical key path it names (without the root $).

Returns

The path segments, or null when a parameter token has no value, so there is no writable path.

translate

export declare function translate<const P extends Projection, D = unknown>(projection: P, viewOp: ViewOp, physical: D, ctx: ProjectionParams<P>, schema?: DocumentSchema): PhysicalOp<D>[];

Translate one logical ViewOp into the disjoint physical ops that realize it. Reads physical to resolve neighbour indices and current placements; ctx.me supplies the $me token. Throws if the op targets a non-writable field or a placement write is attempted without a current user.

The emitted ops are intentionally small and path-disjoint — a cross-column move writes tasks[id].status and taskOrder[me][id].index, never the whole card — so a concurrent title edit on tasks[id].title merges without conflict.

remove and unlink emit only the *primary* delete (the entity, or the membership key). Cascade/orphan cleanup is no longer derived here: it's a write-boundary concern handled by repairReferences against the schema's M4 block when the mutated draft is validated, so it runs for *every* write (lens-driven or raw updateDocument callback), not just a translate edit.

Interfaces

Position

export interface Position

A slot in an ordered list, given by the ids of its neighbours in visual order (whatever the sortBy direction).

nextId?: string | null;

The entity shown just after the slot; null or absent means the end of the list.

prevId?: string | null;

The entity shown just before the slot; null or absent means the start of the list.

Projection

export interface Projection

A declarative read model over a normalized physical record: where to read, and the pipeline of Primitives that reshapes its rows into a logical view. Define one with defineProjection and evaluate it with get.

params?: readonly string[];

Names of the view parameters the token paths reference as $<name>, e.g. ["me"] for a per-user order. Each needs a value in the ProjectionContext at evaluation.

pipeline: readonly Primitive[];

The primitives applied to the source record's rows, in order.

source: string;

Token path to the record the projection reads, e.g. $.tasks (see resolveTokenPath).

Writability

export interface Writability

Whether a target of a logical view (a field or an axis) can be written back to the physical document.

reason?: string;

Why the target is read-only, for tooling or a disabled control; absent on writable targets.

set?: boolean;

True for a record-set field rendered by keys: not writable as a whole value, but its membership is edited with addToSet / removeFromSet.

writable: boolean;

Whether a write translates back: for a field, a whole-value setField; for an axis, a move (or, for a nestBy axis, addToGroup / removeFromGroup).

WritabilityMap

export interface WritabilityMap

The static writability of a projection's logical view, derived from its pipeline alone (see analyzeWritability).

pick and rename keep value fields writable; groupBy, nestBy and sortBy introduce writable axes; aggregate, resolve and keys mark their output field read-only; filter and memberOf do not block edits to the visible rows.

fields: Record<string, Writability>;

A verdict per field the pipeline names, keyed by logical field name. Only fields named by a primitive appear; without a pick, the source's other fields are not listed.

group?: {
        field: string;
        membership?: boolean;
    } & Writability;

The grouping axis, absent without a grouping primitive. field is the group-key field as named at the groupBy step (or the membership path of a nestBy); membership: true marks a nestBy axis, edited with addToGroup / removeFromGroup rather than a single-group move.

order?: Writability;

The ordering axis, absent without a sortBy; reordering writes the placement it sorts by.

Types

InferProjection

export type InferProjection<P extends Projection, Physical> = IsTuple<P["pipeline"]> extends true ? FoldPipeline<P["pipeline"], EntityKeyOf<Physical, P["source"]>, EntityOf<Physical, P["source"]>, Physical, false> : Rendered<EntityKeyOf<Physical, P["source"]>, EntityOf<Physical, P["source"]>>[] | Record<string, Rendered<EntityKeyOf<Physical, P["source"]>, EntityOf<Physical, P["source"]>>[]>;

The logical-view type get(projection, doc, ctx) produces for a Projection literal P over a physical document type Physical. Pass typeof projection (defined via defineProjection) and the schema's InferJsonSchema type:

const board = defineProjection({
  source: "$.tasks",
  pipeline: [groupBy("status"), sortBy("$.taskOrder.$me.$key.index"), pick(["title"])],
});
type Board = InferProjection<typeof board, MyDoc>;
//   → Record<string, { id: string; title: string }[]>

LogicalRow

export type LogicalRow = Record<string, unknown> & {
    id: string;
};

One entity in a projection's logical view: its reshaped value fields plus id, the entity's key in the source record.

PhysicalOp

export type PhysicalOp<D = unknown> = ({
    op: "set";
    path: readonly string[];
    value: unknown;
} | {
    op: "delete";
    path: readonly string[];
}) & DocBrand<D>;

A set or delete at an explicit key path of the physical document, produced by translate and applied with applyPhysicalOps.

A delete path may contain "*" wildcards, which delete the key from every record at that depth (used to remove an entity's placements across all users); a set path is always concrete. D is a type-only brand naming the document the op was derived from, so ops cannot be applied to a document of a different type.

Primitive

export type Primitive = {
    op: "pick";
    fields: readonly string[];
} | {
    op: "rename";
    from: string;
    to: string;
} | {
    op: "sortBy";
    path: string;
    direction?: "asc" | "desc";
} | {
    op: "groupBy";
    field: string;
} | {
    op: "filter";
    field: string;
    equals: unknown;
} | {
    op: "memberOf";
    set: string;
} | {
    op: "aggregate";
    as: string;
    fn: "count";
} | {
    op: "resolve";
    field: string;
    from: string;
} | {
    op: "keys";
    field: string;
} | {
    op: "nestBy";
    membership: string;
};

One step of a Projection pipeline. A projection reads a normalized physical record (e.g. $.tasks) and runs its primitives in order over the record's rows to produce a logical view.

pick, rename, sortBy and groupBy invert: a logical write through them maps back to a physical write. filter, memberOf, aggregate, resolve and keys are read-only in themselves, and nestBy is written through membership toggles (see ViewOp). Each row is an entity keyed by its id in the source record; after groupBy or nestBy the rows are partitioned into named groups and later primitives apply per group. Build primitives with the constructors (pick, sortBy, ...), which keep their literal arguments in the type.

ProjectionContext

export type ProjectionContext = Record<string, string | null>;

The parameter values a projection is evaluated in, substituted for the matching $<name> tokens in its token paths.

A null value marks an absent parameter (e.g. no current user): a path through it resolves to undefined on read and cannot be written. $key is not a parameter; it is bound to the current row's key during evaluation.

ProjectionParams

export type ProjectionParams<P extends Projection> = [ParamNames<P>] extends [never] ? ProjectionContext : {
    [Name in ParamNames<P>]: string | null;
};

The context get requires for projection P: a string | null value for each name in its literal params. A projection without declared params, or one widened to Projection, takes the loose ProjectionContext.

ViewOp

export type ViewOp = {
    op: "setField";
    id: string;
    field: string;
    value: unknown;
} | {
    op: "move";
    id: string;
    toGroup?: string;
    position?: Position;
} | {
    op: "create";
    id: string;
    group: string;
    fields: Record<string, unknown>;
    position?: Position;
} | {
    op: "remove";
    id: string;
} | {
    op: "unlink";
    id: string;
} | {
    op: "addToGroup";
    id: string;
    group: string;
} | {
    op: "removeFromGroup";
    id: string;
    group: string;
} | {
    op: "addToSet";
    id: string;
    field: string;
    member: string;
} | {
    op: "removeFromSet";
    id: string;
    field: string;
    member: string;
};

An edit expressed against a projection's logical view: set a field, move, create, remove or unlink an entity, or toggle group or set membership. translate turns it into path-disjoint PhysicalOps, so concurrent edits to the same entity (a move and a title edit) merge cleanly.