Skip to content

Commit b2d32ce

Browse files
krisnyeclaude
andauthored
feat(data): Schema.createCoerceFunction + convertTypedBuffer (#183)
* feat(data): Schema.createCoerceFunction + convertTypedBuffer A schema→schema value-coercion factory and a TypedBuffer converter built on it, the foundation for automated persisted-data migration. Schema.createCoerceFunction(input, output) analyzes the pair ONCE and returns a reusable per-value converter, or null when no automatic conversion exists (the null return is the feasibility check — no separate predicate). Supported: numbers (clamp to the output's min/max; precision/width loss applied by the destination buffer), boolean/string identity, enum→enum when the input values are a subset, →const, object→object (reorder / drop / fill new fields from their default / recurse), and array→array (element-wise, extend a fixed vector from the item default). Cross-kind changes return null. convertTypedBuffer(source, targetSchema, capacity?) compiles one coerce fn and maps it over the source's elements into a new buffer of the target schema; throws when not convertible. Source values are shared by reference (ECS values are never mutated in place); newly-introduced field defaults are cloned per element. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(data): in-place coerce for archetype columns and store components Follow-ups on the coerce foundation: - Objects: only a numeric STRUCT (all fields packed) requires a default for a new field; a plain object may lack a non-required property, which is now simply omitted rather than blocking the conversion. Red/green tests for both. - convertTypedBuffer gains a `count` param (default: full range) so a table can convert only its live rows, leaving unused capacity at the new default — avoids running an array buffer's undefined tail through the coercer. - coerceArchetypeColumn(archetype, component, targetSchema): converts one column in place, preserving rows and rebuilding the archetype's baked insert (via fromData) so later inserts write through the new column. - coerceStoreComponent(store, component, targetSchema): converts that component in every archetype that carries it and adopts the new schema, so the store's schema is migrated in place. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(data)!: convertTypedBuffer takes a named options object Replace the two similar trailing number params (capacity, count) with `{ capacity?, count? }`. Breaking for direct callers, but the buffer converter is rarely called directly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(data)!: convertTypedBuffer takes a single named-arguments object Fold source/targetSchema/capacity/count into one options object so the call is fully named and never exceeds two ordinal arguments. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent cc5dd84 commit b2d32ce

11 files changed

Lines changed: 744 additions & 0 deletions
Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
// © 2026 Adobe. MIT License. See /LICENSE for details.
2+
3+
import type { Schema } from "../../schema/index.js";
4+
import { TypedBuffer, convertTypedBuffer } from "../../typed-buffer/index.js";
5+
import type { Archetype } from "./archetype.js";
6+
7+
/**
8+
* Convert one component column of `archetype` to `targetSchema` IN PLACE,
9+
* preserving the archetype's live rows. Returns `false` (a no-op) when the
10+
* component is absent from this archetype.
11+
*
12+
* An archetype (a table) bakes its column references into a specialized `insert`;
13+
* swapping a column therefore goes through `archetype.fromData`, which replaces
14+
* the columns AND rebuilds that baked insert so later inserts write through the
15+
* converted column. Only the live `rowCount` rows are converted — unused capacity
16+
* stays at the new column's default.
17+
*
18+
* Throws (via {@link convertTypedBuffer}) if no automatic conversion exists.
19+
*/
20+
export function coerceArchetypeColumn(
21+
archetype: Archetype<any>,
22+
component: string,
23+
targetSchema: Schema,
24+
): boolean {
25+
const columns = archetype.columns as Record<string, TypedBuffer<unknown>>;
26+
const existing = columns[component];
27+
if (existing === undefined) return false;
28+
const converted = convertTypedBuffer({ source: existing, targetSchema, capacity: archetype.rowCapacity, count: archetype.rowCount });
29+
archetype.fromData({
30+
columns: { ...columns, [component]: converted },
31+
rowCount: archetype.rowCount,
32+
rowCapacity: archetype.rowCapacity,
33+
});
34+
return true;
35+
}

packages/data/src/ecs/archetype/index.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,4 @@
22
export * from "./archetype.js";
33
export * from "./create-archetype.js";
44
export * from "./delete-row.js";
5+
export * from "./coerce-archetype-column.js";
Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
// © 2026 Adobe. MIT License. See /LICENSE for details.
2+
3+
import { describe, it, expect } from "vitest";
4+
import { Store } from "./index.js";
5+
import { coerceStoreComponent } from "./coerce-store-component.js";
6+
import { coerceArchetypeColumn, type Archetype } from "../archetype/index.js";
7+
import type { Schema } from "../../schema/index.js";
8+
9+
const f32 = { type: "number", precision: 1 } as const satisfies Schema;
10+
const num = { type: "number" } as const satisfies Schema;
11+
const vec2 = { type: "object", properties: { x: f32, y: f32 } } as const satisfies Schema;
12+
const vec3 = { type: "object", properties: { x: f32, y: f32, z: { type: "number", precision: 1, default: 7 } } } as const satisfies Schema;
13+
const capped = { type: "integer", minimum: 0, maximum: 100 } as const satisfies Schema;
14+
15+
describe("coerceArchetypeColumn — in-place column conversion on a table", () => {
16+
it("adds a struct field to every live row and keeps insert working", () => {
17+
const store = Store.create({ components: { pos: vec2 }, resources: {}, archetypes: { P: ["pos"] } });
18+
const P = store.archetypes.P as any;
19+
const e0 = P.insert({ pos: { x: 1, y: 2 } });
20+
const e1 = P.insert({ pos: { x: 3, y: 4 } });
21+
22+
const arch = store.queryArchetypes(["pos"])[0] as unknown as Archetype<any>;
23+
expect(coerceArchetypeColumn(arch, "pos", vec3)).toBe(true);
24+
25+
// Existing rows carry the new field's default.
26+
expect(store.read(e0)).toEqual({ pos: { x: 1, y: 2, z: 7 } });
27+
expect(store.read(e1)).toEqual({ pos: { x: 3, y: 4, z: 7 } });
28+
// The column now reports the new schema.
29+
expect(arch.columns.pos.schema).toBe(vec3);
30+
31+
// Insert AFTER the swap writes through the rebuilt (converted) column.
32+
const e2 = P.insert({ pos: { x: 5, y: 6, z: 8 } });
33+
expect(store.read(e2)).toEqual({ pos: { x: 5, y: 6, z: 8 } });
34+
});
35+
36+
it("is a no-op returning false when the component is absent", () => {
37+
const store = Store.create({ components: { pos: vec2 }, resources: {}, archetypes: { P: ["pos"] } });
38+
const p = store.queryArchetypes(["pos"])[0] as unknown as Archetype<any>;
39+
expect(coerceArchetypeColumn(p, "missing", num)).toBe(false);
40+
});
41+
});
42+
43+
describe("coerceStoreComponent — in-place schema change across the whole store", () => {
44+
it("converts the component in every archetype and adopts the new schema", () => {
45+
const store = Store.create({
46+
components: { hp: num, mana: num },
47+
resources: {},
48+
archetypes: { A: ["hp"], AB: ["hp", "mana"] },
49+
});
50+
const A = store.archetypes.A as any;
51+
const AB = store.archetypes.AB as any;
52+
const e0 = A.insert({ hp: 50 });
53+
const e1 = A.insert({ hp: 70000 });
54+
const e2 = AB.insert({ hp: 90000, mana: 5 });
55+
56+
coerceStoreComponent(store, "hp", capped);
57+
58+
// Every archetype's existing rows are clamped into the new range.
59+
expect(store.read(e0)).toEqual({ hp: 50 });
60+
expect(store.read(e1)).toEqual({ hp: 100 });
61+
expect(store.read(e2)).toEqual({ hp: 100, mana: 5 });
62+
63+
// The store adopts the new schema; the untouched component is unchanged.
64+
expect((store.componentSchemas as Record<string, Schema>).hp).toBe(capped);
65+
expect((store.componentSchemas as Record<string, Schema>).mana).toBe(num);
66+
67+
// Inserts now go through the new (integer-backed) storage: a fractional
68+
// value truncates, proving the column was actually re-typed.
69+
const e3 = A.insert({ hp: 42.9 });
70+
expect(store.read(e3)).toEqual({ hp: 42 });
71+
});
72+
73+
it("throws when a column cannot be automatically converted", () => {
74+
const store = Store.create({ components: { hp: num }, resources: {}, archetypes: { A: ["hp"] } });
75+
(store.archetypes.A as any).insert({ hp: 1 });
76+
expect(() => coerceStoreComponent(store, "hp", vec2)).toThrow(/No automatic TypedBuffer conversion/);
77+
});
78+
});
Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
// © 2026 Adobe. MIT License. See /LICENSE for details.
2+
3+
import type { Schema } from "../../schema/index.js";
4+
import type { Archetype, ReadonlyArchetype } from "../archetype/index.js";
5+
import { coerceArchetypeColumn } from "../archetype/index.js";
6+
7+
// The minimal store surface this helper needs. Declared structurally (with `any`
8+
// in the query signature) so a concretely-typed store assigns without the
9+
// generic-method variance that blocks assignment to `ReadonlyStore<any,…>`.
10+
interface CoercibleStore {
11+
queryArchetypes(include: readonly any[], options?: any): readonly ReadonlyArchetype<any>[];
12+
readonly componentSchemas: object;
13+
}
14+
15+
/**
16+
* Change a component's schema across the whole store IN PLACE: convert that
17+
* component's column in every archetype that carries it, then adopt
18+
* `targetSchema` as the component's declared schema so subsequently-created
19+
* archetypes back it the new way.
20+
*
21+
* This is the store-level counterpart to {@link coerceArchetypeColumn}, and the
22+
* building block for migrating a persisted schema forward without a full reload.
23+
* Throws if any archetype's column has no automatic conversion to `targetSchema`.
24+
*/
25+
export function coerceStoreComponent(
26+
store: CoercibleStore,
27+
component: string,
28+
targetSchema: Schema,
29+
): void {
30+
// queryArchetypes returns ReadonlyArchetype, but these are the live Archetype
31+
// instances that carry the fromData/insert surface the column swap needs.
32+
for (const archetype of store.queryArchetypes([component])) {
33+
coerceArchetypeColumn(archetype as unknown as Archetype<any>, component, targetSchema);
34+
}
35+
(store.componentSchemas as Record<string, Schema>)[component] = targetSchema;
36+
}

packages/data/src/ecs/store/index.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
// © 2026 Adobe. MIT License. See /LICENSE for details.
22
export * from "./store.js";
3+
export * from "./coerce-store-component.js";
34
export * from "./archetype-components.js";
45
export type { ArchetypeSchema, ArchetypeRowOf, ArchetypeHandleOf } from "./archetype-row.js";
56
export type { EntityReadValues, EntityUpdateValues, ArchetypeQueryOptions } from "./core/core.js";
Lines changed: 192 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,192 @@
1+
// © 2026 Adobe. MIT License. See /LICENSE for details.
2+
3+
import { describe, it, expect } from "vitest";
4+
import { createCoerceFunction } from "./create-coerce-function.js";
5+
import { Schema } from "./index.js";
6+
import type { Schema as SchemaType } from "./schema.js";
7+
8+
const num = { type: "number" } as const satisfies SchemaType;
9+
const f32 = { type: "number", precision: 1 } as const satisfies SchemaType;
10+
const bool = { type: "boolean" } as const satisfies SchemaType;
11+
const str = { type: "string" } as const satisfies SchemaType;
12+
13+
// createCoerceFunction never returns null in the "possible" cases — narrow it.
14+
const coercer = (input: SchemaType, output: SchemaType) => {
15+
const fn = createCoerceFunction(input, output);
16+
expect(fn).not.toBeNull();
17+
return fn!;
18+
};
19+
20+
describe("createCoerceFunction — numbers", () => {
21+
it("number → number with no bounds is identity (F64→F32 precision loss is the buffer's job)", () => {
22+
const fn = coercer(num, f32);
23+
expect(fn(3.14159)).toBe(3.14159);
24+
expect(fn(-100)).toBe(-100);
25+
});
26+
27+
it("clamps to the output maximum / minimum when declared", () => {
28+
expect(coercer(num, { type: "number", maximum: 100 })(150)).toBe(100);
29+
expect(coercer(num, { type: "number", maximum: 100 })(50)).toBe(50);
30+
expect(coercer(num, { type: "number", minimum: 0 })(-5)).toBe(0);
31+
const both = coercer(num, { type: "number", minimum: 0, maximum: 10 });
32+
expect([both(-1), both(5), both(99)]).toEqual([0, 5, 10]);
33+
});
34+
35+
it("caps a wide integer down to a narrow integer range (the U32→'U16' case)", () => {
36+
const fn = coercer({ type: "integer" }, { type: "integer", minimum: 0, maximum: 65535 });
37+
expect(fn(70000)).toBe(65535);
38+
expect(fn(-5)).toBe(0);
39+
expect(fn(40000)).toBe(40000);
40+
});
41+
});
42+
43+
describe("createCoerceFunction — scalars & enums", () => {
44+
it("boolean → boolean and string → string are identity", () => {
45+
expect(coercer(bool, bool)(true)).toBe(true);
46+
expect(coercer(str, str)("hi")).toBe("hi");
47+
});
48+
49+
it("enum → enum only when every input value is an output value", () => {
50+
const widen = coercer({ enum: ["a", "b"] }, { enum: ["a", "b", "c"] });
51+
expect(widen("a")).toBe("a");
52+
expect(createCoerceFunction({ enum: ["a", "z"] }, { enum: ["a", "b"] })).toBeNull();
53+
});
54+
55+
it("→ const collapses any input to the constant", () => {
56+
const fn = coercer(num, { const: 42 });
57+
expect(fn(1)).toBe(42);
58+
expect(fn(999)).toBe(42);
59+
});
60+
});
61+
62+
describe("createCoerceFunction — objects", () => {
63+
it("reorders fields (order is irrelevant to the produced value)", () => {
64+
const fn = coercer(
65+
{ type: "object", properties: { x: f32, y: f32 } },
66+
{ type: "object", properties: { y: f32, x: f32 } },
67+
);
68+
expect(fn({ x: 1, y: 2 })).toEqual({ x: 1, y: 2 });
69+
});
70+
71+
it("fills a new field from its default", () => {
72+
const fn = coercer(
73+
{ type: "object", properties: { x: f32, y: f32 } },
74+
{ type: "object", properties: { x: f32, y: f32, z: { type: "number", default: 9 } } },
75+
);
76+
expect(fn({ x: 1, y: 2 })).toEqual({ x: 1, y: 2, z: 9 });
77+
});
78+
79+
it("requires a default for a new field of a numeric STRUCT (every field is packed)", () => {
80+
// f32 properties pack as a struct → a missing field would be NaN, so a
81+
// new field without a default makes the conversion impossible.
82+
expect(createCoerceFunction(
83+
{ type: "object", properties: { x: f32 } },
84+
{ type: "object", properties: { x: f32, z: f32 } },
85+
)).toBeNull();
86+
});
87+
88+
it("omits a new NON-required field with no default on a plain object", () => {
89+
// f64 properties do not pack as a struct → a plain object may simply lack
90+
// an optional field, so this is convertible and the field is left off.
91+
const fn = coercer(
92+
{ type: "object", properties: { a: num } },
93+
{ type: "object", properties: { a: num, b: num } },
94+
);
95+
expect(fn({ a: 1 })).toEqual({ a: 1 });
96+
});
97+
98+
it("still requires a default for a new REQUIRED field on a plain object", () => {
99+
expect(createCoerceFunction(
100+
{ type: "object", properties: { a: num } },
101+
{ type: "object", properties: { a: num, b: num }, required: ["b"] },
102+
)).toBeNull();
103+
});
104+
105+
it("drops fields the output does not declare", () => {
106+
const fn = coercer(
107+
{ type: "object", properties: { x: f32, y: f32, z: f32 } },
108+
{ type: "object", properties: { x: f32, y: f32 } },
109+
);
110+
expect(fn({ x: 1, y: 2, z: 3 })).toEqual({ x: 1, y: 2 });
111+
});
112+
113+
it("coerces (and clamps) a retyped field", () => {
114+
const fn = coercer(
115+
{ type: "object", properties: { x: num } },
116+
{ type: "object", properties: { x: { type: "integer", minimum: 0, maximum: 10 } } },
117+
);
118+
expect(fn({ x: 50 })).toEqual({ x: 10 });
119+
});
120+
121+
it("gives each element its own copy of an object default (no aliasing)", () => {
122+
const fn = coercer(
123+
{ type: "object", properties: { x: f32 } },
124+
{ type: "object", properties: { x: f32, meta: { type: "object", properties: { n: { type: "number", default: 0 } } } } },
125+
);
126+
const a = fn({ x: 1 }) as { meta: object };
127+
const b = fn({ x: 2 }) as { meta: object };
128+
expect(a.meta).toEqual({ n: 0 });
129+
expect(a.meta).not.toBe(b.meta); // distinct instances
130+
});
131+
132+
it("is not convertible when an existing sub-field cannot be converted", () => {
133+
// Nested numeric struct gains a defaultless field ⇒ the sub-conversion,
134+
// and therefore the whole conversion, is impossible.
135+
expect(createCoerceFunction(
136+
{ type: "object", properties: { p: { type: "object", properties: { x: f32 } } } },
137+
{ type: "object", properties: { p: { type: "object", properties: { x: f32, y: f32 } } } },
138+
)).toBeNull();
139+
});
140+
});
141+
142+
describe("createCoerceFunction — arrays", () => {
143+
it("extends a fixed vector, filling from the item default (vec3 → vec4)", () => {
144+
const fn = coercer(
145+
{ type: "array", items: f32, minItems: 3, maxItems: 3 },
146+
{ type: "array", items: { type: "number", default: 0 }, minItems: 4, maxItems: 4 },
147+
);
148+
expect(fn([1, 2, 3])).toEqual([1, 2, 3, 0]);
149+
});
150+
151+
it("is not convertible when a longer fixed output has no item default", () => {
152+
expect(createCoerceFunction(
153+
{ type: "array", items: f32, minItems: 3, maxItems: 3 },
154+
{ type: "array", items: f32, minItems: 4, maxItems: 4 },
155+
)).toBeNull();
156+
});
157+
158+
it("truncates a fixed vector (vec3 → vec2)", () => {
159+
const fn = coercer(
160+
{ type: "array", items: f32, minItems: 3, maxItems: 3 },
161+
{ type: "array", items: f32, minItems: 2, maxItems: 2 },
162+
);
163+
expect(fn([1, 2, 3])).toEqual([1, 2]);
164+
});
165+
166+
it("maps every element of a variable-length array (with clamp)", () => {
167+
const fn = coercer(
168+
{ type: "array", items: num },
169+
{ type: "array", items: { type: "integer", minimum: 0, maximum: 5 } },
170+
);
171+
expect(fn([1, 10, 3])).toEqual([1, 5, 3]);
172+
});
173+
});
174+
175+
describe("createCoerceFunction — incompatible kinds return null", () => {
176+
it.each([
177+
["number → object", num, { type: "object", properties: { x: f32 } } as SchemaType],
178+
["object → number", { type: "object", properties: { x: f32 } } as SchemaType, num],
179+
["number → boolean", num, bool],
180+
["number → enum", num, { enum: [1, 2, 3] } as SchemaType],
181+
["array → object", { type: "array", items: num } as SchemaType, { type: "object", properties: { x: f32 } } as SchemaType],
182+
])("%s", (_label, input, output) => {
183+
expect(createCoerceFunction(input, output)).toBeNull();
184+
});
185+
});
186+
187+
describe("createCoerceFunction — namespace exposure", () => {
188+
it("is reachable as Schema.createCoerceFunction", () => {
189+
expect(typeof Schema.createCoerceFunction).toBe("function");
190+
expect(Schema.createCoerceFunction(num, f32)!(1.5)).toBe(1.5);
191+
});
192+
});

0 commit comments

Comments
 (0)