Context
Every WASM boundary crossing today goes through hand-written pack/unpack code
against a manual wire format:
- Render out (TS → C):
pack() / packSize() in ops.ts write a
Uint32Array into WASM linear memory. reduce() in src/clayterm.c reverses
the exact same layout with rd() and offset arithmetic.
- Input events (C → TS): the parser writes event structs into linear
memory; input.ts walks them back out with a DataView (see readEvent).
The wire format is duplicated across the packer, the size calculator, and the
unpacker. Every field addition or reorder is a lockstep change across TS and C,
policed only by comments (see ops.ts:217-227, which explicitly warns that C
treats field order as an invariant).
This issue is to explore whether an off-the-shelf zero-copy interop
framework would serve us better. It is not a proposal to adopt one.
Candidate frameworks
Non-exhaustive; add more in comments.
- FlatBuffers — schema-driven, zero-copy reads via
vtable-guided field access, wide language support.
- Cap'n Proto — similar model, more compact vtables,
wide language support.
- Shared C headers only — skip the framework, share a
.h describing the
layout, generate TS accessors from it. Lightest possible dependency.
Potential benefits
- One schema as the single source of truth in place of three hand-synced code
paths.
- Eliminates the explicit unpack step on the C side (zero-copy access, though
the exact cost model differs by framework — see open questions).
- Type-safe generated accessors on both sides in place of offset math and bit
shifts.
- Forward compatibility — schema-based frameworks allow adding fields with
defaults instead of hard-breaking every wire change.
- Cross-language reach — third parties consuming
clayterm.wasm from Go, Rust,
Python, etc. get generated bindings instead of having to reimplement the
packer.
- Debuggability — schema-based frameworks can dump a buffer to JSON for
inspection; the current format is opaque bytes.
Open questions to answer
- WASM binary size delta. The framework runtime lands in
clayterm.wasm.
How much does each candidate add? Compare against current baseline reported
by size-report.yml.
- JS bundle size delta. The builder/reader library ships to consumers.
How much does each candidate add over the ~600 lines of custom pack code?
- Per-frame allocation profile. The current packer writes straight into
WASM linear memory with no intermediate objects. Do the candidate TS
builders introduce allocations that create GC pressure at frame rate?
- Access cost on the C side. "Zero-copy" is not free — flatbuffers and
Cap'n Proto both use vtable-guided lookups. What is the actual per-op cost
vs. the current rd() loop?
- Buffer size delta. Current bit-packing (padding in a word, attrs in the
color alpha byte) is dense. How much larger does a typical frame get with
each candidate?
- Toolchain cost.
flatc / capnp need to plug into the Makefile and
Deno tasks. What does the dev setup look like?
- Scope. Does the framework replace both directions (render + input),
or just one?
Context
Every WASM boundary crossing today goes through hand-written pack/unpack code
against a manual wire format:
pack()/packSize()inops.tswrite aUint32Arrayinto WASM linear memory.reduce()insrc/clayterm.creversesthe exact same layout with
rd()and offset arithmetic.memory;
input.tswalks them back out with aDataView(seereadEvent).The wire format is duplicated across the packer, the size calculator, and the
unpacker. Every field addition or reorder is a lockstep change across TS and C,
policed only by comments (see
ops.ts:217-227, which explicitly warns that Ctreats field order as an invariant).
This issue is to explore whether an off-the-shelf zero-copy interop
framework would serve us better. It is not a proposal to adopt one.
Candidate frameworks
Non-exhaustive; add more in comments.
vtable-guided field access, wide language support.
wide language support.
.hdescribing thelayout, generate TS accessors from it. Lightest possible dependency.
Potential benefits
paths.
the exact cost model differs by framework — see open questions).
shifts.
defaults instead of hard-breaking every wire change.
clayterm.wasmfrom Go, Rust,Python, etc. get generated bindings instead of having to reimplement the
packer.
inspection; the current format is opaque bytes.
Open questions to answer
clayterm.wasm.How much does each candidate add? Compare against current baseline reported
by
size-report.yml.How much does each candidate add over the ~600 lines of custom pack code?
WASM linear memory with no intermediate objects. Do the candidate TS
builders introduce allocations that create GC pressure at frame rate?
Cap'n Proto both use vtable-guided lookups. What is the actual per-op cost
vs. the current
rd()loop?color alpha byte) is dense. How much larger does a typical frame get with
each candidate?
flatc/capnpneed to plug into the Makefile andDeno tasks. What does the dev setup look like?
or just one?