diff --git a/.agent-notes/2026-08-19-cbor-legible-wire/README.md b/.agent-notes/2026-08-19-cbor-legible-wire/README.md new file mode 100644 index 000000000..728e8a595 --- /dev/null +++ b/.agent-notes/2026-08-19-cbor-legible-wire/README.md @@ -0,0 +1,20 @@ +# The CBOR-legible wire + +[`cbor-legible-wire.md`](./cbor-legible-wire.md) is the design document of +record for converting the rumors wire protocol to end-to-end +deterministic-encoding CBOR: every directed stream of a V2 session parses as +an RFC 8742 CBOR sequence, the on-disk bookmark became a fully CBOR-parseable +file (format v4), a three-level observation hook was added, and the opaque +hexdump wire snapshots were supplanted by CBOR reflection rendering. Its +decision record carries the owner rulings and implementer resolutions the +work was built under, including the post-review rulings on error taxonomy, +feature gating, and hook identity. + +Retired as implemented: the design shipped on this branch, survived a +multi-round adversarial review, and its invariants live inline at the code +they govern, per the repository's rules. The body is byte-identical to the +document's last revision in `design/`; its citation of +`design/payload-depth-limit.md` resolves to +[`../2026-08-20-payload-depth-limit/payload-depth-limit.md`](../2026-08-20-payload-depth-limit/payload-depth-limit.md). +The wire-corpus figures quoted inside were measured at the commits the +document names; the code they describe has the current numbers. diff --git a/.agent-notes/2026-08-19-cbor-legible-wire/cbor-legible-wire.md b/.agent-notes/2026-08-19-cbor-legible-wire/cbor-legible-wire.md new file mode 100644 index 000000000..2e17983c2 --- /dev/null +++ b/.agent-notes/2026-08-19-cbor-legible-wire/cbor-legible-wire.md @@ -0,0 +1,422 @@ +# A CBOR-legible wire protocol, and the observation hook + +Status: accepted, in implementation; tracked as +[rumors#35](https://github.com/oxidecomputer/rumors/issues/35), and +where that issue and this document differed, the rulings below resolve +toward the issue. Owner: Finch. Origin: design conversation, +2026-08-19. Builds on the version-keying migration's +uniform-CBOR rulings (payloads and the wire's version atom are already +CBOR; each supply-record body is already a two-item CBOR sequence). + +## Goal + +Every directed stream of a session — data streams and the control stream +alike — parses as a CBOR sequence with standard tag unwrapping, so that a +tool knowing nothing about rumors can unfold a recorded session into a +legible tree, down to exactly the atoms that are honestly rumors-private. +The concrete payoff: a generic debugger for rumors sessions that needs no +knowledge of the internal format or of the application's message types +(which are the application's own CBOR, legible for free), and an +observation hook that feeds it — whose first consumer is a `tracing` +adapter with deep structural inspection of live sessions. + +## Why no stream length is needed + +The form is RFC 8742 *CBOR sequences*: concatenated data items, no count, +no total length, no terminator. That is exactly the shape of an unbounded +stream, and it degrades gracefully — a truncated capture is a valid-prefix +sequence. (CBOR's indefinite-length containers also need no length up +front but want a closing break code an aborted session never writes; +sequences are the right choice.) + +## The layers, current form → CBOR spelling → cost + +| Layer | Today | CBOR spelling | Recurring cost | +|---|---|---|---| +| Frame signal | one dense byte (stream × state, 17 × 10 codes) | unsigned int item carrying the same dense code | +1 byte for codes ≥ 24 (most); see the signal ruling below | +| Frame | signal ‖ raw body | array `[signal]` or `[signal, body]` | +1 byte array head | +| Record framing | u32 BE record header | **tag 63** ("embedded CBOR sequence in a byte string"): each record is `63(bstr(tagged version ‖ payload))` | ≈ 0 (tag 2B + bstr head 1–5B vs flat 4B; often equal, −1 for small records) | +| Record body | CBOR bstr(version) ‖ CBOR(payload) — already a sequence | the version atom gains its tag; payload unchanged | +3 bytes (the version tag) | +| Run length | u32 BE | the whole run is `63(bstr(record*))`; its byte-string head is the run length | ≈ 0 (tag 2B + head 1–5B vs flat 4B) | +| Query child listing | raw `(radix ‖ 24-byte hash)*` | map `{radix: hash}` (+3 B/child; ruled over the +2 B/child alternating array for the canonicality coincidence below) | the one hot cost, **measured** at the calibration cells: the whole change moved the calibrated per-disputed-message intercept 35 → 43 B — +3.9% at the design record (207 → 215 B/message), +8% at mid-size records, +14% at minimal `u64` records | +| Greeting | fixed-offset block + frames | one item, `24(bstr(map))`: a text-keyed map (`{"listing": {radix: hash}, "set_len": …, "version": , "protocol": "rumors", "max_version_bytes": …, "payload_depth_limit": …, "target_message_size": …}`, keys in deterministic order) behind the embedded-item tag, so the control-stream reader gets the item's length up front. The `protocol` entry is the rumors magic — tag 55799 announces only "CBOR", never whose — the version atom rides tagged, dissolving its old bare-canonical spelling, and the `payload_depth_limit` entry is held to exact equality across the pair (`Error::PayloadDepthMismatch`) | few dozen bytes, once per session | +| Preamble magic | 6 raw bytes opening a 25-byte fixed block | one 30-byte self-described item: `55799(["rumors", version: uint, network: bstr, intent: uint])` — the magic survives as the text item, so "not rumors" still diagnoses at the preamble | once per session | +| Party hand-off | one length-framed frame carrying the party's canonical bytes on the control stream | tagged bstr (the party atom, tagged per the table below) | few bytes, once per hand-off | +| Stream open label | epoch byte ‖ index byte | two leading uint items | +0–1 bytes per stream | +| Epilogue marker | one byte (`.`) | the text item `"."` — one byte dearer than a small int, and a generic tool renders the dot | +1 byte, once per session | + +Notes on the spellings: + +- **Tag 63 is the load-bearing find.** The u32 record header earns its + keep by giving O(1) record skip and budget pricing independent of + payload shape (nested CBOR containers are not O(1)-skippable — their + headers carry counts, not subtree byte lengths; only strings are). + Tag 63's byte string preserves both properties exactly, while telling a + generic tool "unwrap me and parse the inside as a sequence." The + ledger's charge-before-custody ordering is untouched. +- **The listing map's key order coincides with canonicality.** CBOR + deterministic encoding mandates ascending keys; the wire's canonical + form mandates strictly ascending radixes. Under the map spelling the + two disciplines are one discipline — that coincidence is why the map + is ruled in over the byte-cheaper alternating array. +- **The wire is deterministic-encoding CBOR, as a stated contract**: + shortest-form headers everywhere, one spelling per value. This is what + keeps the byte-pinning snapshot discipline meaningful after the change. +- The existing `record_len` pricing pattern (exact header arithmetic, + pinned against an actual push) generalizes to every priced length + above. + +## Where the opaque boundary stays, and why + +Version and party atoms remain opaque byte strings. Their canonical +bit-level codings are the crate's semantics; re-spelling them as CBOR +structure on the wire would be true structural re-encoding — larger, +slower, and a second spelling of the exact thing the tree pins +byte-for-byte. The generic debugger shows "a 37-byte version atom"; +rendering the atom's *meaning* is the public skyline iterator's job +(`design/version-skyline-iterator.md`) — a rumors-aware lens over the +rumors-blind skeleton is one `Plateau` walk away, and the two designs +are deliberate complements. + +## The bookmark: fully CBOR-parseable on disk + +The stored bookmark follows the same property (ruled 2026-08-19): the +whole file parses as CBOR, not just its payload. The v4 form: + +``` +55799( [ format_version: int (= 4), integrity: bstr, payload: 24(bstr(map)) ] ) +``` + +where the map is Network (16-byte bstr key) → array of clocks, each +clock a tagged byte string (`CLOCK_TAG(bstr(party ‖ version canonical +bytes))`). The integrity hash covers the encoded format-version item +followed by the encoded tag-24 payload item — every array item except +the integrity item itself, a CBOR-visible region rather than an offset +convention, and coverage no weaker than v3's (which hashed the version +too). The fixed opening (the 55799 tag and array head) sits outside +the hash; corrupting it fails shape validation instead, and the +committed corruption sweep proves rejection stays total over every +byte of the file. The raw magic string has no analogue in a +self-described CBOR file and the v4 format (and public API) carries +none. + +`FormatError`'s taxonomy is meaning-named with typed carriers (ruled +2026-08-19: variants say what they diagnose, not what stage tripped): +`NotABookmark` holds a `FrameDefect` naming the "not self-described +CBOR / wrong shape" failure, `Record` carries a typed `RecordDefect` +for a payload that is not the record this codec writes, +`VersionMismatch` and `HashMismatch` are unchanged in meaning, and +truncation is rejected at every strict prefix. This is a +format-version bump under the bookmark's own convention. + +## Tagged atoms: context-free identity for the opaque byte strings + +The opaque atoms gain CBOR tags — selectively — so their identity travels +with them rather than living in protocol position (ruled 2026-08-19). A +tagged atom is self-describing anywhere it appears: a wire capture, a +bookmark, a log line, a pasted hex snippet. That turns the generic +debugger's "37-byte atom" into a dispatch point: a thin *rumors lens* +keyed on nothing but a tag table sends version atoms to the public +skyline iterator and renders them semantically, with zero +protocol-position knowledge. Tags are the bridge between the +rumors-blind skeleton and the rumors-aware lens. + +**Placement rule (the crux): tags belong to the transport codecs, never +to the serde impls.** A `Version` whose *serde* implementation emitted +tags would stop being format-agnostic — an application payload +containing a `Version`, serialized to JSON, would break on a +CBOR-specific concept tunneled through serde. Instead the wire and +bookmark codecs (already hand-written at the framing layer) write +`tag ‖ untagged-serde-bytes` and hand-read the tag before delegating +decode. `before`'s serde impls stay untagged and backend-agnostic; the +tags are protocol vocabulary, owned where the protocol is spelled. + +Consequences for parsing: no wholesale non-serde parser is required — +only the points that already hand-parse read tags. (Two library facts: +`ciborium::tag`'s `Required`/`Accepted` wrappers do tunnel tags through +serde, but as a ciborium-specific magic-newtype mechanism that would +format-lock `before`'s impls — deliberately not used; and +`ciborium::Value` preserves tags natively, so generic consumers get them +for free.) + +Tag / don't-tag: + +- **Tagged**: version atoms and party atoms wherever the protocol spells + them (supply records, the greeting, the party hand-off, the + bookmark's stored clocks). + Their contexts are diverse, and their per-instance cost (+3 bytes for + a first-come-first-served-range tag) lands on payload-dominated paths + or once-per-session surfaces. +- **Untagged**: hashes inside listings — one context, + position-determined, and +3 on a 25-byte child is ~12% on the + dispute-heavy path, the one place bytes are dear. Structure already + names them. Signals and counts likewise: position suffices. + +Tag numbers come from the IANA first-come-first-served range (32768+ +per RFC 8949 §9.2 — everything below is standards-action or +specification-required; numbers through 65535 still encode in 3 +bytes). The honest path is registering a small contiguous block (FCFS +registration is lightweight); squatting risks a generic tool someday +rendering these atoms with someone else's semantics. The provisional +block is based at 53845 (hex D255, the ASCII bytes "RU" with the FCFS +high bit set), chosen against the live registry's unassigned space: +party, version, then clock (the bookmark's stored party ‖ version +concatenation, tagged whole rather than split), with a small reserve. +Until registration lands, the numbers live in one pinned constant +table, and the capture renderer learns their names (the sanctioned +renderer-vocabulary re-accept class). + +## The signal-redundancy ruling + +Within one recorded directed stream the signal's stream component is +constant, so signals *could* re-base to state-only codes (≤ 9, always one +CBOR byte). But the dense code's redundant stream component is what the +`Mislabeled` check validates against the transport label — a conformance +bug detector with committed fault-matrix coverage. Ruling: keep the +redundancy and pay the byte (codes ≥ 24 cost two). + +## The observation hook + +The capture path is a public hook — the `observe` module's three +traits, scoped peer, session, directed stream (ruled 2026-08-19; the +module's rustdoc is the documentation of record, this section the +design rationale): + +- **A handler attaches at construction** (`Peer::observe`, and + `Bootstrap::observe` for the joining session that runs before the + peer exists — attaching an `Arc` field is what retired the builder's + `Copy`; its retry affordance survives as `Clone`). For each session + the peer enters (gossip, and equally bootstrap and retire — a + capture that skips session kinds is a debugger with blind spots), + the handler is asked for a **per-session sub-handler**. The + sub-handler's creation carries what identifies the session — kind + and protocol — and its lifetime is the session's. It deliberately + carries no session number: numbering is the observer's own concern, + counted inside its handler with its own synchronization, exactly + like message interleaving. + The role election is deliberately *not* part of that identity: it + does not exist yet at session start. It is delivered by a dedicated + notification when decided (after the greetings, before any data + stream), and never for an equal-versions session, which elects no + one. +- **The per-session handler yields a per-directed-stream handler** for + each directed stream as it opens: the control stream's two + directions at session start, each data stream when first spoken or + read (a stream the session never uses yields no handler). The + creation call carries the directed-stream identity — control, or + data with its speaking role and wire index, plus the direction; the + handler's lifetime is the stream's. +- **The per-stream handler is invoked once per protocol message, in + stream order.** Invocation order within one directed stream is that + stream's message order; **the hook imposes no cross-stream + synchronization** — streams pump concurrently, and the library does + not make every observed frame contend on a session-wide point just to + serialize observation. A consumer that wants the observed + interleaving reconstructs it without a lock from a session-scoped + atomic ordinal in its own per-session handler, stamping each message + as it arrives. A slow handler back-pressures only its own stream; + the hook must still never block on protocol progress, documented at + the hook. +- **The per-message payload is the message's wire bytes, not parsed + values**: `bytes: &[u8]`, **exactly one CBOR item** per invocation — + a data frame, or one control item (preamble, greeting, hand-off, + epilogue; the two-byte stream-open label is addressing, not an + item). Received items are captured off the transport, never + re-encoded; only complete, accepted items are delivered. Two + deliberate choices here: borrowing keeps the hot path zero-copy + (attachment costs one branch per frame unattached, one contiguous + materialization per frame attached), and bytes-not-types keeps the + hook *itself* rumors-blind — no protocol type appears in its + signature, so the hook's API is stable across wire evolution and its + consumers parse with any CBOR library (or none). Stream identity + lives at the per-stream handler's creation rather than on every + message, so the message call carries only what varies per message. +- Attachment is dynamic (`Arc` held as an `Option`), not a + generic parameter on `Peer`: one branch per frame when unattached, + and the public type stays unparameterized. An observability surface + does not warrant monomorphization. +- **Only the CBOR dialect is observable**: `Protocol::V1` sessions are + not observed, because the frozen legacy wire is not a CBOR sequence + and cannot honor the one-item contract. + +## The two consumers + +Both consumers are in scope for the implementation lane (ruled +2026-08-19); each is the dogfood proof, from a different angle, that +the hook's bytes-only signature suffices. + +**The snapshot extractor.** A CBOR reflection renderer that supplants +the opaque hexdump wire snapshots with human-readable introspection: +the snapshot suites capture sessions through the public hook and pin a +deterministic rendering of every observed item — structure unfolded +(ints, text, arrays, maps, embedded items), tagged atoms named from +the tag table, opaque byte strings as full hex. The byte-pinning +discipline survives the legibility twice over: because the wire is +deterministic-encoding CBOR as a stated contract, a rendering that +shows every item's complete content is injective on wire bytes — two +different byte streams cannot render identically, with anything the +walk cannot vouch for falling back to explicit exact hex — and the +harness holds the rendered items to the transport capture as a +totality oracle (label plus concatenated items reproduce every wire +byte). So the legible snapshot still pins the wire, while a reviewer +can finally *audit* what moved and why. + +**The tracing adapter.** A separate crate (or feature-gated module) so +the core keeps its dependency surface: sessions open `tracing` spans +(session identity as span fields), every observed frame is an event +within its span, and the CBOR structure maps to structured fields — +ints and text directly, maps by key, atoms as lengths-plus-hex. Because +the wire is CBOR all the way down, the adapter is a *generic* +CBOR-to-tracing bridge plus a thin naming layer; deep inspection of +application payloads comes free, since they are the application's own +CBOR. + +## The committed contract + +- **The rumors-blind render test, stated as a property over arbitrary + sessions**: randomized peer contents and payloads drive real sessions + of every kind the capture harness can pair, every directed stream's + capture is parsed with a generic RFC 8742 walk plus standard tag + unwrapping (55799, 63, 24; unknown tags tolerated), and the proptest + asserts everything parses with no bytes outside CBOR items. The + parser knows nothing of rumors. This is the tamper-evident form of + the legibility promise; prose claims of legibility are decoration + without it. +- **Capture validity is a two-property contract, and both properties + are permanent** (ruled 2026-08-19): the render proptest above keeps + its transport-level capture forever — it is the proof that + *external* capture (a third party tapping the wire, no rumors code + involved) yields valid CBOR — while the observation hook carries its + own *internal*-capture properties: every hook invocation is exactly + one CBOR item, and concatenating a stream handler's items + byte-equals that directed stream's transport capture. The + whole-stream property implies the per-item property only if the hook + is bug-free; the pairing tests exactly that implication. Neither + test supplants the other. +- The full snapshot corpus re-accepts as one deliberate, owner-ruled + pre-release format change, named in the re-accepting commit. +- Re-derived (never transcribed) readings: the dispute-wire closed form + and crossover, the window-solve constants, digestshare, decode-alloc + meters, and the affected wasm32 wire-door pins. + +## Cost summary + +An M/L codec lane, comparable to the borsh→CBOR wire migration: the +codec layer (signal/frame/streams/greeting/bookmark format) rewritten, +hand-parsed as today (delegating to ciborium is *not* required — the +structural validation and exact pricing stay first-class), plus the hook +threading through the session drivers, plus the re-accept and re-pin +wave. Recurring wire cost, measured at the calibration cells: +8 B per +disputed message end to end (the calibrated intercept moved 35 → 43 B), +which is +3.9% at the design record size and grows toward +14% only for +minimal `u64`-record corpora; essentially zero relative cost on bulk +supply (+3 B per record for the version tag, ≈0 for the framing). A +one-off whole-session check (measured, not an instrument: the +fixed-corpus gossip bench's 5,000-shared / 2,500-per-side-differing +cell, single-byte payloads — the metadata-dominated worst corner) +read +16.4% total session bytes, about 51 KB on a 308 KB session, +falling toward the +3.9% figure as payloads approach the design +record size. The +committed snapshot corpus — toy sessions dominated by their per-session +constants — stands at 9,853 wire B carrying 1,776 digest B (74 digests +at 24 B, 18.0%, measured by `tools/digestshare` over the reflection +renders); against the pre-conversion corpus's 5,273 B that is an 87% +growth, a denominator that overweights the once-per-session surfaces +by design, and the per-message figures above are the hot-path claim. +(The greeting's `payload_depth_limit` entry accounts for 23 B per +greeting of that figure: the corpus measured 8,887 wire B without it. +The remaining growth over the figures previously quoted here predates +that entry; the corpus had moved under the quoted numbers, and both are +re-measured here.) +What does not change: session semantics, the +deadlock-freedom argument (framing-independent; the hook adds +observation, never a protocol dependency), and every validation +property, re-denominated. + +## Sequencing + +After the version-keying branch merges: same review season, one format +era. Pre-release is the cheap moment for a wire change; once a release +ships, this is a new protocol version by the hard rules. The render +test lands with the codec lane itself; the two consumers (the snapshot +extractor and the tracing adapter) land in the same lane, downstream of +the codec and the hook. + +## Decision record + +- 2026-08-19 (Finch): the +1 byte per record for CBOR-legible record + bodies is accepted; the record body is a CBOR *sequence*, deliberately + not an array-wrapped tuple. +- 2026-08-19 (Finch): pursue full stream legibility — structural CBOR + for listings and greetings; payloads legible via their containing + records; the generic-debugger use case is the design's purpose. +- 2026-08-19 (Finch): the on-disk bookmark format becomes fully + CBOR-parseable under the same property. +- 2026-08-19 (Finch, shape; Claude, refinements): the observation hook — + Peer-attached handler, per-session sub-handler carrying session + identity, per-message bytes-level invocation; the tracing adapter is + the first consumer. +- 2026-08-19 (Finch): the opaque atoms gain CBOR tags, placed in the + transport codecs (never the serde impls), per the tag/don't-tag table + above. +- 2026-08-19 (Finch): keep the signal's redundant stream component and + pay the byte; the `Mislabeled` conformance check keeps its wire-side + witness. +- 2026-08-19 (Finch): listings are spelled as the map, for the + canonicality coincidence; cost still measured at the calibration + cells before pinning. +- 2026-08-19 (Finch): the hook is three-level (peer → session → + directed stream), per the issue's framing; the library imposes no + cross-stream ordering cost — a consumer wanting the interleaving + reconstructs it from its own session-scoped atomic ordinal. +- 2026-08-19 (Finch): the greeting map carries a `protocol` entry as + the rumors magic, since the self-described CBOR tag alone does not + name the protocol. +- 2026-08-19 (Finch): the implementer picks a provisional contiguous + tag block from the FCFS range, pinned in one constant table; the + IANA registration (or renumbering to the registered block) is + Finch's, before first release. +- 2026-08-19 (Finch): both consumers are in scope for the + implementation lane — the CBOR reflection snapshot extractor + supplants the opaque hexdump wire snapshots, and the tracing adapter + lands alongside rather than trailing. +- 2026-08-19 (Finch): capture validity is a permanent two-property + contract — the transport-captured whole-stream render proptest + (external capture) and the hook's per-item plus concatenation + differential (internal capture); neither migrates into the other. +- 2026-08-19 (Claude, implementer resolutions within the hook ruling, + each flagged for review): the role election rides a dedicated + session-handler notification rather than session-creation identity + (it does not exist at session start); `Protocol::V1` sessions are + unobserved (the frozen wire cannot honor the one-item contract); + and the `Bootstrap` builder's `Copy` gave way to `Clone` so the + handler can attach before the peer exists. +- 2026-08-19 (Finch): a one-off whole-session bytes measurement + (before vs after the conversion, mirroring the fixed-corpus gossip + bench shape) runs as a final bandwidth sanity check; deliberately + not landed as an instrument. +- 2026-08-19 (Finch): error taxonomies name what they diagnose. The + bookmark's variants are meaning-named (`NotABookmark`, `Record`), + and the handshake's failure cases are enumerated as typed public + variants — `PreambleMalformed` carrying a `PreambleDefect`, + `PreambleTruncated` carrying byte counts — with no collapse into + `io::Error`. +- 2026-08-19 (Finch): `PROTOCOL_MAGIC` is V1 vocabulary, exposed only + under the `protocol-v1` feature; the V2 wire's protocol identity is + the preamble's `"rumors"` text item and the greeting's `protocol` + entry. A V2 endpoint still recognizes the legacy bytes internally, + to diagnose cross-dialect pairings. +- 2026-08-19 (Finch): session numbering is consumer-side, like + message interleaving — the hook's session identity carries no + ordinal; an observer wanting "the peer's Nth session" counts inside + its own handler with its own synchronization. +- 2026-08-20 (Finch): the greeting map gains a `payload_depth_limit` + entry, held to exact equality across the pair — the payload depth + limit is a property of the shared set (every replica must hold and + forward all content), so it is Network-like, never negotiated: + negotiating down is unsound, since a peer may already hold messages + deeper than a negotiated bound. A deliberate pre-release wire + format change, with its snapshot corpus re-accepted in the + implementing commit; the full design (symmetric enforcement, the + minted codec, the closure-scoped batch) lives in + `design/payload-depth-limit.md`. diff --git a/.agent-notes/2026-08-20-cbor-wire-review/README.md b/.agent-notes/2026-08-20-cbor-wire-review/README.md new file mode 100644 index 000000000..b0992a9ce --- /dev/null +++ b/.agent-notes/2026-08-20-cbor-wire-review/README.md @@ -0,0 +1,39 @@ +# The CBOR-wire adversarial review packet + +[`REVIEW.md`](./REVIEW.md) is the adversarial review packet of record for +the CBOR-legible wire stack (the branch this note lives on). It served as +the working spec for the entire fix-and-feature campaign that followed: +every finding carries an executable **Resolution** block (file, mechanism, +acceptance criteria), each written to stand without the review's own +context, and the packet was amended in place as owner rulings landed. + +What it holds, in its own order: the functional chase list; findings by +severity (bugs, contract–prose mismatches, risks, behavioral notes), each +marked **verified** (ran or constructed) versus **assessed** (read only); +dispositions for all fourteen charter seeds; round-4 assumption checks; +public-API-delta judgments; simplification candidates; the clean-bill +sections (where no issues were found, and what was considered and +dismissed); residual risks and test gaps; and the out-of-range payload +depth observation, whose owner disposition grew into the full ten-step +implementation spec for the payload-depth-limit feature (the knob, the +symmetric greeting exchange with equality abort, and the peer-constructed +codec that carries the limit beside the serializer and deserializer +function pointers). + +Retired as executed: every in-range resolution landed through the +worktree-isolated fix lanes and their review rounds, and the depth-limit +spec shipped as the feature that followed. Reading notes for today's +tree: + +- Line anchors and commit SHAs cite the review-time branch states + (`a5c4ca1a`, re-anchored at `d05a6d03` after the PR #37 rebase). Both + predate the signed rebase onto main, so those exact SHAs survive only + in the `backup/cbor-wire-pre-37` and `backup/cbor-wire-pre-resign` + branches; the same changes live on this branch under different SHAs. +- The packet's vocabulary predates two later sweeps: the error-taxonomy + rename (the send-side `EncodeError` naming and the scope-qualified + adapter pair) and the crate-wide constructor-naming pass. Where the + packet and the code disagree on a name, the code is current. +- The rebase-audit and dispatch-sequencing notes describe campaign + mechanics (re-sign ordering, seed re-checks) that completed; they are + history here, not open instructions. diff --git a/.agent-notes/2026-08-20-cbor-wire-review/REVIEW.md b/.agent-notes/2026-08-20-cbor-wire-review/REVIEW.md new file mode 100644 index 000000000..bafcbca7f --- /dev/null +++ b/.agent-notes/2026-08-20-cbor-wire-review/REVIEW.md @@ -0,0 +1,1620 @@ +# Adversarial review: the CBOR-legible wire stack (rumors#35) + +Reviewed: `review/cbor-wire` @ `a5c4ca1a9b7f836f3b47d2fad4ae5d1a1fc27834` +(base `15bd905e`, 11 commits + 1 merge), plus `w2/tracing-adapter` @ +`526cbf87`. Method: the charter's four rounds — full read of the codec +stack, the instruments, the seams, and constructed assumption checks — +file by file over every non-snapshot change. Every claim below is marked +**verified** (I ran or constructed it) or **assessed** (read only). +Constructed witnesses were added as temporary scratch tests, run, and +removed; the worktree is clean, `REVIEW.md` excepted (it sat at the +review SHA through the review and is re-anchored at `d05a6d03` since +the rebase note below). + +Each finding carries a **Resolution** block written to be executable +without this review's context: file, mechanism, and acceptance criteria. +Items that change public API or gate policy are marked **owner-gated** +and must go to Finch before landing; everything else is executable +directly in the implementing worktree. Line numbers cite `a5c4ca1a` +for files the post-review rebase left untouched (byte-identical at +`d05a6d03`) and are restated at `d05a6d03` anchors where it moved them; +see the rebase note below. + +Verification I ran: the three dispute-wire calibration cells (with +measured values), 605 tests across the lib and the affected integration +suites (observe, wire_legibility, handshake, all three snapshot suites, +target_message_size, decode_alloc) — all green; `tools/digestshare` +(figures match the design doc exactly); a per-commit +`cargo check --all-targets --all-features` bisection sweep over all 12 +commits; the gate's `clippy-default` rumors leg; `cargo check` of the +adapter crate at the branch tip; three constructed witnesses (below); +and a one-off orphan-snapshot sweep. + +**Rebase note.** After this review froze, main merged PR #37 (internal +height- and value-erasure), and the branch was rebased onto that merge +(`991f4663`) as `d05a6d03` — same 12-commit topology; the tip is +gate-clean per the rebasing agent's report (told, not re-verified +here), and a fresh per-commit check sweep over the rebased range is +recorded at seed 2. Every referent in this packet was re-checked +against the rebased tree. Files the erasure left untouched carry over +exactly (capture.rs, handshake.rs, observe.rs, error.rs, cbor.rs, +link.rs, the bookmark tree, tests/observe.rs, tests/dispute_wire.rs); +the erasure-moved referents are restated in place at their `d05a6d03` +anchors — B1's call-site sweep, B2a's witness (`Message` and `LeafRun` +are no longer generic; `records` takes the peer-minted deserializer), +and the depth-limit spec's step 3, which the erasure *simplified*: +both dialects' payload ingress now funnel through one minted +`PayloadDeserializer` (`message.rs`). The rebasing agent's judgment +calls were reviewed: dropping the leaf-level trailing-bytes variant in +favor of the deserializer's exactly-one-value `InvalidData` is +consistent with `from_slice`'s documented contract (if R5's +typed-error extension is ruled in, this class is a candidate to ride +along); the observe-hook re-threading through the erased +`Reconciliation` funnel is assessed from instrument evidence, not +line-read — the wire-byte differential, the capture-complementarity +suites, and tests/observe.rs are unchanged and green at the tip, and +those are the instruments a threading mistake would trip. **Dispatch +sequencing**: the rebased commits are unsigned (the signing agent was +locked during the rebase); re-sign and force-push *first* — the +re-sign rewrites every SHA — and only then branch fix work off the +settled tip. Commit SHAs named in this packet's dispositions +(`186037d9`, `0db57d00`, `5367a82e`, and the rest) are the *reviewed* +history, preserved at the local ref `backup/cbor-wire-pre-37`; the +findings themselves are restated against the rebased files, so an +executor never needs those commits. + +**Post-rebase rulings (Finch, review follow-up).** Every owner-gated +question this packet posed is now ruled, each per this review's +recommendation; executors treat these as decided: + +- **B2**: enforcement alternative *declined* — the prose rescope plus + boundary-pinning witnesses is the resolution of record. +- **R2**: keep-and-document; do *not* delete `NetworkTruncated`. +- **R3**: re-export `HeadError`; it stays exhaustive (no + `#[non_exhaustive]`). +- **R5**: the typed-error ruling *extends* to the identity hand-off + and the greeting — implement the sized-M shape in the resolution. +- **`SessionKind`**: add `#[non_exhaustive]` now, pre-release. +- **R6 step 4**: add the bare-lib clippy gate leg (one justfile line). +- **Seed 13**: the pairing test (`tests/snapshot_liveness.rs`), not + the cargo-insta runner. +- **digestshare**: gate-wire it (justfile recipe in the lint tier). +- **Depth-limit feature**: the minted-codec shape is approved (ruling + recorded at step 5 of the implementation spec). + +**Rebase audit trail.** The rebasing agent disclosed ten finer-grained +judgment calls; each is dispositioned here. Items marked *seed* go +into the adversarial review rounds as dispute-don't-confirm seeds: + +1. Assertion-strength choices in decode/tests.rs + (`a_zero_length_record_is_structurally_valid`, + `supplied_record_errors_are_typed`) — *seed*: re-derive the error + paths from `parse_record` (frame.rs:326–369) independently; the + agent's reasoning is verified only by the tests passing. +2. Testdoc accuracy on stitched test bodies (decode/tests.rs and + error_atlas.rs especially) — *seed*: the gate's testdoc leg checks + that doc comments exist, not that they are still true; re-read each + stitched test's stated invariant against its merged body. +3. Payload-type anchoring: erased `Message::new(0)` would infer `i32` + where `LeafRun::` used to anchor `u64`; every such literal was + made explicit (`0u64`) rather than relying on the encodings + coinciding — recorded; the change is strictly toward explicitness. +4. `pushed_runs_validate_and_iterate` (frame/tests.rs) reads payloads + back through `records(Message::deserializer::())` + + `.arc::()` — rebase-authored code, neither parent's — *seed*: + review as new code (`arc` is the checked-downcast panic path, + acceptable in a test). +5. capture.rs adopted wholesale from `0db57d00` — **discharged by + verification**: the file at `d05a6d03` is byte-identical to + `0db57d00`'s *and* to `a5c4ca1a`'s (the file this review read), and + the post-rebase sweep compiles it under + `--all-targets --all-features`. B1's anchors are exact. +6. Turbofish removals in `remote/` were regex substitutions, not + eyeball edits — recorded; the compiler and clippy backstop, and + reviewers eyeball the sites they visit anyway. +7. Observe-threading placement in gossip.rs (`observe.begin` after the + deserializer mint, before the preamble; `bootstrap_v2` gained a + trailing `observe` parameter; V1 paths deliberately get none) — + *seed*: derive the required placement independently from the hook + contract (begin-before-first-byte; election-before-data; the V1 + exclusion) and judge the code against the derivation, not the + agent's argument. +8. The keep-both merge heuristic once produced syntactically-plausible + garbage (duplicated `connected` return lines; compiler-caught) — + *seed as a class*: hunt for merge duplicates that DO compile — + duplicated match arms, repeated doc paragraphs, double writes. +9. Mid-branch bisectability was unverified by the rebase (tree-level + hop proof only; intermediate commits never individually compiled; + `fa6cbc26` committed from git's staged auto-resolution) — + **partially discharged**: this review's post-rebase sweep + check-verifies all 12 commits; full-gate verification remains + tip-only, matching the lane's original endpoint-gating disclosure. + `fa6cbc26`'s diff is small enough to eyeball in review. +10. Dead-import removals (`DeserializeOwned`/`PhantomData`/`std::io` + in streams.rs/start.rs) were grep-justified — recorded; clippy's + clean pass corroborates. + +--- + +## Chase list: the functional axis + +Every finding below, re-sorted by what is actually at stake — so the +set of "bugs to chase" is exact. The tiering in the findings sections +is by review severity (claims violated, discipline breached); this list +is by functional exposure. + +- **Functional defects in shipped code introduced by this stack: none + found.** Across all four rounds, nothing surfaced where a production + peer, replica, bookmark, or observer reaches wrong state, wrong + bytes, a panic, or divergence. The shipped-code items in this packet + are diagnostic-fidelity and API-shape concerns only. (One *pre- + existing* functional edge — an undocumented payload nesting-depth + limit, asymmetric between encode and decode — was surfaced by B1's + user-chosen-`T` analysis and is dispositioned in the out-of-range + observation at the end of this document; it exists at the base + commit too and is not this stack's.) +- **Demonstrated failure, test instrumentation only: B1.** A real, + constructed crash (SIGABRT), but in the `test-internals` renderer; no + committed input reaches it, and the trigger is a payload a test + author would have to choose. What makes it chase-worthy is the crash + demonstration plus the false in-tree bound claim plus the hard rule's + standing demand for the property. This is the entire "bug" chase + list, strictly construed. +- **Instrument that can silently mask a real regression: R1.** The + minimal calibration cell's ±2 band is the one place found where a + genuine (small-record wire cost) regression passes green. The + standing gaps in the residuals section (no orphan-snapshot guard; + digestshare liveness unwired) are the same species but only fail to + *alert*, they never assert falsely. +- **Loud-when-bitten latency: R4's test hardcode.** At epoch ≥ 24 it + panics with a missing-handler message — it cannot pass wrongly — and + no current suite reaches that epoch. Annoyance deferred, not a mask. +- **Prose-contradicts-code / discipline-only, no functional exposure: + B2** (reclassified; constructions real, stakes documentary), + **R2** (all three arms unreachable by arithmetic; the fabricated + diagnostic can never be emitted), **R4's doc phrase, R6, seed 3's + `declared` nit, S1, S2, S4.** +- **API-shape and ruling questions, owner's call: R3** (unnameable + `HeadError` — ergonomics and rustdoc integrity, nothing functional), + **R5, `SessionKind`'s openness, R6's gate-leg proposal.** + +## Findings, by severity + +### Bugs + +**B1. The capture renderer recurses without bound on input-controlled +nesting: a crafted (or merely unlucky) application payload overflows the +stack.** — **verified by construction.** + +`MAX_DEPTH = 64` bounds one `parse_node` tree +([capture.rs:327](file:///Users/oxide/src/rumors-review/src/tree/mirror/streaming/remote/codec/capture.rs)), +but `render_embedded_as` re-parses each embedded byte string (tag 24/63) +**at depth 0** (capture.rs:594) and unfolds it through +`render_node → render_tag → render_embedded_as` — one Rust stack frame +chain per embedding level, with no budget that survives the byte-string +boundary. Failure scenario, constructed: a chain of +`24(bstr(24(bstr(…))))` 200,000 levels deep (≈1.4 MB of bytes, buildable +outside-in in one pass) fed to `render_item` → **SIGABRT, "has +overflowed its stack"** (reproduced under nextest). A supply record's +payload is the application's own CBOR, so any snapshot/test corpus whose +payload nests embedded-CBOR tags reaches this through the ordinary +harness path (`render_hook_capture → render_frame → supply run → record +items`). + +This contradicts three in-tree claims: the hard rule ("no traversal +recurses on input-controlled depth"), the module doc's depth-bound +fallback claim (capture.rs:24–26), and the committed test +`nesting_past_the_depth_bound_falls_back`, which exercises only +`parse_node` — exactly the path that *is* bounded — leaving the render +recursion untested. + +Functional-axis statement, to be exact about what is at stake: the +surface is `test`/`test-internals`-gated (never in a shipped artifact), +no committed corpus nests deeply enough to trigger it (the suites' +payloads are flat byte strings and `u64`s), and the trigger is a payload +a test author would have to write — so no shipped functionality and no +current suite is exposed. When it does fire it fires loudly (an abort, +never a wrong render accepted). What keeps it in the bug tier rather +than the prose tier: unlike B2, the violated claim comes with a +demonstrated behavioral failure — a harness that dies on legal input — +and the hard rule demands the bounded-traversal property with a +committed stress test, which does not exist. + +Can a user-chosen payload type `T` reach this class in *production*? +**No** — verified along the whole chain: + +- Every production parse of payload structure delegates to + `ciborium::de::from_reader` (all call sites swept, restated at + `d05a6d03`: the peer-minted `PayloadDeserializer` — the one funnel + both dialects' wire ingress passes through — plus the typed + rehydrators and V1's outer byte-string unwrap, + `message.rs:113/156/178/220`; the version atoms, `tree/wire.rs:205` + and `frame.rs:361`; `bookmark/format.rs:464`), and `from_reader` + constructs its + deserializer with a hard recursion cap of 256 scopes covering array, + map, and tag descent alike (verified in ciborium-0.2.2 source: + `recurse: 256`, `Error::RecursionLimitExceeded` at zero). Peer bytes + nested past the cap yield a typed decode error and a clean session + abort, never unbounded recursion — whatever `T` is, including + `ciborium::Value` itself. +- No shipped code hand-walks CBOR structure recursively: the only + self-recursive walkers in the tree are this renderer (test-gated) and + the wire-legibility test binary. Tag-24/63 embedded byte strings are + never *unfolded* in production — they parse as one flat + `Tag(Bytes)` node — so B1's reset-across-the-boundary mechanism has + no production analogue. +- The tracing adapter (the production-facing consumer) parses through + the same capped `from_reader` and renders under its own shrinking + budgets; worst-case stack is a few hundred frames by construction. +- The encode side recurses only over the user's *own in-memory value* + via their `Serialize` impl — recursion that value's construction and + `Drop` already entail; serde's ownership, not this crate's. +- The one inheritance path: a user who opts into the unstable, + doc-hidden `test-internals` feature and drives `render_hook_capture` + on their own captures inherits B1 as stated. + +This analysis surfaced one adjacent *functional* edge that is out of +this review's range (it predates the stack — payloads were already +CBOR before it): the ciborium decode cap is asymmetric with encode. See +the out-of-range observation at the end of this document. Contrast: the +tracing adapter got this right — its `UNFOLD_BUDGET` decrements across +embedded boundaries (rumors-tracing `render.rs`), and `ciborium`'s own +recursion limit (256, verified in ciborium-0.2.2 source) bounds its +parse. + +> **Resolution** (directly executable, in +> `src/tree/mirror/streaming/remote/codec/capture.rs`): +> +> 1. Thread one combined depth counter through the whole render walk, so +> structural descent and embedded re-parses draw on a single +> `MAX_DEPTH` budget: +> - Add a `depth: usize` parameter to `render_node`, `render_tag`, +> `render_listing`, `render_embedded`, and `render_embedded_as`. +> - Entry points pass `0`: the `render_item` call (capture.rs:283–290) +> and `render_frame`'s body loop (capture.rs:269–278). These already +> call `parse_node(&mut rest, 0)`; keep that. +> - `render_node` passes `depth + 1` wherever it recurses (array +> items, map values, tag content, and the nested-value fallthroughs +> in `render_listing`); `render_tag` passes its `depth` through +> unchanged (it adds no structural level of its own). +> - `render_embedded_as` replaces both of its `parse_node(&mut rest, 0)` +> calls with `parse_node(&mut rest, depth)`, and renders the parsed +> items at `depth + 1`. +> - `render_node` additionally checks `depth >= MAX_DEPTH` at entry +> and, when exceeded, emits the existing hex fallback +> (`fallback(...)`) — this guards the path where a tree parsed just +> under the bound is rendered from a deeper starting point. Note the +> fallback needs the node's exact bytes; the cheapest correct shape +> is to check the budget *before* parsing (i.e. rely on +> `parse_node(depth)` erroring and the caller's existing fallback), +> which the changes above already achieve — the `render_node` entry +> check is then only needed if any call site renders a node at a +> depth greater than the depth it was parsed at; with the wiring +> above no such site remains, so the invariant to keep is: **every +> `render_*` call site passes a depth ≤ the depth its node was +> parsed at**. State that invariant in a comment on `render_node`. +> - The combined bound also caps the per-level `format!("{indent} ")` +> growth (≤ MAX_DEPTH levels × 2 spaces), so no separate fix is +> needed there. +> 2. Update the module doc (capture.rs:22–27): the depth-bound sentence +> must state that the bound spans embedded-byte-string unfolds, not +> only one parsed tree. +> 3. Commit two tests beside `nesting_past_the_depth_bound_falls_back` +> (capture/tests.rs): +> - *Deep embedded chain falls back, never overflows*: build a +> tag-24 chain ≥ 10 × MAX_DEPTH levels deep, outside-in (compute +> each level's length first: `len[0] = 1` for a `0x00` innermost +> uint; `len[i+1] = 2 + cbor::head_len(len[i] as u64) + len[i]`; +> then write tag-24 head + bstr head from outermost to innermost, +> ending with `0x00`). Call `render_item`; assert it returns and +> the output contains the depth-fallback marker text and an +> `h'…'` hex line (the injectivity fallback). +> - Same shape through the frame path: wrap the chain as a supply +> record payload and drive `render_frame`; assert no panic/overflow +> and a fallback line. (This pins the harness-reachable path, not +> just the helper.) +> 4. Acceptance: both new tests pass under nextest; the previously +> constructed 200k-level witness (same construction, larger N) no +> longer aborts; the full capture/tests suite and the three snapshot +> suites re-run byte-identical (`cargo insta` must report **no** +> snapshot changes — this fix must not move any accepted render, +> because no committed corpus nests past the bound). + +### Contract–prose mismatches (constructions verified; no functional impact) + +**B2. The "deterministic encoding enforced on ingress" claim is false at +two layers where decoding delegates to ciborium: non-shortest-form (and +indefinite-length) spellings are accepted.** — **constructions verified, +both; the functional analysis is assessed from the code.** +*Reclassified from the bug tier: no crate functionality rests on +ingress spelling enforcement (analysis below), so what survives is a +prose-vs-code mismatch — the tree claims an ingress guarantee it does +not perform — plus an inconsistent, unstated enforcement boundary.* + +- *B2a — record version atom*: `parse_record` + ([frame.rs:326–369](file:///Users/oxide/src/rumors-review/src/tree/mirror/streaming/remote/codec/frame.rs)) + reads the version-atom *tag* through the canonical head grammar, then + hands the version byte string to `ciborium::de::from_reader` (and the + payload to the peer-minted deserializer, itself a plain + `from_reader`), both of which accept widened and indefinite-length + heads. Constructed (at `a5c4ca1a`; the parse structure is unchanged + at `d05a6d03`): a record whose version bstr head is spelled + `0x59 0x00 len` instead of the canonical short head decodes `Ok` + through `LeafRun::from_encoded` + `records(...)`. This contradicts + the codec module doc ("decoding rejects any other spelling", + codec.rs:8) and `LeafRun`'s own doc ("a records iterator therefore + never fails structurally, only on a record's canonical content", + frame.rs:76–78). +- *B2b — bookmark payload*: `unframe` enforces shortest-form heads on + the frame, but `walk` + ([format.rs:462](file:///Users/oxide/src/rumors-review/src/bookmark/format.rs)) + parses the payload with `ciborium::Value`. Constructed: + `decode(&frame(&[0xbf, 0xff]))` — the record map spelled as an + *indefinite-length* map, a spelling this codec never writes, with a + correct hash over those bytes — decodes `Ok` to the empty record. This + contradicts format.rs:39–42 ("the decoder rejects any other + spelling"). + +Why no functionality rests on this — what each half of the contract +actually carries: + +- **Encoder-side determinism (intact) is what licenses the byte-pinning + snapshot discipline**: equal semantics ⇒ equal bytes. Ingress plays no + part in that, and the renderer's injectivity argument is likewise + self-contained (its own walk falls back to exact hex on non-canonical + heads). +- **Replica correctness is spelling-independent.** No wire framing byte + is stored or hashed: payload bytes propagate verbatim from the + originator to every replica (the decode adapter retains the exact + slice; re-supply writes it back unchanged), so message identity sees + one byte sequence per message regardless of the framing around it. + Version and party atoms are decoded to values and re-encoded + canonically from the value on every hop — `Version::decode` enforces + before's bit-level coding, which *is* semantics-bearing and *is* + enforced — so a widened CBOR head around an atom does not survive one + hop, and cannot perturb paths, hashes, or convergence. +- **What ingress structurally must reject, it still rejects.** + Indefinite-length and reserved heads are unparseable by the + exact-read/O(1)-skip machinery and die wherever the protocol + hand-parses; listing order is enforced as the protocol walk's own + invariant. Both predate and stand apart from the CBOR determinism + contract, and both are intact. +- **What remains for ingress spelling enforcement is conformance + diagnosis only** — catching a hypothetical third-party encoder that + emits widened spellings — and per the model of record that machinery + is a bug detector, never a boundary. There is also a + circular-justification argument against building it: this crate's own + encoder cannot produce the spellings such a check would catch, so + against the only implementation that exists the detector never fires. + +The residue whichever way one leans: the enforcement boundary is +currently *inconsistent* — frames, signals, listings, record framing, +greeting, preamble, and the bookmark frame all reject non-shortest +spellings; the record's version-atom head and the bookmark payload do +not — and no prose states where the boundary sits. The tree must say +what IS. + +> **Resolution** (primary; directly executable; docs plus acceptance +> pins, zero behavior change): +> +> 1. Re-scope the three claim sites to state the actual contract — +> determinism is the *encoder's* promise; ingress promises structure +> and definite lengths, with spelling additionally judged only where +> the codec hand-parses: +> - `src/tree/mirror/streaming/remote/codec.rs:6–10`: replace the +> "and decoding rejects any other spelling" clause with words to +> the effect of: "The wire is *emitted* as deterministic-encoding +> CBOR — shortest-form heads, definite lengths, one spelling per +> value — which is what keeps the byte-pinning snapshot discipline +> meaningful. Ingress validates structure everywhere; every head +> the codec hand-parses additionally rejects indefinite lengths +> and non-shortest spellings, while a record's version atom and +> application payload are decoded by a general CBOR reader that +> judges neither — the atom's *content* canonicality is enforced +> by its own strict decoder." **(Amended after the round-1 fix +> review: this resolution's first wording claimed "definite +> lengths everywhere", which the general-reader positions refute +> by construction — an indefinite-length payload map and an +> indefinite-length version-atom byte string both decode Ok. The +> fix round's F1 corrects the transcribed sentence and pins both +> indefinite witnesses. Any roster of hand-parsed positions must +> be complete — preamble and greeting included — or omitted.)** +> - `frame.rs` `LeafRun`/`records` doc (:73–78): replace "only on a +> record's canonical content" with "only on a record's content — +> a version atom whose content bytes fail the strict `Version` +> decoder (its CBOR byte-string head is read by a general CBOR +> parser and not re-judged for shortest form), or an application +> payload that does not decode". +> - `src/bookmark/format.rs:39–42`: replace "and the decoder rejects +> any other spelling" with "the frame's own heads are rejected in +> any other spelling; the embedded payload is decoded by a general +> CBOR reader, so its one-spelling property is the encoder's +> (equal records produce equal files), not an ingress check". +> 2. Pin the boundary so it cannot drift silently in either direction — +> commit this review's witnesses as *acceptance* tests whose doc +> comments cite the scoped contract: +> - In `src/tree/mirror/streaming/remote/codec/decode/tests.rs`, +> beside `supplied_record_errors_are_typed`, using its +> `raw_record` helper: build record content = version tag head +> (`cbor::write_head(MAJOR_TAG, crate::tags::VERSION_TAG)`), then +> the widened byte-string head `0x59 0x00 ` followed by the +> canonical atom bytes of `Version::new()` (obtain them by +> ciborium-serializing `Version::new()` and stripping its first +> byte, whose low bits are the length), then a ciborium-serialized +> `0u64` payload. Assert `LeafRun::from_encoded(raw_record( +> &content)).unwrap().records(Message::deserializer::()) +> .next().unwrap()` is `Ok` (the minted deserializer is how every +> test in that file drives `records` at `d05a6d03`). Doc +> comment: "Pins the stated ingress boundary: the version atom's +> CBOR head is not spelling-judged (the atom's content is, by +> `Version::decode`); flipping this to rejection is a deliberate +> contract change, not drift." +> - In `src/bookmark/format/tests.rs`: +> `decode(&frame(&[0xbf, 0xff]))` is `Ok` and empty — same doc +> pattern ("the payload's spelling is not ingress-judged; the +> hash binds bytes, the frame binds shape"). +> 3. Acceptance: all suites green; **zero** snapshot movement; +> `grep -rn "any other spelling" src/` returns nothing. +> +> **Alternative (owner-gated): enforce a uniform ingress boundary +> instead.** The case for it is conformance diagnosis with precedent — +> the signal-redundancy ruling paid recurring wire bytes to keep the +> `Mislabeled` detector — and the case against is stated above (the +> detector would never fire against any existing implementation). If +> Finch rules for enforcement, record the ruling in +> `design/cbor-legible-wire.md`'s decision record and implement: +> +> - *B2a, in `frame.rs` `parse_record`*: replace the ciborium parse of +> the version with the hand-parse the greeting already uses +> (greeting.rs:152–174 is the model): after the version-tag check, +> `cbor::read_head` again; require `MAJOR_BSTR` (else +> `DecodeLeafError::Version(InvalidData)`); `HeadError::Truncated` → +> the existing `UnexpectedEof` version error, other `HeadError`s → +> `InvalidData` with the head error's Display; `usize::try_from` the +> length, split the atom off `input` (shortfall → `UnexpectedEof`), +> decode via `Version::decode` with the error mapping `decode_party` +> uses (party.rs:114–126). The payload keeps its minted-deserializer +> parse. The +> step-2 witnesses above then invert into rejection assertions +> (`Err(DecodeLeafError::Version)`, `InvalidData`), plus a canonical +> control case asserting `Ok` against over-tightening. +> - *B2b, in `src/bookmark/format.rs`*: re-encode-and-compare at the +> trust boundary: extract `encode`'s map-building + +> `ciborium::ser::into_writer` body into a private +> `fn record_payload(record) -> Vec` (one spelling authority, +> called by `encode` too); in `decode`, after `walk` succeeds, +> compare `record_payload(&record)` to the payload slice; mismatch → +> new `RecordDefect::NonCanonical` variant (additive; the enum is +> `#[non_exhaustive]`), docstring "the payload decodes, but not from +> the one spelling this codec writes". This closes widened heads, +> indefinite lengths, and key order in one total check. Witnesses +> invert to rejection; add a widened-map-head case (re-spell +> `sample_record()`'s payload head `0xa1` as `0xb8 0x01`, re-frame). +> - Acceptance either way: zero snapshot movement (encoders untouched); +> `wire_legibility`, `observe`, and the corruption/truncation sweeps +> green. + +### Risks + +**R1 (seed 4). The minimal-record calibration cell sits exactly at the +±2 tolerance edge, and the band's stated rationale is wrong.** — +**verified** (ran the cells: minimal implied **50** vs expected **52**; +mid 107/107 and design 215/215 exact). +`TOLERANCE_BYTES`'s doc +([dispute_wire.rs:99–101](file:///Users/oxide/src/rumors-review/tests/dispute_wire.rs)) +says the counts are deterministic and the slack "only absorbs +integer-division adjacency"; window.rs:215–218 correctly documents the +2 B residual as *systematic* (denser batching at small records). The two +rationales contradict, and the operational effect is real: a genuine ++1..+4 B per-message regression confined to small records moves the +implied value to 51..54 and stays inside the band — invisible — while +the design cell (payload-dominated) stays green. The affine-law claim +("three collinear points") is also not literally true: the minimal point +is 2 B off the line. + +> **Resolution** (directly executable, in `tests/dispute_wire.rs` and +> `src/tree/mirror/streaming/window.rs`): +> +> 1. In dispute_wire.rs, split the band into what it actually absorbs: +> - `const TOLERANCE_BYTES: usize = 1;` with its doc reduced to the +> integer-division-adjacency sentence only. +> - New `const MINIMAL_CELL_RESIDUAL: usize = 2;` documented as: "the +> minimal cell reads this many bytes *under* the intercept — +> small records batch more densely, so their share of per-frame +> framing is smaller (the mechanism is stated at +> `DISPUTE_OVERHEAD_BYTES`); a measured value moving off +> `intercept + payload − MINIMAL_CELL_RESIDUAL` by more than the +> division tolerance is a real framing change, in either +> direction." +> - `minimal_records_pin_the_fixed_overhead`: `expected = +> fixed_overhead_bytes() + U64_ENCODED_BYTES - +> MINIMAL_CELL_RESIDUAL` (= 50), tolerance ±1. +> - The mid and design cells keep their current `expected` values, +> tolerance now ±1 (they sit exact today; verify by running — +> expected prints are `107` and `215`). +> - Reword the module doc's "three collinear points" linearity +> sentence: the law is affine over the interior and design points, +> with a stated, pinned −2 B residual at the minimal end. +> 2. In window.rs:215–218, point the parenthetical at the per-cell +> constant instead of "the cells' tolerance band absorbs" (e.g. +> "pinned as the minimal cell's own residual constant in +> `tests/dispute_wire.rs`"). +> 3. Acceptance: `cargo nextest run --test dispute_wire` green; +> mutation check by hand: temporarily add 1 to the minimal cell's +> measured value (e.g. `implied + 1` in the assertion) and confirm +> the test now fails — then revert. All three cells' printed +> `implied` values unchanged (50, 107, 215). + +**R2 (seed 8, first clause). Three defensive arms in `decode_v2` are +unreachable, none documented as such — the charter's disclosure +("documented without overclaiming") is not true of the tree.** — +**verified by arithmetic + grep.** *(Correction to this review's first +issue: I initially reported two dead arms; writing the resolution +surfaced a third.)* +`decode_v2` ([handshake.rs:219–271](file:///Users/oxide/src/rumors-review/src/tree/mirror/handshake.rs)) +always receives exactly 30 bytes (`Staged::validate` slices `want`, +which is `V2_PREAMBLE_LEN`). Passing the version check forces a one-byte +version head (`0x02`; wider spellings of 2 are `NotShortest` → +`PreambleDefect::Version`, other values → `VersionMismatch` before the +network parse); passing the network filter forces the one-byte head +`0x50`. So consumption before the intent item is always 13 + 16 = 29 +bytes, leaving exactly one: +- `input.len() < NETWORK_LEN` (line 254, `NetworkTruncated`) can never + fire; +- the intent item is one byte, so `intent.value ≤ 23` and + `u8::try_from` (line 267) can never fail — and if it ever did, it + fabricates `IntentInvalid { byte: 0xff }`, a byte the peer never sent; +- after a one-byte intent, `input` is always empty, so the + `TrailingBytes` arm (line 264) can never fire either. +Grep finds no documentation of any of the three. + +> **Resolution** (mostly directly executable; the variant-removal +> alternative is **owner-gated**): +> +> 1. Intent width arm (handshake.rs:267–268): per the crate's +> disposition ladder this is a truly-unreachable branch that must +> structurally exist → make it assert. Replace the +> `.map_err(|_| Error::IntentInvalid { byte: u8::MAX })` with +> `.expect("the 30-byte preamble leaves exactly one byte for the +> intent item, whose one-byte head's value is at most 23")`, keeping +> `.and_then(Intent::from_byte)` → adjust to +> `Intent::from_byte(u8::try_from(intent.value).expect(...))?`. This +> removes the diagnostic fabrication outright. +> 2. `NetworkTruncated` and `TrailingBytes`: keep the arms (the slice +> bound is load-bearing for `split_at`, and the trailing check guards +> any future caller with non-fixed input), and document the +> defensive status in both places: +> - On each variant's rustdoc +> (handshake.rs:333–335 and :341–343): one sentence of the form +> "Defensively reachable only: the fixed 30-byte V2 preamble with a +> validated version and network head always leaves exactly +> 16 bytes / no trailing byte; this variant guards the width +> arithmetic against future layout drift, not any input the current +> dialect admits." (State it positively and undated, per the +> no-ghost-references rule.) +> - A short comment at each construction site restating the same +> derivation in one line. +> 3. Add the missing reachable-defect constructions (they exist for +> `Intent` only) as unit tests in +> `src/tree/mirror/handshake/tests.rs`, using the existing `staged` +> helper pattern: +> - `PreambleDefect::Version`: a 30-byte frame with byte 11 (the +> version item) spelled `0x38` (a major-1 head) — assert +> `Err(Error::Malformed { defect: PreambleDefect::Version })`; and +> a second case with bytes 11–12 = `0x18 0x02` (widened spelling of +> 2) asserting the same (this is the determinism witness at this +> layer). +> - `PreambleDefect::Network`: byte 12 = `0x51` (bstr of length 17) — +> assert the `Network` defect. +> - For the two defensive variants, add an explicit exemption note in +> the test module (the error-atlas `EXEMPT_MARKERS` pattern is the +> house style): a comment block naming both variants and the width +> derivation, so an auditor sees the absence of constructions is +> deliberate. +> 4. **Owner-gated alternative**: delete `NetworkTruncated` (and fold +> its check into an `expect`), shrinking the just-ruled public +> `PreambleDefect` enum. Do not take this path without Finch's +> ruling; the enum is `#[non_exhaustive]` and was owner-shaped in +> `5367a82e`. +> 5. Acceptance: handshake tests green including the new constructions; +> `intent_byte_space_is_exhaustive` and +> `arbitrary_preamble_decodes_by_the_oracle` unchanged and green; +> `cargo clippy --workspace --all-targets --all-features -- -D +> warnings` clean. + +**R3 (seed 7). `LeafRunError::Head`'s `source` field is publicly +unnameable.** — **verified.** +`LeafRunError` is re-exported at +[error.rs:41](file:///Users/oxide/src/rumors-review/src/error.rs); +its `Head { source: HeadError }` payload type is `pub` only inside the +crate-private `tree::mirror::cbor` module and re-exported nowhere. A +user can bind `source` and use it via `std::error::Error`, but cannot +write its type, match its variants (`Truncated`/`Indefinite`/`Reserved`/ +`NotShortest` — precisely the taxonomy the determinism contract makes +interesting), or follow the rustdoc link (it renders dead). + +> **Resolution** (directly executable; additive API, so note it in the +> PR description for Finch's review pass): +> +> 1. Re-export the type along the same path its carrier travels: in +> `src/tree/mirror/streaming/remote/codec.rs`'s public export block +> (the `pub use error::{...}` / `pub use frame::{...}` cluster at +> :84–96), add `pub use crate::tree::mirror::cbor::HeadError;` — then +> it rides the existing +> `pub use crate::tree::mirror::streaming::remote::{...}` list in +> `src/error.rs:38–44`; add `HeadError` to that list. (Route through +> the codec module rather than exporting `cbor` itself: the head +> grammar is codec vocabulary; the `cbor` module stays private.) +> 2. Check `HeadError`'s rustdoc reads as public API (it already +> documents each variant); add `#[non_exhaustive]` **only if** Finch +> wants room for future head-grammar defect classes — default: leave +> it exhaustive, since RFC 8949's head grammar is closed +> (**owner-gated** either way, one line). +> 3. Acceptance: a doc build (`just gate`'s docs leg, or +> `cargo doc -p rumors --no-deps`) shows `LeafRunError::Head`'s +> `source` type as a live link; `rumors::error::HeadError` is +> nameable from an integration test (add one line to any existing +> tests/ suite that imports it and matches `HeadError::NotShortest` +> to pin the reachability). + +**R4. The stream-open label is documented as "two-byte" but is 2–4 +bytes, and `tests/observe.rs` hardcodes the two-byte shape.** — +**verified by reading.** +The label is two CBOR uint items (streams.rs:59–69); an epoch ≥ 24 +encodes in two bytes, so a link past its 24th session writes a 3-byte +label. [observe.rs:101](file:///Users/oxide/src/rumors-review/src/observe.rs) +("the two-byte stream-open label") is wrong for that case — the design +doc's own table says "+0–1 bytes per stream", so the doc knows better. +`tests/observe.rs:189/196/220/226` index `blob[1]` and slice `blob[2..]` +— correct for the epochs these tests reach, wrong at epoch ≥ 24. The +failure mode when it bites is loud, not masking: `blob[1]` would read +the epoch's value byte (≥ 24, outside the 0..17 stream-index range), the +handler lookup fails, and the test panics with a missing-handler +message — it cannot pass wrongly. No production code depends on the +two-byte assumption (the snapshot harness parses labels through +`stream_label`). + +> **Resolution** (directly executable): +> +> 1. `src/observe.rs:101`: replace "the two-byte stream-open label" with +> "the stream-open label (two leading unsigned-int items)". Check +> the same phrase does not recur elsewhere +> (`grep -rn "two-byte" src/ crates/rumors-tracing/` and fix any +> other label reference the same way; the phrase at framing.rs, if +> any, refers to the V1 length header and is unrelated). +> 2. `tests/observe.rs`: import `stream_label` from +> `rumors::testing` (the file already imports from +> `crate::common::gossip_snapshot`, which re-exports the testing +> door; either path works). In `assert_mirrors` (:184–199) and +> `assert_received_mirrors_remote` (:219–229) replace the +> `blob[1]` / `&blob[2..]` pairs with: +> `let ((_, index), label_len) = stream_label(blob);` and +> `&blob[label_len..]`. Drop the now-redundant `blob.len() >= 2` +> assertion (stream_label panics with a named reason on a malformed +> label, which is the harness contract). +> 3. Acceptance: `cargo nextest run --test observe` green; +> `grep -n "blob\[" tests/observe.rs` returns nothing. + +**R5 (seed 8, second clause). The io::Error collapse one layer below +the handshake: assessed, and I recommend extending the ruling.** — +**owner-gated (public error surface).** +The disclosed sites hold up as disclosed: +[party.rs:141](file:///Users/oxide/src/rumors-review/src/tree/mirror/party.rs) +and [greeting.rs:305](file:///Users/oxide/src/rumors-review/src/tree/mirror/streaming/remote/codec/greeting.rs) +map malformed-CBOR heads to `io::Error(InvalidData)` (party's whole +malformed surface rides `Error::Io`; the greeting's rides the mirror's +`HandshakeDecode`). The current state is *self-consistent* — the +`Error::Io` table row explicitly documents "a wire framing fault outside +the streaming mirror (counterparty bug: report it)" — but it is not +consistent with the ruling's principle: an application cannot +programmatically distinguish "transport died" from "counterparty sent a +malformed identity hand-off" without string-sniffing `ErrorKind`, and +that is the same diagnosis the owner ruled worth typing at the preamble. +The hand-off especially is high-stakes (identity in flight; the +retire/bootstrap recovery guidance differs between the two causes). + +> **Resolution** (two steps; step 1 is the executable one): +> +> 1. Put the decision to Finch, framed as: "the 2026-08-19 error-taxonomy +> ruling typed the preamble's failures; the identity hand-off and the +> greeting still collapse malformed-peer-bytes into +> `io::Error(InvalidData)`. Extend the ruling one layer, or record a +> scoping decision?" Record the outcome in +> `design/cbor-legible-wire.md`'s decision record either way. +> 2. If extended, the implementation shape (sized M, no wire change): +> - New public `HandOffDefect` enum (mirroring `PreambleDefect`'s +> style): `NotPartyTagged`, `NotAByteString`, `UnaddressableLength`, +> `HeadMalformed(HeadError)` (nameable once R3 lands), +> `Undecodable` (carrying the `before::error::Decode` rendered or +> typed). New `Error::HandOffMalformed { defect }` and +> `Error::HandOffTruncated { }` variants (the promised-hand-off +> close; today `UnexpectedEof` under `Io`). Wire them in +> `party.rs::receive_v2`/`read_head`/`decode_party`; the V1 path +> keeps `Error::Io` (frozen dialect). +> - Greeting: replace `ReadGreetingError::Decode(io::Error)`'s +> payload with the existing typed `GreetingError` (it already +> enumerates every failure; today it is flattened to a string at +> greeting.rs:277–281), and surface it through the mirror's +> `HandshakeDecode` variant as a typed source instead of +> `io::Error`. +> - Update the `Error` doc table rows; update +> `tests/common/sim.rs::is_honest_error` (malformed stays +> dishonest; a typed hand-off *truncation* joins +> `PreambleTruncated` as honest-cut); sweep +> `src/tree/mirror/party/tests.rs` (its `io_error` helper and the +> typed-error assertions move to the new variants). +> - Acceptance: party and greeting test suites re-pin every failure +> to its typed variant; the disruption suite green (a new +> committed seed, if one shakes out, is committed per policy); +> `just gate` clean. + +**R6. `TAG_SELF_DESCRIBED` is dead code in the default-features lib, and +55799 is spelled three independent ways.** — **verified** (`cargo check +-p rumors` warns "constant `TAG_SELF_DESCRIBED` is never used"; the +gate's clippy legs never observe that configuration — `--tests` keeps +the capture module alive — so the gate is green while a bare check of +the shipped lib warns). +The production preamble spells 55799 as the `V2_PREFIX` byte literal +(handshake.rs:78), the bookmark as its own `SELF_DESCRIBED` byte literal +(format.rs:72), and the constant is used only by the test-gated +renderer. Even the pin test `prefix_matches_the_writers` writes the +literal `55799`, not the constant. + +> **Resolution** (directly executable except the gate-leg question): +> +> 1. Gate the constant to its users: in +> `src/tree/mirror/cbor.rs:45–47`, add +> `#[cfg(any(test, feature = "test-internals"))]` on +> `TAG_SELF_DESCRIBED` (matching the capture module's own gate at +> codec.rs:60–61). +> 2. Tie all three spellings to one authority by committed tests: +> - `src/tree/mirror/handshake/tests.rs::prefix_matches_the_writers`: +> replace the literal `55799` with `cbor::TAG_SELF_DESCRIBED`. +> - `src/bookmark/format/tests.rs`: add a test asserting +> `SELF_DESCRIBED` equals the rendering of +> `cbor::write_tag(&mut v, cbor::TAG_SELF_DESCRIBED)` (3 bytes), +> doc: "the bookmark's opening literal is the self-described tag's +> one canonical spelling; the shared constant is the authority." +> (`SELF_DESCRIBED` is private to `format.rs`; the test module is +> its child and sees it.) +> 3. Acceptance: `cargo check -p rumors` (no flags, default features) +> emits **zero** warnings; `just gate` clean; the two pin tests +> green. +> 4. **Owner-gated** follow-up worth proposing while there: a +> `clippy-default` leg that lints the bare lib +> (`cargo clippy -p rumors --lib -- -D warnings`, without `--tests`), +> so dead code in the shipped configuration cannot recur unseen. One +> justfile line; Finch's call because it lengthens the gate. + +### Behavioral notes (bytes unchanged, behavior worth knowing) + +**N1 (seed 6). Query listings and frame heads now read per-item against +the transport.** — **assessed.** +The old wire's length-framed bodies arrived in one bulk read; the V2 +decoder reads each head byte-by-byte (`read_head_async`: a 1-byte read +plus the extension) and each digest in a 32-byte `read_exact`. A +full-fan query is on the order of a thousand small reads on an +unbuffered transport. The greeting was deliberately embedded (tag 24) to +avoid exactly this ("no incremental map walk happens against the +transport", greeting.rs:6–8) — data-stream queries walk incrementally. +The documented mitigation stands (framing.rs:19–24: wrap raw sockets in +`BufReader`; caller-owned buffering is session-safe). + +> **Resolution** (optional, docs-only, directly executable): the +> BufReader guidance currently lives in a crate-private module doc +> (framing.rs). Surface one sentence where a `Link` builder will see it: +> in `src/link.rs`'s rustdoc (the transport-contract section), add +> "Reads are exact and item-granular; on an unbuffered transport, wrap +> the read half in `tokio::io::BufReader` — caller-owned buffering +> outlives a session and is safe across session boundaries." Acceptance: +> `just readme` re-derived if the crate-level docs changed (they should +> not); docs leg green. If left undone, no invariant is at risk. + +--- + +## Seed dispositions (charter's 14, in order) + +1. **Red docs-only gate pair** — verified: `d6262c03` touches only + `///`/`//!`/comment lines (mechanically checked); nothing non-docs + rode in. *No action.* +2. **Bisection** — verified: all 12 commits in `15bd905e..a5c4ca1a` pass + `cargo check --all-targets --all-features`. The adapter branch's + disclosed broken merge (`7403bd72`) stands as disclosed; its tip + builds. *Rebase addendum*: the rebase to `d05a6d03` replayed these + commits and the rebasing agent gated only the tip plus its three + conflict stops, so the verdict was re-established rather than + carried over — all 12 rebased commits in `991f4663..d05a6d03` pass + the same check sweep (**verified**, post-rebase). *No action.* +3. **`SUPPLY_FRAME_OVERHEAD` envelope** — verified against the encoder: + the envelope charges heads at their widest (10 B; actual 6–10), the + encoder's `admits` and the decoder's `covers` share the single + `covers` boundary so they cannot drift, `record_len` is pinned + against an actual `push`, and the flush loop + (adapter/encode.rs:225–239) checks before every post-first push. + Under-batching is bounded at 4 B/frame and documented. One nit: the + `OverbatchedRun.declared` field reports the *charged* envelope, + overstating the actual frame by up to 4 B while reading as a wire + fact. + > **Resolution** (directly executable, docs-only): on + > `DecodeErrorKind::OverbatchedRun` + > (codec/error.rs:148–151), add a field doc on `declared`: "the + > frame's charged wire size — its run body plus the + > `SUPPLY_FRAME_OVERHEAD` envelope at its widest — which may exceed + > the actual frame by the envelope's head slack (at most 4 bytes)"; + > and adjust the `#[error]` text from "occupies {declared} wire + > bytes" to "charges {declared} wire bytes". Acceptance: error-atlas + > suite green (its markers match on the prefix "kind: + > OverbatchedRun(declared=", which does not change). +4. **Minimal-cell ±2 band** — broken as suspected; finding **R1**. +5. **Mutants roster** — verified untouched: every exclusion predates the + stack and names suanpan/before skyline code; nothing excludes the new + codec/bookmark/hook code, so no in-diff green-washing. Whether the + new code's mutants die is unmeasured — see the residual-risks + resolution (mutants campaign) below. +6. **Per-entry query decode** — finding **N1**. +7. **`LeafRunError::Head` visibility** — finding **R3**. +8. **Handshake taxonomy** — every `PreambleDefect` variant has exactly + one construction site in code, and the disruption honesty classifier + admits the typed truncation correctly (sim.rs). But three of the + decoder's defensive arms are unreachable and undocumented — finding + **R2** (which also carries the missing-construction resolutions). + Sibling collapses — finding **R5**. +9. **"Uncontended by construction" mutexes** — verified by reading every + `control_sent`/`control_received` site: per direction the writers are + protocol-sequential phases owning the half exclusively (preamble → + greeting → hand-off → epilogue), and the only concurrency (preamble + write ∥ read; greeting send ∥ receive; epilogue send ∥ receive) pairs + *different* mutexes. The locks are leaf locks, held only across the + synchronous `message()` call, no await under lock. *No action.* +10. **Renderer panic/failure split** — principled and documented at the + module boundary (capture.rs:40–45). Assessed clean — except that one + fallback trigger (the depth bound) does not hold on the + embedded-unfold path: finding **B1**. +11. **No session ordinal** — verified: `SessionInfo` carries none, the + library holds no numbering state, the hook docs teach the + observer-side atomic pattern, and the adapter implements it + correctly (peer-level `AtomicU64` for session ordinals, per-session + shared `Arc` for message ordinals, ordinal advanced even + when the subscriber is disabled). Two nits, resolved under + Simplifications S2 (the `bootstrap.rs:170` prose vestige) and + Missing tests T2 (concurrent-session numbering untested). +12. **Extractor re-accept** — verified structurally: `0db57d00` touches + no wire-producing file (renderer, its tests, harness, test surface, + and exports only), so wire bytes could not move in it; the totality + oracle (`assert_items_account_for`, applied per stream to control + and every data stream, both endpoints, with a count check that no + hook stream lacks transport bytes and vice versa) re-attests + items==wire on every run, and the received directions are separately + held to the peer's sent capture in tests/observe.rs. Attacks on the + oracle's coverage came up empty except B1's renderer robustness and + the injectivity analysis: I could not construct two byte streams + rendering identically — canonical-head enforcement in `parse_node`, + exact-width float/simple preservation, escaped text, full hex for + byte strings, and explicit-hex fallbacks close the paths I tried + (arity is recoverable from body presence; embedded contents show + byte counts). Injectivity: assessed sound, modulo B1. *No action + beyond B1.* +13. **Orphaned snapshots** — verified: a name-resolution sweep over all + 24 `tests/snapshots/*.snap` plus the src-side insta snapshots finds + every one referenced by its generating test. The corpus still has no + *standing* defense. + > **Resolution** (directly executable): add + > `tests/snapshot_liveness.rs`, modeled on + > `tests/seed_liveness.rs` (same doc-comment style: the invariant, + > the resolution rules transcribed, the provenance note): + > - Walk `tests/snapshots/*.snap`; for each, split the stem on the + > first `"__"` into `(suite, name)`; require `tests/.rs` + > to exist and to contain either `fn (` or the quoted + > string `""` (explicitly named snapshots — the bookmark + > pins use this form). + > - Walk `src/**/snapshots/*.snap`; for each + > `rumors____tests__.snap`, map `` to + > `src//tests.rs` and apply the same + > containment rule. + > - Fail with the orphan's path and the file searched. Skip + > nothing; `.snap.new` files (pending insta output) fail loudly + > as "unaccepted snapshot committed". + > - It is an ordinary integration test, so `just test-all` (and + > therefore the gate) runs it with no justfile change. + > - Acceptance: green on the current tree; delete-resistant + > demonstration: temporarily add a stray + > `tests/snapshots/gossip_snapshot__nonexistent.snap` and confirm + > it fails naming that file — then remove it. + > Rejected alternative, for the record: + > `cargo insta test --unreferenced=reject` is the mature-tool path + > but couples the gate to the cargo-insta runner and a full-suite + > invocation; the pairing test matches the tree's existing + > seed-liveness pattern and runs in the same harness. If Finch + > prefers the tool, wire `cargo insta test --unreferenced=reject` + > as a distinct justfile recipe in the gate's test tier instead — + > **owner-gated** because it is a gate/tooling change. +14. **Known pin gaps** — confirmed as disclosed, with one sharpening and + one refutation: the session corpus contains no nonempty Query frame + (only `QueryEmpty`; nonempty listings appear only inside greetings), + *but* the codec-level `canonical_frame_atlas_snapshot` byte-pins a + one-child Query frame at every placement, so the gap is corpus-level + composition, not spelling. digestshare is confirmed absent from the + justfile. The depth-64 bound does **not** hold on every walk path — + refuted by construction (**B1**). + > **Resolution, query fixture** (directly executable): add one + > gossip-snapshot test (in `tests/gossip_snapshot.rs`, alongside + > `deep_trie_divergence`) whose corpus provokes a nonempty Query + > frame. Construction guidance: a nonempty Query needs a disputed + > interior node whose reply *lists children* rather than shipping + > or pruning them — two peers sharing a common prefix under which + > **both** sides hold children (≥ 2 shared radixes) with differing + > content on at least one, so neither side's subtree is absent (an + > absent side yields supplies; identical subtrees yield matches; + > an empty listing yields `QueryEmpty`). Iterate the corpus until + > the accepted snapshot's render contains a signal line matching + > `/ Query(` **and** a body line matching + > `{ / listing: child(ren) /` with n ≥ 1 — assert exactly that + > in the test body *before* the `insta::assert_snapshot!`, so the + > fixture cannot silently degrade back to `QueryEmpty` under a + > future corpus change (the liveness floor for this pin). Accept + > the new snapshot; the re-accepting commit names the new fixture + > (it adds a snapshot; it re-accepts nothing existing). + > **Resolution, digestshare liveness**: see Residual risks below + > (**owner-gated** gate change). + +--- + +## Round-4 assumption checks not covered above + +- **O(1) record and run skip under tag 63** — assessed sound: + `record_head` reads exactly two heads and returns the content length; + `RecordSlices::next` jumps by length; nested containers inside a + record's payload are never walked during skip or structural + validation (`from_encoded` chains lengths only). *No action.* +- **Charge-before-custody** — verified by reading both supply paths + (adapter/decode.rs:168–182 and 372–388): `ledger.charge(1)` lands + after structural parse, before `Leaf::leaf` takes payload custody, + per record, while the reply is open. The overbatch gate rejects + before buffering (both decoders, mirrored logic, shared + `lone_record_spans` boundary), with committed corner classification. + *No action.* +- **Budget memory argument** — verified by reading: runs stay encoded on + both sides; the decoder yields records one at a time into a bounded + channel; the only whole-run decoded materialization is the test-gated + renderer. *No action.* +- **Deadlock freedom** — assessed: `src/link.rs` is byte-untouched in + the range; the hook adds no protocol dependency (handlers are + synchronous, per-stream, invoked outside any protocol lock; control + observers are leaf mutexes; a blocking handler stalls only its own + directed stream, as documented). *No action.* +- **Greeting key order** — verified: the `KEYS` roster is exactly CBOR + deterministic (length-first bytewise) order, and `parse_greeting` + demands the exact roster in that order. *No action.* +- **Preamble fixed width** — verified: 11 + 1 + 17 + 1 = 30, pinned to + the writers by `prefix_matches_the_writers`. *No action.* +- **Merge seam (`9aff980b`)** — verified: empty combined diff (no manual + conflict resolution); disjoint file sets except `lib.rs`; the merged + tree's cross-references compile and pass. *No action.* +- **V1 frozen** — verified: `crates/before`, `src/link.rs`, all 28 + alternating snapshots and the V1 codec untouched; the single V1-side + test change is a mechanical adaptation to the shared handshake's new + signature, not a behavior change. The charter's "all V1 tests + untouched" is inexact on that one file; the wire claim ("zero byte + movement") holds. *No action.* +- **Wire × hook byte identity** — verified by the committed differential + (`observation_never_changes_the_wire`) plus reading the capture path + (received items are transport bytes via `CaptureRead`; the one + re-encoding — the over-budget lone-record prefix — is byte-identical + because heads are canonical-enforced; sent items are the exact buffers + just flushed). *No action.* + +## Public API deltas — judgment + +- `rumors::observe` — well-shaped: three levels mirror the session + machinery, bytes-only contract keeps it rumors-blind, `SessionInfo`/ + `StreamInfo`/`StreamId` are `#[non_exhaustive]`, decline-at-any-level + is cheap, and the rustdoc is genuinely at the crate's standard. Two + nits: + - `SessionKind` is *not* `#[non_exhaustive]` although it enumerates + lifecycle operations the crate could plausibly grow; the crate's + recent practice (`1e458d69`) marks open diagnostic taxonomies + non-exhaustive. + > **Resolution** (**owner-gated**, one attribute): ask Finch whether + > `SessionKind` is an open taxonomy. If yes, add + > `#[non_exhaustive]` to it (src/observe.rs:207) *now, pre-release* + > (adding it later is the breaking change), and confirm no in-tree + > exhaustive `match` on it exists outside the crate boundary + > (grep shows none today; the tracing adapter formats it with + > `?kind` and never matches). If no, record one sentence at the + > enum ("closed by design: a session is entered by exactly these + > operations") so the asymmetry against the crate's convention + > reads as deliberate. + - `Observer::session`'s "called once per session" reads as + unconditional while V1 sessions never call it (the module doc states + the exclusion). + > **Resolution** (directly executable): in the method doc + > (src/observe.rs:130–134), append "— for sessions of an observable + > dialect; see the module docs' `Protocol::V1` exclusion." +- `Peer::observe` / `Bootstrap::observe` / `Bootstrap` losing `Copy` for + `Clone` — right call, honestly documented, retry affordance + preserved. The bootstrap-ordinal prose vestige is resolved under S2. +- `rumors::tags` — minimal and correctly scoped; provisional-numbers + caveat and serde-placement rule stated at the module. The + triple-spelling of 55799 is resolved under R6. +- `BOOKMARK_MAGIC` removed, `BOOKMARK_FORMAT_VERSION` u16→u64 (=4), + `FormatError` restructured with typed `FrameDefect`/`RecordDefect` — + coherent, meaning-named, `#[non_exhaustive]` where open; docs + excellent. `Error::VersionMismatch.remote_version` u16→u64 matches + the V2 preamble's uint field. *No action.* +- `PreambleDefect` + `PreambleMalformed`/`PreambleTruncated` — good + shape and docs; dead arms resolved under R2. +- `PROTOCOL_MAGIC` behind `protocol-v1` — correct, and the V2 endpoint's + internal legacy diagnosis is tested both whole and truncated. *No + action.* +- `rumors-tracing` — clean minimal surface, deliberately not `Clone`, + correct explicit span parenting, bounded rendering with all four + budgets, honest "when not to use it" section. Nit: the crate doc's "a + disabled target costs the enabled-check alone" omits the (documented- + in-code, deliberate) relaxed `fetch_add` per message. + > **Resolution** (directly executable, on `w2/tracing-adapter`): in + > `crates/rumors-tracing/src/lib.rs`, amend the cost sentence to "a + > disabled target costs the enabled check plus one relaxed atomic + > increment (the ordinal must advance even unobserved; see + > `StreamAdapter::message`)". Then re-derive the README + > (`just readme`; the crate is wired into `tools/readme` per the + > branch's diff). Acceptance: `readme-check` green. +- Testing surface (`render_hook_capture`, `HookCapture`, `HookStream`, + `assert_items_account_for`, `stream_label`; `render_v2_capture` + removed) — appropriate for a doc-hidden test-internals door. *No + action.* + +## Simplification candidates + +Adoptable-now (behavior-preserving; each is its own one-commit fix): + +- **S1.** [gossip/tests.rs:130](file:///Users/oxide/src/rumors-review/src/peer/gossip/tests.rs) + test doc says the epilogue read "consumes exactly one byte"; the + marker is two bytes (the test body is correct). Test-doc correctness + is a stated crate invariant. + > **Resolution**: change the doc comment's first line to "Reading the + > marker consumes exactly the marker's bytes, leaving later bytes + > untouched." (Byte-count-free, so it cannot rot again.) No code + > change. +- **S2.** [bootstrap.rs:170](file:///Users/oxide/src/rumors-review/src/peer/bootstrap.rs) + ordinal vestige (seed 11): "its session ordinals counting from the + join (session `0`)" speaks as if a defined numbering exists; by owner + ruling the hook carries none. + > **Resolution**: replace the clause with "the joined peer then keeps + > the handler exactly as [`Peer::observe`] would attach it; an + > observer that numbers sessions will count the join as the first + > session it sees." Grep `session 0`/`ordinal` across `src/` to + > confirm no other vestige (the observe.rs doc-example's `ordinal` + > is the observer-side pattern and stays). +- **S4.** [greeting.rs:213](file:///Users/oxide/src/rumors-review/src/tree/mirror/streaming/remote/codec/greeting.rs) + `uint(input, _key)`: the parameter is unused. + > **Resolution**: use it for diagnostics — change the signature to + > `fn uint(input: &mut &[u8], detail: &'static str)` and have the + > three callers pass `"set_len is not an unsigned int"`, + > `"max_version_bytes is not an unsigned int"`, + > `"target_message_size is not an unsigned int"`, returned as the + > `Shape` detail (replacing the current shared string). Acceptance: + > greeting and handshake suites green; no test asserts the old shared + > string (grep `"greeting size entry"` — only the source). + +Design-proposal (moves readings or API; each **owner-gated**): + +- **S3.** `bookmark/format.rs`'s `push_head`/`Reader::head` + re-implements `mirror/cbor.rs`'s `write_head`/`read_head` (plus the + third 55799 spelling, R6). Both are property-tested, but one + canonical-head implementation is the crate's own stated ideal, and + drift between them is the class B2 shows the prose already lost track + of. + > **Resolution sketch** (for the round that takes it): replace + > `push_head` with `cbor::write_head` (byte-identical output — pin by + > leaving `frame_empty`/`frame_non_trivial` untouched), and rebuild + > `Reader::head` on `cbor::read_head` with a thin adapter mapping + > `HeadError::Truncated` → `FormatError::Truncated { len }` and every + > other `HeadError` → `NotABookmark { defect }` (the caller's + > defect), preserving the exact-position `Truncated.len` semantics + > (the reader's `at` bookkeeping stays). Acceptance: both bookmark + > snapshot pins byte-identical; the corruption, truncation, and + > version-spelling suites green with unchanged variant assertions. +- R3's `#[non_exhaustive]` question, R5's typed hand-off/greeting + errors, and `SessionKind`'s openness — specified at their findings. + +## Where I found no issues + +The canonical head grammar itself (cbor.rs: shortest-form check, widened +/indefinite/reserved/truncation rejection, async/sync agreement — the +proptests are the right ones and I could not construct a hole); signal +grammar and phase validation (the 340-placement atlas with exact bytes +is excellent); frame arity/shape enforcement; the two decoders' mirrored +over-budget logic; encoder flush algebra (seed 3); ledger +charge-before-custody; the greeting's exact-roster parse; preamble +diagnostic ordering (magic → version → semantics, with the cross-dialect +diagnosis tested in both directions and at the truncation boundary); the +bookmark frame's integrity totality (every byte hash-covered or +exact-compared; the corruption and truncation sweeps are +value-independent, so their proof generalizes past the `^0xff` flip); +the error atlas's two-ended coverage enforcement; the hook's threading +(begin-before-first-byte, election-before-data, V1 exclusion, control +handlers minted ahead of the preamble); the internal-capture suite (both +directions, election complementarity, one-item checks); the +wire-legibility property (genuinely rumors-blind walker); seed-liveness +auto-covering the new proptest seed files; the docs-only repair commit; +the merge seam; the re-derived pins (each read from its instrument, no +transcribed constants — the probe's old hardcoded `28` was correctly +dissolved into `dispute_overhead_bytes()`); digestshare's figures and +its renderer contract; and the design document's cost table against the +shipped constants (the +3 B/child, 35→43, +3.9%/+14%, 1.8 MB reply, and +30-byte preamble figures all reconcile with code or measurement). + +## Considered and dismissed + +Candidates examined during the review and dismissed, recorded so the +packet is self-contained (previously these dispositions lived only in +review conversation): + +- **The greeting's declared length is uncapped (u64)** where the old + wire's framing capped at u32: dismissed — `read_payload`'s memory + tracks receipt, never the declaration, so a large declaration costs + only what the transport actually delivers; consistent with the stated + memory policy and with V1's behavior at its own bound. +- **`resume_payload`/`reserve_exact` over-allocation edge**: in + principle `Vec::reserve_exact` may over-reserve, letting `read_buf` + read past the payload boundary; dismissed — capacity is clamped to + `len` on every growth step, the global allocator honors exact + requests in practice, and the code predates this stack. +- **`chunk_boundary_cuts(0)` underflows**: dismissed — a `cfg(test)` + helper with no zero-total caller; the underflow is unreachable from + any committed test. + +## Residual risks and test gaps + +- **No mutants evidence over the new code** (seed 5's flip side): the + roster is clean, but no campaign result over the codec/bookmark/hook + rewrite is in evidence. B2 is a useful calibration point for suite + blindness even after its reclassification: nothing noticed that two + decode sites delegate spelling judgment to a tolerant parser — in + contract once B2's prose rescope lands, but the same blindness would + hide an *unintended* delegation or a lost structural check. + > **Resolution** (directly executable, resource-heavy): run + > `cargo mutants --file 'src/tree/mirror/**' --file 'src/bookmark/**' + > --file 'src/observe.rs'` from the repo root — the campaign + > configuration of record (nextest, `--all-features`, dev profile) is + > already in `.cargo/mutants.toml` and applies to the plain + > invocation. Prefer running it on the big remote box (ox-east-1, via + > the `building-on-illumos` sync flow) rather than a laptop; it is a + > long, parallel run. Triage every survivor by the roster header's + > disposition ladder (refactor → assert → exclude-with-rationale). + > Sequencing (owner-ruled in review follow-up): the campaign runs + > LAST — after the review-fix round reaches quiescence and after the + > depth-limit/batch feature lands — so the single authoritative run + > measures the fully settled code; the mutated files include exactly + > the surfaces those rounds move. Deliverable: either zero survivors, + > or each survivor dispositioned per the ladder in the same change + > that records it. +- **Missing taxonomy constructions**: `PreambleDefect::Version`/ + `Network` (resolved under R2), the greeting's `NotShortest` + rejection, and the bookmark's `Integrity`/`PayloadTag`/ + `PayloadByteString` defects (covered positionally by the corruption + sweep but never asserted as their typed variants). + > **Resolution** (directly executable): + > - Greeting: in + > `src/tree/mirror/streaming/remote/codec/greeting/tests.rs`, add a + > test that takes a canonical `encode_greeting(&sample(vec![]))`, + > locates the `set_len` value head inside the embedded map (use the + > existing `find` helper on the key bytes `b"set_len"`; the value + > head follows the 8-byte key region: 1 head byte + 7 text bytes), + > re-spells that one-byte uint as the widened `0x18 ` form + > (splicing one byte in, and fixing the embedded byte-string length + > head and outer item accordingly — or simpler: build the malformed + > *map* directly by copying `greeting_map`'s output and splicing, + > then call `parse_greeting` on the map bytes, which needs no outer + > fix-up), and asserts + > `Err(GreetingError::Head(HeadError::NotShortest))`. + > - Bookmark: in `src/bookmark/format/tests.rs`, three targeted + > flips on `frame(b"payload")` asserting typed defects: + > byte at the integrity head's offset (the `0x58` of + > `INTEGRITY_HEAD`; compute the offset as + > `SELF_DESCRIBED.len() + 1 + ` — 3 + 1 + 1 = 5 + > today — rather than hardcoding) → + > `NotABookmark { defect: FrameDefect::Integrity }`; + > the payload tag byte (`0xd8`, at integrity offset + 2 + 32) → + > `PayloadTag`; the payload byte-string head re-spelled widened + > (`0x58 0x07` for the 7-byte payload, with the hash recomputed + > over the re-spelled covered region so only the spelling check can + > reject — the `non_canonical_version_spelling_is_rejected` test is + > the template) → `PayloadByteString`. + > - Acceptance: each new test names its variant in a `matches!`; all + > suites green. +- **B1's class needs a standing test** — specified in B1's resolution + (the two deep-nesting tests). Optionally mirror an + unfold-budget-exhaustion test in the adapter's `render/tests.rs` + (nest tag-24 five levels — one past `UNFOLD_BUDGET` — and assert the + innermost renders as a raw `h'…'` byte string, not unfolded): the + adapter passes today; the test pins that it stays true. +- **Orphan-snapshot recurrence** — specified at seed 13. +- **Query-frame session fixture** — specified at seed 14. +- **Adapter concurrency**: session numbering under genuinely concurrent + sessions is argued from the atomic, tested only sequentially. + > **Resolution** (directly executable, on `w2/tracing-adapter`): in + > `crates/rumors-tracing/tests/adapter.rs`, add a test that clones one + > observed peer's `Rumors` handle, creates two in-memory link pairs + > and two counterparty peers, and drives both gossip sessions inside + > one `tokio::join!` (current-thread runtime is fine; the sessions + > interleave at await points, which is the property under test — + > `session()` re-entrancy). Assert: exactly two `session` spans, with + > `ordinal` fields `{0, 1}` as a *set* (order between concurrent + > sessions is unspecified), and each session's message ordinals dense + > from 0 (reuse the existing density assertion). Acceptance: test + > green repeatedly (`--no-capture -j1` and default). +- **Label-width latency** — resolved under R4. +- **digestshare liveness** fires only when the tool is run by hand. + > **Resolution** (**owner-gated**, gate change): add a justfile recipe + > `digestshare:` running `./tools/digestshare` (its exit code already + > carries the liveness verdict; a comment above the recipe should say + > it checks the renderer-vocabulary contract, not a threshold) and + > list it in the gate's lint tier next to the other `tools/` linters. + > If Finch rules digestshare stays a manual measurement aid, record + > that in the tool's docstring ("not gate-wired by decision: the + > liveness guard protects only interactive runs") so the gap reads as + > chosen rather than missed. + +## Out-of-range observation (predates the stack; owner-ruled) + +Surfaced while answering "can a user-chosen `T` crash production?" +(B1's functional-axis analysis), and disposed here explicitly rather +than mentioned in passing. It is **not** part of this stack's findings: +payloads were CBOR before the review range (the version-keying +migration), so the edge exists at the base commit too. **Assessed from +source** (ciborium ser/de internals plus the crate's call sites), not +constructed. + +**Payload nesting depth has an undocumented, asymmetric functional +limit.** `ciborium`'s serializer has no recursion cap, so `send` +accepts a payload value of any nesting depth (recursion there is over +the user's own in-memory value); `ciborium::de::from_reader` caps +decode at 256 scopes. Consequence: a payload nested deeper than 256 +container/tag scopes is accepted and stored locally, but **every +transfer of that leaf to any peer fails** — the receiver's record +decode returns `RecursionLimitExceeded` (typed, as +`DecodeLeafError::Message`), the session aborts cleanly, and the retry +fails the same way for as long as the divergence persists: a +deterministic gossip wedge on a locally-legal input. Nothing in the +crate's payload-facing documentation states a depth limit. + +> **Owner disposition (Finch, ruled in review follow-up)**: the +> recursion limit becomes a configurable setup value on the `Peer`, +> enforced **symmetrically** (send-side admission and decode-side +> ingress judge the same bound), defaulting symmetrically to a +> reasonable number. Refined in the same follow-up: the limit is +> **exchanged in the greeting and must match exactly**; a mismatch in +> either direction is an unconditional, typed abort at the handshake. +> Rationale of record: negotiating down is unsound — a peer whose +> session limit dropped below its own configured limit may already +> hold messages deeper than the negotiated bound, content it is then +> not allowed to gossip — so any negotiation scheme merely relocates +> the failure to mid-session, conditional on which leaves actually +> differ. Parameter equality trades that (sometimes-crashy with some +> peers on some messages) for a deterministic fail-fast on mixed +> configurations at every pairing. Structurally, the limit is a +> property of the *shared set* — every replica must be able to hold +> and forward all content — so it is Network-like (pairwise equality, +> transitively fleet-wide agreement), not +> `target_message_size`-like (a per-session resource trade where the +> minimum is safe). The knob's rustdoc carries the contract inline; +> this packet records the ruling. +> +> **Implementation spec** (executable without further context; every +> formerly open sub-decision is now owner-ruled in place — steps 5 and +> 6 carry those rulings): +> +> 1. **The knob.** A builder-style setter in the mold of +> `Peer::target_message_size`: `#[must_use]` on `Peer`, plus +> `Bootstrap` and the `BookmarkedBootstrap` passthrough (the join +> session decodes supplied records before a `Peer` exists — same +> pattern as `run_budget`/`observe`). The value follows the peer +> through `into_rumors`, cloning, reunion, bookmarking, and +> retirement, like every other setup value. Store beside +> `run_budget` in the config fields. +> 2. **The default.** Recommend `DEFAULT_PAYLOAD_DEPTH_LIMIT = 256` — +> exactly the decode bound today's code already enforces implicitly +> (ciborium's `from_reader` default), so a fleet upgrading together +> sees no acceptance change on existing content; the only new +> rejections are send-side (landing on the author of an over-deep +> value) and the handshake mismatch (landing on mixed +> configurations). Wire interop with pre-change code is governed by +> the greeting format change in step 6, not by this constant. State +> that rationale in the constant's doc. A named constant, exported +> beside `DEFAULT_TARGET_MESSAGE_SIZE`. +> 3. **Decode side.** The value-erasure landed by PR #37 concentrated +> wire payload ingress — both dialects — into one parse: the +> peer-minted `PayloadDeserializer` +> (`Message::deserializer::`, `message.rs:148–166`), reached via +> `Message::from_wire` from V2's `parse_record` and from V1's +> `Message::from_reader` (whose own outer parse unwraps a flat byte +> string). Replace that one inner `ciborium::de::from_reader` with +> `ciborium::de::from_reader_with_recursion_limit(input, limit)`. +> The minted fn is a plain function pointer and cannot capture a +> runtime value, so the limit rides as data, in the **ruled +> minted-codec shape** (the ruling and its rationale are recorded +> at step 5): a small `Copy` struct pairing the minted serializer +> and deserializer fn pointers with the `PayloadDepthLimit` field, +> minted at `Peer` construction where the bare `deserializer` field +> is minted today (`peer.rs:225`) and threaded wherever that field +> travels. This preserves the erasure's property that sessions stay +> non-generic. Parses that +> stay at the library default, each structurally flat: V1's outer +> byte-string unwrap (`message.rs:220`), the version atoms +> (`tree/wire.rs:205`, `frame.rs:361`), and the bookmark payload +> walk. Note in the commit message that the V1 freeze is byte-level +> and this moves no byte; only the local acceptance bound becomes +> configurable, symmetrically with V2. +> Deliberately *not* threaded from peer config: +> `Message::from_slice`/`from_bytes` (public constructors over +> caller-supplied bytes — the caller's trust domain; they take the +> limit as an explicit parameter instead, per step 5) and the +> bookmark payload walk (crate-authored, structurally flat). +> 4. **Send side (the symmetric half).** Enforce at serialization time +> — concretely inside `Message::try_new`, step 5's constructor: +> after ciborium-serializing — the crate's own output, so +> definite-length and canonical — run an O(n) *iterative* depth +> scan over the +> produced bytes using the crate's head grammar (`cbor::read_head` +> with an explicit stack of remaining-child counts: an array head +> pushes its count, a map head pushes 2× its count, a tag pushes +> one; depth is the stack's high-water mark; bail as soon as it +> exceeds the limit). Do **not** transcribe ciborium's scope +> accounting into prose or constants — pin the symmetry with a +> differential proptest instead: generate values nested to depths +> around the limit (arrays, maps, and tags mixed), and assert the +> send-side scanner and +> `from_reader_with_recursion_limit::` agree on +> accept/reject at limit and limit ± 1. That test is the instrument +> that keeps "symmetric" true against either side drifting. +> 5. **Send fallibility and the batch lifecycle (ruled, refined across +> follow-ups).** Three rulings compose here: `send` eagerly creates +> the `Message` at invocation, returning a typed error on a depth +> violation; a failed batch commits nothing; and the batch is +> reshaped into a **closure scope**, so batch state cannot exist +> across an await point (by language rule — a synchronous closure +> body cannot await) and commit becomes explicit code that runs iff +> the closure returns `Ok`. `Batch` currently has no consumers, so +> the reshape is contained to the crate's own tutorial, doctests, +> and tests. Facts verified in-tree that this leans on: +> `Batch::send` already serializes eagerly (`batch.rs`: +> "Serialization runs here, not at commit"), so batching's +> efficiency gain — one tree traversal, one commit, one wakeup — is +> untouched; and building a batch holds no lock, so running a user +> closure while building is sound. Implementation: +> - **Thread the limit into `Message` creation itself** (without +> this, every creation site silently reverts to the library +> default): give `Message` a fallible, limit-taking constructor — +> `Message::try_new(message: T, limit) -> Result PayloadDepthError>` (`Message` is type-erased; the constructor +> is generic exactly as `Message::new` is) — which serializes +> (`to_vec`), runs the +> step-4 depth scan over the produced bytes, and errors past the +> limit. The error carries the configured limit and says the +> value exceeded it (the scanner may bail at limit + 1; it need +> not report the true depth). +> - **Failure-class split, stated at the constructor**: a +> `Serialize`-impl failure keeps `Message`'s existing documented +> panic contract (serializability is the caller's obligation — +> programmer error, unchanged); a depth violation is the typed +> error (data-driven — the value's shape can carry end-user +> data). +> - **The closure-scoped API (ruled)**: `Rumors::batch` becomes +> `fn batch(&self, f: F) -> Result +> where F: for<'s> FnOnce(&'s mut Batch<'_, T>) -> Result`. +> The scope handle keeps the `Batch` name and carries the private +> fields (the `&watch::Sender>`, the action list, the +> depth limit from peer config). `E` is fully generic and +> unbounded: a closure `?`s `Batch::send`'s `PayloadDepthError` +> into its own error type (or returns it directly), and returning +> any `Err` deliberately cancels the batch — an explicit abort +> affordance the RAII design never had. Scope methods lose their +> chaining returns (statement sequencing inside the closure +> replaces fluent chaining): +> `fn send(&mut self, T) -> Result<(), PayloadDepthError>` +> minting via `Message::try_new` with the carried limit, and +> `fn redact(&mut self, &Version)`. Single-action sugar stays on +> `Rumors`: `fn send(&self, T) -> Result<(), PayloadDepthError>` +> and infallible `fn redact(&self, &Version)`, each committing +> immediately. +> - **Commit-on-`Ok`; the lifecycle table collapses**: +> `Rumors::batch` runs the closure and performs the +> `send_if_modified` commit only on `Ok`. The scope type's `Drop` +> impl is **deleted** (and with it the `thread::panicking()` +> guard and the empty-list check). Each previously documented row +> falls out: a send error commits nothing (the ruled +> cancel-on-error, now structural); a user `Err` commits nothing; +> a panic unwinds past the commit call, committing nothing; and +> the async-cancellation prefix-commit hazard becomes +> *unrepresentable* — a cancellation lands between polls, and the +> closure runs inside one poll. Delete +> `a_cancelled_batch_commits_its_prefix` +> (`tests/single_peer.rs`) together with the hazard it pins, and +> rewrite the `Batch` docs' lifecycle prose as what IS — the +> batch commits iff the closure returns `Ok`, all-or-nothing — so +> the "performance optimization, not an atomicity guarantee" +> section inverts into a stated guarantee, with no ghost +> references to the drop-driven semantics (provenance lives in +> git). +> - **Leak-proofing, all static** (the enforcement that makes the +> no-await fiat real; every item is load-bearing): the +> higher-ranked `for<'s>` bound with `R` and `E` quantified +> outside it, so nothing borrowing through the handle can be +> returned (futures included) and no outer variable can stash the +> `&'s mut` (`'s` unifies with no outer lifetime); no `Clone`, no +> `Default`, no public constructor on the scope type (no owned +> escape, no `mem::swap` donor); fields private, public methods +> exactly `send`/`redact`. On variance: in the signature above, +> `'s` rides only on the `&'s mut` handle, whose inherent +> invariance in its pointee plus the HRTB already close the +> variance tricks — no marker is needed, and this simpler shape +> is preferred. Only if the implementation instead threads the +> scope lifetime *into* the type (a +> `Batch<'s, 'env, T>` received as `&'s mut Batch<'s, 'env, T>`, +> the literal `std::thread::scope` shape) does it also need that +> pattern's invariance marker (`PhantomData<&'s mut &'s ()>`). +> Pin the two principal leak vectors as `compile_fail` doctests +> on `Rumors::batch` (stash into an outer `Option`; return the +> handle) — doctests, so no new dev-dependency. +> - **Re-entrancy**: the closure may call `rumors.send(...)` or +> `rumors.batch(...)` on the same handle — building holds no +> lock, and the outer commit runs only after the closure returns +> — so nesting is sound and produces separate commits, +> inner-before-outer: one doc sentence, one test. +> - **Public rehydration constructors**: `Message::from_slice` and +> `Message::from_bytes` gain the limit as an explicit parameter +> (they have no peer context), passed to +> `from_reader_with_recursion_limit`; pre-release, change the +> signatures rather than minting `_with_limit` variants. This is +> the trap the threading rule closes: an application on a raised +> fleet limit must be able to rehydrate its own stored deep +> messages, which the implicit default would refuse. The depth +> failure surfaces through their existing `io::Result` as +> `InvalidData` (ciborium's `RecursionLimitExceeded`), documented. +> `Message::from_wire` (the wire path's ingress constructor) is +> where step 3's limit-carrying deserializer lands; no separate +> change. One admission sweep the erasure makes necessary: check +> whether any *other* public `Message` constructor (`new`, +> `from_arc`) can reach a peer's set — if one can, it takes the +> same limit-checked path, else state the admission invariant +> (only `Rumors::send`/`Batch::send`/wire ingress insert) where +> the constructors are documented. +> - **Shape suggestion**: mint a `PayloadDepthLimit` newtype +> (newtypes over bare `usize` in public signatures, per house +> style) carrying the default via `Default` and used uniformly by +> the `Peer`/`Bootstrap` knob, `Batch`, `Message` constructors, +> and the greeting codec. +> - **The minted codec (ruled — proposed by Finch in review +> follow-up, endorsed with one refinement, approved)**: push all +> serde bounds to `Peer` construction by minting a payload +> *serializer* there too, beside the deserializer, both carrying +> the configured depth, and using them pervasively. The +> refinement: a plain fn pointer cannot capture a runtime value, +> so the concrete shape is a minted codec — a small `Copy` struct +> pairing the two fn pointers with the `PayloadDepthLimit` field +> — threaded wherever the deserializer travels today. What it +> buys: the limit is unmissable (every `Message` creation and +> every ingress parse in the peer's orbit goes through the one +> codec value, closing the threading trap structurally rather +> than by sweep); `T: Serialize` bounds drop from +> `Rumors::send`/`Batch::send` (bounds concentrate at +> construction, finishing for `Serialize` what the erasure did +> for `DeserializeOwned`); and the greeting reads the limit off +> the codec sessions already carry. Cost, accepted in the ruling: +> `Peer` construction demands `T: Serialize` even for a peer that +> never sends — symmetric with construction already demanding +> `DeserializeOwned` to mint the deserializer (forwarding needs +> neither bound, since gossip re-supplies cached bytes). +> Accordingly `Message::try_new`'s body is the minted +> serializer's target, and every "thread the limit" instruction +> in steps 3–6 reads as "thread the codec". +> - **Caller sweep**: the tutorial module, doctests, and every +> in-tree `.send(`/`.batch(` use migrate to the closure form or +> the single-action sugar, gaining `?`/`expect` as appropriate +> (`Batch` has no consumers outside the tree, so the sweep ends +> at the crate boundary). +> - **Tests**: commit-on-`Ok` — a closure batching sends and a +> redact commits once, atomically (observers see one wakeup); a +> depth-violating `send` inside the closure, `?`-propagated, +> commits **nothing**, earlier-queued actions included (tree +> unchanged, no wakeup — the cancel-on-error pin); a user `Err` +> return commits nothing; a panicking closure commits nothing +> (`catch_unwind` in the test); the re-entrancy ordering test; +> the two `compile_fail` leak doctests; and `from_slice` at a +> raised limit rehydrates a deep message that the default-limit +> call rejects (both directions asserted). +> 6. **Parameter equality at the handshake (ruled — no longer a +> sub-decision).** The greeting carries the sender's configured +> limit, and a session proceeds only if the two values are equal; +> a mismatch in either direction is a typed, unconditional abort +> after the greetings are exchanged and before anything else — in +> particular before the equal-versions early return, so mixed +> configurations surface even on converged, no-op sessions. +> - *Wire*: a new entry in the greeting map. Suggested key: +> `"payload_depth_limit"`; recompute the deterministic key order +> (length-first, then bytewise — at 19 characters it ties +> `"target_message_size"` on length and sorts before it on +> content) and update both the `KEYS` roster in +> `codec/greeting.rs` and `parse_greeting`'s exact-roster check +> (which becomes a seven-entry map). This is a deliberate, +> owner-ruled pre-release wire format change: re-accept the V2 +> snapshot corpus in the implementing commit, naming this change; +> re-run `tools/digestshare` and update the corpus figures in +> `design/cbor-legible-wire.md`; run the `tests/dispute_wire.rs` +> cells (the few-byte greeting growth amortizes to well under +> their bands at 8,192 divergent messages — verify, don't +> assume); add the greeting-table row and a decision-record entry +> for this ruling to `design/cbor-legible-wire.md`. +> - *Check placement*: in `proxy/start.rs`, in both +> `complete_connect` and `accept`, immediately after both +> greetings are in hand and before `connected()` runs its +> equal-versions resolution. Both sides detect symmetrically, +> like `NetworkMismatch`. +> - *Error*: a new typed public variant per the taxonomy ruling +> (errors name what they diagnose), e.g. +> `Error::PayloadDepthMismatch { local, remote }`, documented in +> the `Error` table ("fix the configuration: the limit is a +> fleet-wide parameter; align it and reconnect"). +> - *V1*: the frozen greeting cannot carry the parameter, so V1 +> sessions keep decode-side-only enforcement; the knob's docs +> state that content-conditional failure remains possible on the +> legacy dialect. +> - *Achieved invariant*, worth stating in the knob's rustdoc: with +> send-side admission (step 4) plus handshake equality, no V2 +> session between conforming peers can fail on payload depth at +> all — over-deep values are rejected at their author at the +> moment of choice, and mismatched fleets are rejected at hello. +> Changing the limit is therefore a fleet-coordinated +> configuration event, like changing the selected [`Protocol`] — +> document it in that register, not as a tuning knob. +> 7. **Tests to commit** (beyond the differential in step 4): a +> boundary pin — a payload at exactly the default depth round-trips +> peer-to-peer over an in-memory link; one level deeper is rejected +> at send with the typed error; a decode-side ingress test feeding +> a hand-crafted over-deep record through the codec test helpers (a +> nonconforming sender must still die typed at ingress, since +> send-side enforcement only binds this crate's own API); and the +> handshake-equality pair — two peers with different limits abort +> with the typed mismatch on both sides and open no data stream +> (assert via the capture harness or the observation hook), plus an +> equal-raised-limits control that gossips clean. +> 8. **Docs.** The knob's rustdoc states: what counts as a nesting +> scope (by reference to the differential test as the accounting's +> pin, not a prose transcription), the three enforcement points and +> the achieved invariant from step 6 (send rejects at the author; +> handshake rejects mismatched fleets; ingress rejects +> nonconforming implementations), the default and its rationale, +> and the fleet-coordination framing for changing the value. Sweep +> the rest of the rustdoc for drop-commit language about batches +> (crate docs, `Rumors` method docs, the tutorial) and restate it +> in the commit-on-`Ok` form. If crate-level docs change, re-derive +> READMEs (`just readme`). +> 9. **Sequencing against the review-fix round.** Every resolution in +> this packet's findings sections moves zero wire bytes (B1 and B2 +> assert zero snapshot movement in their acceptance criteria), so +> the review fixes and this feature can land in either order — but +> do not interleave them: this feature's greeting change is the +> sole snapshot re-accept in flight, and per the hard rules the +> re-accepting commit must contain exactly that deliberate change +> and name it. If R1's per-cell re-pinning lands *after* this +> feature, measure its cells at the then-parent before pinning +> (attribution discipline: never fold this feature's greeting +> growth, however sub-band, into R1's recorded numbers). +> 10. **Record of decisions.** This package (the knob, symmetric +> enforcement, greeting equality, and the closure-scoped batch) +> has outgrown a review packet; give it a small design document +> beside the wire doc whose decision record transcribes the +> rulings currently held only here: the depth limit as a property +> of the shared set (hence pairwise equality, not negotiation — +> with the negotiate-down unsoundness argument); eager `Message` +> creation with fallible `send`; a failed batch commits nothing; +> the closure scope as the no-await mechanism, recording the +> rejected fiats (`!Send` binds only futures that must be `Send` +> and misstates the type; `#[must_not_suspend]` is unstable on the +> pinned toolchain; clippy's `await-holding-invalid-types` binds +> only in-repo runs) and the commit-on-`Ok` atomicity inversion; +> the 256 default's rationale; and the V1 carve-out. Per the house +> rules, the design doc cites code, code cites nothing back, and +> the knob's rustdoc carries every invariant inline; this packet +> remains as review provenance only. diff --git a/.agent-notes/2026-08-20-payload-depth-limit/README.md b/.agent-notes/2026-08-20-payload-depth-limit/README.md new file mode 100644 index 000000000..76a22a094 --- /dev/null +++ b/.agent-notes/2026-08-20-payload-depth-limit/README.md @@ -0,0 +1,18 @@ +# The payload depth limit + +[`payload-depth-limit.md`](./payload-depth-limit.md) is the design document +and decision record for the payload-depth package: the configurable, +fleet-symmetric nesting-depth limit exchanged in the greeting and held to +exact equality; the peer-minted payload codec; admission by the receiver's +exact decode, extended to full value faithfulness (`Eq` mandated, decoded +value must equal the value sent); the closure-scoped batch with commit-on-Ok; +the exact ciborium pin with its bump playbook; and the vendor evaluation that +retained ciborium. + +Retired as implemented: every ruling in the record shipped on this branch, +and the load-bearing invariants live inline at the code (the knob's rustdoc +and the crate root's payload contract), per the repository's rules — this +document is provenance. The body is byte-identical to the document's last +revision in `design/`; its citation of `design/cbor-legible-wire.md` resolves +to +[`../2026-08-19-cbor-legible-wire/cbor-legible-wire.md`](../2026-08-19-cbor-legible-wire/cbor-legible-wire.md). diff --git a/.agent-notes/2026-08-20-payload-depth-limit/payload-depth-limit.md b/.agent-notes/2026-08-20-payload-depth-limit/payload-depth-limit.md new file mode 100644 index 000000000..adcb39017 --- /dev/null +++ b/.agent-notes/2026-08-20-payload-depth-limit/payload-depth-limit.md @@ -0,0 +1,235 @@ +# The payload nesting-depth limit + +The configurable bound on how deeply a message payload's decode may +recurse, enforced identically at both ends, and the send-path shape +built around it. This document records the design and its rulings; the +contracts of record are the rustdoc — start at +`Peer::payload_depth_limit`, whose docs carry every invariant inline, +then `Rumors::batch` for the batch lifecycle. + +## The problem + +CBOR payloads decode through a recursion-limited reader: a nesting bound +must exist, or a deeply nested value overflows the decode stack. But a +fixed, implicit, decode-only bound has a failure mode of its own: the +serializer accepts a value of any depth, so a payload nested past the +decoder's bound is accepted and stored locally while every transfer of +that leaf to any peer fails — a deterministic gossip wedge on a +locally-legal input, persisting for as long as the divergence does — and +the payload-facing documentation named no limit at all. + +## The design + +One limit, `PayloadDepthLimit`, a peer setup value like the selected +protocol, enforced at three points that share a single computation: + +- **Send admission**: `Message::try_new` serializes, then runs the + peer's minted deserializer — the exact fn every receiver's wire + ingress runs for the payload type, at the same limit — over the + just-serialized bytes, and requires the decoded value to equal the + value sent (by the payload type's own `Eq`, mandated in the payload + bounds); a payload the decode rejects or misreads is a typed + `EncodeError` at its author, at the moment of choice (`Depth` for the + recursion limit, `Roundtrip` for a type whose `Deserialize` rejects + its own `Serialize` output, `Unfaithful` for an encoding that decodes + to a different value). +- **Handshake equality**: the V2 greeting carries each side's configured + limit, and a session proceeds only if the two are exactly equal; + a mismatch is `Error::PayloadDepthMismatch` on both sides, after the + greetings and before anything else (the converged-session + short-circuit included). +- **Wire ingress**: every payload parse in the peer's orbit runs under + `from_reader_with_recursion_limit` at the configured limit, so + over-deep *content* from a nonconforming implementation still dies + typed at decode. The bound governs the decode's recursion, not the + bytes' shape: byte patterns the engine consumes without recursing + (deep tag chains in scalar positions, say) decode fine and are + harmless, so ingress bounds what can be *accepted*, never the + structure of what can be *sent at* it. + +Together: between conforming peers, no V2 session can fail on payload +depth at all — by construction within a decode-engine version, because +admission and ingress are one computation rather than two accountings +held in agreement. The knob's number is engine-defined (the decode +engine's recursion accounting for the peer's type), not a structural +property of RFC 8949 CBOR. Changing the limit is a fleet-coordinated +configuration event, in the same register as changing the selected +protocol. + +## The minted codec + +The limit rides a `PayloadCodec` minted at `Peer` construction: a small +`Copy` struct pairing a payload serializer and deserializer (both fn +pointers, generic over the payload type only at the mint) with the +`PayloadDepthLimit`, threaded everywhere the bare minted deserializer +traveled before. What this provides: the limit cannot be missed — every `Message` +creation and every ingress parse in the peer's orbit goes through the +one codec value, so no creation or ingress site can carry a different +bound — the payload type's serde bounds concentrate at construction +(`Serialize` and `Eq` join `DeserializeOwned` there and drop from +`Rumors::send`/`Batch::send`), and the greeting reads the limit off the +codec sessions already carry. The accepted cost: `Peer` construction +demands `Serialize` even for a peer that never sends, symmetric with +already demanding `DeserializeOwned` for a peer that never receives +(forwarding needs neither bound, since gossip re-supplies cached bytes). + +## The closure-scoped batch + +Fallible send forced the batch lifecycle question, and the answer +reshaped `Batch`: `Rumors::batch` runs a synchronous closure over +an exclusive `&mut Batch` scope handle and commits everything queued iff +the closure returns `Ok`. The scope type has no `Drop` impl, no `Clone`, +no `Default`, no public constructor; the higher-ranked closure bound +(with the result and error types quantified outside it) keeps the handle +from escaping, pinned by two `compile_fail` doctests. The lifecycle +collapses to one sentence — the batch commits iff the closure returns +`Ok`, all-or-nothing — and each case of the former drop-driven lifecycle +follows from that shape: a send error propagated out commits nothing; a +user `Err` commits nothing (a deliberate abort the RAII design never +offered); a panic unwinds past the commit call; and async cancellation +cannot observe a half-built batch, because a cancellation lands between +polls and the whole closure runs inside one poll. Batching's efficiency +gain — one tree traversal, one commit, one wakeup — is untouched, and +batches nest (building holds no lock; inner commits land before the +outer batch). + +## Decision record + +- **Symmetric, configurable, exchanged for equality** (Finch): the + recursion limit becomes a peer setup value, enforced symmetrically + (send-side admission and decode-side ingress judge the same bound), + carried in the greeting, and required to match exactly; a mismatch in + either direction is an unconditional, typed abort at the handshake. + Negotiating down is unsound: a peer whose session limit dropped below + its own configured limit may already hold messages deeper than the + negotiated bound, content it is then not allowed to gossip — so any + negotiation scheme merely relocates the failure to mid-session, + conditional on which leaves actually differ. Parameter equality trades + that for a deterministic fail-fast on mixed configurations at every + pairing. Structurally the limit is a property of the *shared set* — + every replica must be able to hold and forward all content — so it is + Network-like (pairwise equality, transitively fleet-wide agreement), + not `target_message_size`-like (a per-session resource trade where the + minimum is safe). +- **Admission is the receiver's exact codepath** (Finch, ruled on the + adversarial review's critical finding): the feature first shipped + send-side admission as a byte-level structural scan, differentially + tested against decoding the bytes as `ciborium::Value` — and the + review refuted that oracle by construction. The decode engine's + recursion accounting is *type-dependent*: decoding a serde enum + prices its variant scope (a unit variant costs one step a `Value` + decode of the same bytes does not), so no byte-level oracle can equal + the receiver's `T`-decode, and an enum payload at exactly the limit — + the crate docs' own recommended versioning shape — was admitted at + send and wedged every receiver. The ruling: admission *is* the + ingress computation — `Message::try_new` runs the minted deserializer + over the just-serialized bytes and discards the value — deleting the + scan and its differential outright, since with one computation there + is no second accounting to keep in agreement. Corollaries: the knob's + number is engine-defined, not RFC-structural; a payload type whose + `Deserialize` cannot read its own `Serialize` output now fails typed + at the author (`EncodeError::Roundtrip`) instead of at every + receiver. Residual, stated: two binaries whose ciborium versions + account recursion differently could still diverge on acceptance at + equal limits — a fleet upgrades its decode engine in coordination, + like the limit itself. +- **Faithful encoding, checked by `Eq`** (Finch): payload types must be + `Eq` — table stakes for any wire message type — and every send + requires the value decoded from the just-serialized bytes to equal + the value sent, rejecting inequality as the typed + `EncodeError::Unfaithful`. The deep why: `rumors` exists to + synchronize causal messages so a fleet can replicate a + causally-convergent state machine from the stream; a message that + decodes to anything other than what was meant violates that premise, + allowing any state machine driven by consumption of the stream to + diverge arbitrarily. The runtime check guarantees no implementation + can pollute the set with a value replicas would read differently. + Considered and rejected: a byte-fixpoint check (re-serialize the + decoded value, compare bytes) needs no `Eq` bound but cannot catch + the known lossy class — `Some(None)` serializes to CBOR null and + decodes as `None`, and re-serializing that `None` is byte-identical, + so the divergence is invisible in byte space; the value space is + finer than the byte space, so value equality is the only total + instrument. The check is send-side only (ingress holds no original to + compare against), and the bound is `Eq` rather than `PartialEq` by + design: equality must be an equivalence relation for the check to be + total and never spurious, which excludes NaN-capable float fields + from payload types. +- **ciborium pinned exactly (`=0.2.2`)** (Finch): the workspace pins its + CBOR engine to one exact version. Within a binary, admission and + ingress run the same compiled deserializer, so their symmetry is exact + by construction; across a mixed-version fleet, the no-failure + invariant holds only if every build shares one recursion accounting. + ciborium documents no accounting contract — the pricing of variant + scopes, tags, and containers is an implementation detail free to move + between releases — so the exact pin is what turns "same engine" from + an assumption into a property of the build. The pin's own manifest + comment carries the condensed rationale and procedure. +- **The engine bump playbook** (Finch): a ciborium bump is a + deliberate, fleet-coordinated event, never a routine dependency + refresh. Procedure: build the workspace at the old and the candidate + versions and run an accept/reject verdict differential over a seeded + corpus of payloads at and around the limit. Identical verdicts make a + pure bump. Differing verdicts mean the accounting moved: the bump + then ships with a greeting accounting stamp, so mixed fleets fail + fast at the handshake instead of content-conditionally mid-session. + A verdict-*strictening* bump additionally requires re-validating + stored payloads before gossiping — a replica may hold content the new + accounting rejects, which it would then not be allowed to forward: + the negotiate-down unsoundness argument, applied across time instead + of across peers. +- **Vendor evaluation: cbor2 rejected** (Finch): cbor2 (ldclabs) was + evaluated as an alternative engine — public configurable decode + recursion limit, active maintenance, deterministic-encode mode — and + rejected on supply-chain provenance: its implementation is a + from-scratch rewrite first committed 2026-06-12, roughly two months + old at evaluation, published atop an older repository's history; + insufficient provenance and maturity for this dependency's trust + position. Successor criteria, should ciborium's dormancy ever become + a liability: understood (source-verified) recursion accounting, + genuine provenance and maturity, serde round-trip fidelity, and a + configurable decode limit. Encoder byte-compatibility is explicitly + not a criterion: payload spelling carries no identity, and snapshots + re-accept deliberately. +- **Eager, fallible send** (Finch): `send` creates the `Message` at + invocation and returns the typed error on a depth violation; a + `Serialize`-impl failure keeps the documented panic contract + (programmer error), while depth is data-driven and therefore an error. +- **A failed batch commits nothing** (Finch): cancel-on-error, + all-or-nothing, earlier-queued actions included. +- **The closure scope is the no-await mechanism** (Finch): batch state + cannot exist across an await point by language rule — a synchronous + closure body cannot await. Rejected fiats: `!Send` binds only futures + that must be `Send` and misstates the type; `#[must_not_suspend]` is + unstable on the pinned toolchain; clippy's + `await-holding-invalid-types` binds only in-repo runs. Commit-on-`Ok` + inverts the old "performance optimization, not an atomicity + guarantee" into a stated guarantee. +- **The minted codec** (Finch, endorsed with the fn-pointer refinement): + serde bounds concentrate at construction; a plain fn pointer cannot + capture a runtime value, so the concrete shape is the codec struct + with the limit as data. +- **The 256 default** (Finch): exactly the decode bound the previous + code enforced implicitly (the decoder's `from_reader` default), so a + fleet upgrading together sees no acceptance change on existing + content; the only new rejections are send-side (landing on the author + of an over-deep value) and the handshake mismatch (landing on mixed + configurations). +- **The V1 carve-out** (Finch): the frozen V1 greeting cannot carry the + parameter, so V1 sessions keep decode-side-only enforcement, and + content-conditional failure remains possible on the legacy dialect; + the knob's rustdoc states it. + +## Where the pieces live + +- The knob and its full contract: `Peer::payload_depth_limit`, + `Bootstrap::payload_depth_limit` (rustdoc). +- The codec and the admission decode: `src/message.rs` (`PayloadCodec`, + `Message::try_new`, `EncodeError`). +- The greeting entry and the equality check: + `src/tree/mirror/streaming/remote/codec/greeting.rs` and + `src/tree/mirror/streaming/remote/proxy/start.rs`; the wire-format + ruling's row lives in `design/cbor-legible-wire.md`. +- The batch lifecycle: `Rumors::batch` and `Batch` (rustdoc), pinned in + `tests/single_peer.rs`. +- The peer-to-peer boundary pins: `tests/payload_depth.rs`. diff --git a/.agent-notes/AGENTS.md b/.agent-notes/AGENTS.md new file mode 100644 index 000000000..62f56e9b5 --- /dev/null +++ b/.agent-notes/AGENTS.md @@ -0,0 +1,29 @@ +# How to use this directory + +If you are a large-language model reading this, you are allowed to use this +directory to dump design notes, sketches, one-off demos, etc. + +For each new note, create a new directory formatted using the hyphen-separated +catenation of the current ISO 8601 datestamp and a hyphen-separated short +summary of the topic of the note. In the directory, you can place one or more +files of whatever format best suits the content (Markdown, Typst, LaTeX, +AsciiDoc, source code, etc.). If your notes on a given topic exceed a reasonable +length for a human to scan, please break them up into numbered sections, each +linked from the overall top-level note by whatever means is native to that +note's format (i.e. Markdown links, Typst includes, etc.). Consider the best +format for presenting the idea that you're expressing: anything requiring heavy +mathematical notation is easier to read in rendered Typst; code discussion is +easier in GitHub-flavored markdown, especially if you make your root note in a +given subdirectory a `README.md` file with line-range-linked GitHub permalinks. + +You can put whatever you want in a notes directory; this is your durable +scratchpad for preserving artifacts of ideation and experimentation. However, +whatever you put there needs to be legible *just as much* for human beings as it +does for yourself or other LLMs. Strive for succinct, expository, +teaching-register prose, parsimonious expression of clear ideas, and minimal +duplicative fluff in your phrasing, while preserving legibility and eschewing +jargon unless it is genuinely insight-provoking. + +If you are generating an orientation README for a document in the course of +retiring it into this directory, it is essential that you read the document +rather than imagining what it might contain. diff --git a/.agent-notes/CLAUDE.md b/.agent-notes/CLAUDE.md new file mode 100644 index 000000000..43c994c2d --- /dev/null +++ b/.agent-notes/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md diff --git a/.agent-notes/README.md b/.agent-notes/README.md index b35c58f18..90ab33bd1 100644 --- a/.agent-notes/README.md +++ b/.agent-notes/README.md @@ -8,29 +8,3 @@ models during development. They should not necessarily be considered to be audited, up-to-date, correct, sensible, or coherent (internally or externally), without verifying against reality, howsoever this applies. - -## To LLMs - -If you are a large-language model reading this, you are allowed to use this -directory to dump design notes, sketches, one-off demos, etc. - -For each new note, create a new directory formatted using the hyphen-separated -catenation of the current ISO 8601 datestamp and a hyphen-separated short -summary of the topic of the note. In the directory, you can place one or more -files of whatever format best suits the content (Markdown, Typst, LaTeX, -AsciiDoc, source code, etc.). If your notes on a given topic exceed a reasonable -length for a human to scan, please break them up into numbered sections, each -linked from the overall top-level note by whatever means is native to that -note's format (i.e. Markdown links, Typst includes, etc.). Consider the best -format for presenting the idea that you're expressing: anything requiring heavy -mathematical notation is easier to read in rendered Typst; code discussion is -easier in GitHub-flavored markdown, especially if you make your root note in a -given subdirectory a `README.md` file with line-range-linked GitHub permalinks. - -You can put whatever you want in a notes directory; this is your durable -scratchpad for preserving artifacts of ideation and experimentation. However, -whatever you put there needs to be legible *just as much* for human beings as it -does for yourself or other LLMs. Strive for succinct, expository, -teaching-register prose, parsimonious expression of clear ideas, and minimal -duplicative fluff in your phrasing, while preserving legibility and eschewing -jargon unless it is genuinely insight-provoking. diff --git a/Cargo.toml b/Cargo.toml index 58e44c33d..02aeebf98 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -6,6 +6,66 @@ members = [ "crates/suanpan", ] +# The version of record for every dependency in the workspace: member +# manifests inherit an entry with `workspace = true`, adding only their own +# feature selections and `optional` flags (which cargo requires at the +# member). No member declares a version, path, or default-features choice of +# its own. +[workspace.dependencies] +arc-swap = "1" +async-stream = "0.3" +base64 = "0.22" +before = { path = "crates/before" } +bitvec = "1" +blake3 = "1.8" +borsh = "1" +bytes = "1" +# Pinned exactly: depth admission and wire ingress run the same compiled +# deserializer, so decode-recursion accounting is symmetric within a binary, +# and across a mixed-version fleet it holds only if every build shares one +# accounting — which this exact pin provides, since ciborium documents no +# accounting contract. Bumping is a deliberate, fleet-coordinated event: run +# a two-build accept/reject verdict differential over a seeded corpus at the +# old and candidate versions; identical verdicts make a pure bump; differing +# verdicts ship with a greeting accounting stamp so mixed fleets fail fast at +# the handshake; and a verdict-strictening bump additionally requires +# re-validating stored payloads before gossiping. +ciborium = "=0.2.2" +clap = { version = "4", features = ["derive"] } +console_error_panic_hook = "0.1" +criterion = "0.5" +dashu-int = { version = "0.5", default-features = false, features = ["std"] } +dsi-bitstream = { version = "0.10.1", default-features = false, features = ["alloc"] } +futures = { version = "0.3", default-features = false, features = ["std", "async-await"] } +futures-util = { version = "0.3", default-features = false, features = ["std"] } +hex = "0.4" +indicatif = "0.18" +insta = "1.47" +itertools = "0.14" +peak_alloc = "0.3" +pollster = { version = "0.4", features = ["macro"] } +postcard = { version = "1", default-features = false, features = ["use-std"] } +proptest = "1" +rand = "0.8" +rand_chacha = "0.3" +ratatui = "0.30" +rayon = "1" +rumors = { path = "." } +seq-macro = "0.3" +serde = { version = "1", default-features = false } +serde_json = "1" +smallvec = { version = "1.15", features = ["union"] } +stacker = "0.1" +static_assertions = "1.1" +stats_alloc = "0.1" +suanpan = { path = "crates/suanpan" } +surface-scan = { path = "crates/surface-scan" } +thiserror = "2" +tinyvec = { version = "1.11", features = ["alloc"] } +tokio = { version = "1", default-features = false } +tokio-stream = "0.1" +wasm-bindgen = "0.2" + [package] name = "rumors" version = "0.1.0" @@ -55,35 +115,35 @@ protocol-v1 = [] meter = ["before/limb-meter", "before/scan-meter"] [dependencies] -before = { path = "crates/before", features = ["serde"] } -bytes = { version = "1", features = ["serde"] } -blake3 = "1.8" -static_assertions = "1.1" -itertools = "0.14" -serde = { version = "1", features = ["derive"] } -ciborium = "0.2" -seq-macro = "0.3" -smallvec = { version = "1.15", features = ["union"] } -tinyvec = { version = "1.11", features = ["alloc"] } -thiserror = "2.0" -hex = "0.4" -tokio = { version = "1", default-features = false, features = ["io-util", "macros", "sync"] } -tokio-stream = "0.1" -futures = { version = "0.3", default-features = false, features = ["std", "async-await"] } -futures-util = { version = "0.3", default-features = false, features = ["std"] } -async-stream = "0.3" -rand = "0.8" +before = { workspace = true, features = ["serde"] } +bytes = { workspace = true, features = ["serde"] } +blake3 = { workspace = true } +static_assertions = { workspace = true } +itertools = { workspace = true } +serde = { workspace = true, default-features = true, features = ["derive"] } +ciborium = { workspace = true } +seq-macro = { workspace = true } +smallvec = { workspace = true } +tinyvec = { workspace = true } +thiserror = { workspace = true } +hex = { workspace = true } +tokio = { workspace = true, features = ["io-util", "macros", "sync"] } +tokio-stream = { workspace = true } +futures = { workspace = true } +futures-util = { workspace = true } +async-stream = { workspace = true } +rand = { workspace = true } [dev-dependencies] -rumors = { path = ".", features = ["test-internals", "conformance"] } +rumors = { workspace = true, features = ["test-internals", "conformance"] } # The meter feature lights before's instrument surface for this crate's # own tests: the conservation suite reads the exact-bit-length observation # (`encoded_bits`) it denominates identity conservation in. -before = { path = "crates/before", features = ["serde", "meter"] } -proptest = "1" -criterion = { version = "0.5", features = ["html_reports"] } -insta = "1.47" -tokio = { version = "1", features = [ +before = { workspace = true, features = ["serde", "meter"] } +proptest = { workspace = true } +criterion = { workspace = true, features = ["html_reports"] } +insta = { workspace = true } +tokio = { workspace = true, default-features = true, features = [ "rt", "rt-multi-thread", "macros", @@ -94,15 +154,15 @@ tokio = { version = "1", features = [ # delay to virtual time so sweeps cost wall-clock compute only. "test-util", ] } -clap = { version = "4", features = ["derive"] } -rand = { version = "0.8", features = ["small_rng"] } -arc-swap = "1" -pollster = { version = "0.4", features = ["macro"] } -ratatui = "0.30" +clap = { workspace = true } +rand = { workspace = true, features = ["small_rng"] } +arc-swap = { workspace = true } +pollster = { workspace = true } +ratatui = { workspace = true } # Counting global allocator for the decoder allocation meter # (`tests/decode_alloc.rs`): prices decodes in bytes requested from the # allocator, so a declared-length pre-allocation is observable. -stats_alloc = "0.1" +stats_alloc = { workspace = true } [profile.dev] debug = "line-tables-only" diff --git a/README.md b/README.md index 42d88698e..83f7a7b07 100644 --- a/README.md +++ b/README.md @@ -148,12 +148,12 @@ Tokio for convenience; see [Runtime independence](#runtime-independence)): use rumors::Peer; #[tokio::main] -async fn main() -> Result<(), rumors::Error> { +async fn main() -> Result<(), Box> { // The universe's first peer creates it; every later peer bootstraps in. let alice = Peer::::seed().into_rumors(); - // A bare `send` statement commits when its `Batch` drops, right here. - alice.send("the meeting is at noon".to_string()); + // A send commits right here. + alice.send("the meeting is at noon".to_string())?; // A session runs over a `Link`: a control byte stream plus a supply // of independent data streams (see the `link` module); here, the @@ -212,6 +212,11 @@ stops being live, and no redaction object exists anywhere for an observer to yield (`Rumors::redact` explains why none is needed); an application that needs deletion events sends them as ordinary messages of its own. +All of the above observe the *set*. To watch the *wire* instead — every +protocol message of a live session, as raw CBOR items, for debuggers, +recorders, and tracing adapters — attach a handler from the `observe` +module (`Peer::observe`). + ## Transport: bring a `Link` A session's transport is a `Link`: one persistent bidirectional @@ -234,17 +239,41 @@ the caller. The I/O traits are Tokio's runtime-independent `AsyncRead` and `AsyncWrite`; no Tokio runtime, spawning, sockets, or timers are required by this crate. -## Message payloads and compatibility - -Your message type `T` needs `serde::Serialize` and -`serde::de::DeserializeOwned`; payloads are serialized as -CBOR ([RFC 8949](https://www.rfc-editor.org/rfc/rfc8949)). Because -CBOR carries field and variant *names*, reordering `struct` fields -or `enum` variants does not break compatibility with prior versions -of your type `T`; however, *renaming breaks compabitility*. It is worth -designing around this from the get-go: consider an outer `enum` indicating -the version of your application-level message type, even if it starts -out only having one variant, `V1`. +## Choosing a payload type + +Your message type `T` needs `serde::Serialize`, +`serde::de::DeserializeOwned`, `Eq`, `Send`, `Sync`, and +`'static`, all demanded once, at peer construction. Payloads are +serialized as CBOR ([RFC 8949](https://www.rfc-editor.org/rfc/rfc8949)). +Each bound guards replication: + +- **`Serialize` must succeed on every value you send.** CBOR itself + imposes no format-driven failures, so a `Serialize` error is a bug + in the payload type: sending panics. Avoid types whose `Serialize` + is data-dependently fallible (for example `std::path::PathBuf`, + which errors on non-UTF-8 paths). +- **Every encoding must decode back equal to the value sent.** Each + send re-decodes its own encoding with the exact decoder receivers + run and compares by `Eq`; a lossy encoding (for example + `Some(None)` in a nested `Option`, which decodes as `None`) is the + typed `EncodeError`, rejected at the author rather than silently + diverging at every replica. The bound is `Eq` rather than + `PartialEq` so the check is never spurious; this excludes + `f32`/`f64` fields (NaN compares unequal to itself). +- **Nesting depth is bounded.** Decoding a payload may recurse at + most `Peer::payload_depth_limit` steps (256 by default, ample + for ordinary types); an over-deep value is rejected at send. The + limit is held to exact equality fleet-wide at every handshake, so + an admitted payload is transferable everywhere; the knob's docs + carry the full contract. + +On compatibility across versions of your own type: because CBOR +carries field and variant *names*, reordering `struct` fields or +`enum` variants does not break compatibility with prior versions of +your type `T`; however, *renaming breaks compatibility*. It is worth +designing around this from the get-go: consider an outer `enum` +indicating the version of your application-level message type, even +if it starts out only having one variant, `V1`. ## Cargo features diff --git a/benches/gossip_fixed.rs b/benches/gossip_fixed.rs index e9e44739f..174d7990d 100644 --- a/benches/gossip_fixed.rs +++ b/benches/gossip_fixed.rs @@ -291,17 +291,25 @@ fn build_unilateral_redactions( } fn send_all(rumors: &Rumors, messages: Vec) { - let mut batch = rumors.batch(); - for message in messages { - batch.send(message); - } + rumors + .batch(|batch| { + for message in messages { + batch.send(message)?; + } + Ok::<(), rumors::EncodeError>(()) + }) + .expect("flat test payloads are within any depth limit"); } fn redact_all(rumors: &Rumors, versions: &[Version]) { - let mut batch = rumors.batch(); - for version in versions { - batch.redact(version); - } + rumors + .batch(|batch| { + for version in versions { + batch.redact(version); + } + Ok::<(), rumors::EncodeError>(()) + }) + .expect("flat test payloads are within any depth limit"); } /// A seed peer measuring shipped behavior: the default pipeline window is diff --git a/benches/in_memory.rs b/benches/in_memory.rs index fb4e2ccfa..6d07c0feb 100644 --- a/benches/in_memory.rs +++ b/benches/in_memory.rs @@ -57,10 +57,14 @@ const DELTAS: &[usize] = &[1, 100, 10_000]; /// Commit `n` unit payloads to `rumors` as one batch. fn send_units(rumors: &Rumors<()>, n: usize) { - let mut batch = rumors.batch(); - for _ in 0..n { - batch.send(()); - } + rumors + .batch(|batch| { + for _ in 0..n { + batch.send(())?; + } + Ok::<(), rumors::EncodeError>(()) + }) + .expect("flat test payloads are within any depth limit"); } /// A freshly seeded rumor set holding `n` messages, paired with its live @@ -143,11 +147,14 @@ fn bench_redact(c: &mut Criterion) { b.iter_batched( || build(n), |(rumors, versions)| { - let mut batch = rumors.batch(); - for version in &versions { - batch.redact(black_box(version)); - } - drop(batch); + rumors + .batch(|batch| { + for version in &versions { + batch.redact(black_box(version)); + } + Ok::<(), rumors::EncodeError>(()) + }) + .expect("flat test payloads are within any depth limit"); rumors }, BatchSize::PerIteration, diff --git a/benches/support/grid.rs b/benches/support/grid.rs index 5d3e76851..11a6fca2c 100644 --- a/benches/support/grid.rs +++ b/benches/support/grid.rs @@ -55,10 +55,14 @@ pub const REDACTED: &[usize] = &[0, 1, 10, 100, 1_000, 10_000, 100_000]; /// CBOR null byte, so fixtures measure tree / clock / hashing work, not /// payload serialization. pub fn send_units(rumors: &Rumors<()>, n: usize) { - let mut batch = rumors.batch(); - for _ in 0..n { - batch.send(()); - } + rumors + .batch(|batch| { + for _ in 0..n { + batch.send(())?; + } + Ok::<(), rumors::EncodeError>(()) + }) + .expect("flat test payloads are within any depth limit"); } /// Criterion samples for a fixture of the given build magnitude. The largest @@ -138,7 +142,7 @@ pub fn cells() -> impl Iterator { /// Build the two peers for one grid cell. /// -/// `left` is a fresh [`Peer::seed`]; `right` is a genuine disjoint peer minted +/// `left` is a fresh [`Peer::seed`]; `right` is a genuine disjoint peer created /// from it via [`bootstrap_fork`], so their parties are disjoint (the /// precondition for `gossip`). The shared prefix is inserted before the /// split; the `differing` messages and `redacted` deletions are applied @@ -166,15 +170,21 @@ pub fn build(cell: Cell) -> (Rumors<()>, Rumors<()>) { // prefix, so the other must honor `redacted` deletions it never made. // `cells` guarantees `common >= 2 * redacted`, so the slices don't // overlap and are in bounds. - let mut batch = left.batch(); - for version in &shared[..redacted] { - batch.redact(version); - } - drop(batch); - let mut batch = right.batch(); - for version in &shared[redacted..2 * redacted] { - batch.redact(version); - } + left.batch(|batch| { + for version in &shared[..redacted] { + batch.redact(version); + } + Ok::<(), rumors::EncodeError>(()) + }) + .expect("flat test payloads are within any depth limit"); + right + .batch(|batch| { + for version in &shared[redacted..2 * redacted] { + batch.redact(version); + } + Ok::<(), rumors::EncodeError>(()) + }) + .expect("flat test payloads are within any depth limit"); } (left, right) diff --git a/benches/support/latency.rs b/benches/support/latency.rs index 8f76bb73c..81cf821a9 100644 --- a/benches/support/latency.rs +++ b/benches/support/latency.rs @@ -129,7 +129,7 @@ pub struct DelayedWriter { /// The read half of a delayed pipe: bytes surface `delay` after the write. pub struct DelayedReader { shared: Arc>, - /// Timer armed for the head chunk's arrival. Minted lazily on first + /// Timer armed for the head chunk's arrival. Created lazily on first /// need: a `Sleep` must be created inside a runtime with a time driver, /// and pipes are constructed outside one. timer: Option>>, @@ -287,7 +287,7 @@ impl AsyncRead for DelayedReader { } } -/// The delayed-pipe [`Connector`]: each open mints a pipe and announces the +/// The delayed-pipe [`Connector`]: each open creates a pipe and announces the /// read end to the peer's acceptor. /// /// The announcement itself is undelayed; see the module docs for why no @@ -431,7 +431,7 @@ impl DelayedWire { b: Rumors, ) -> ((Rumors, Rumors), Duration) where - T: serde::Serialize + serde::de::DeserializeOwned + Send + Sync + 'static, + T: serde::Serialize + serde::de::DeserializeOwned + Eq + Send + Sync + 'static, { let wall_start = std::time::Instant::now(); let (pair, virtual_elapsed) = self.reconcile(a, b); @@ -470,7 +470,7 @@ impl DelayedWire { b: Rumors, ) -> ((Rumors, Rumors), Duration) where - T: serde::Serialize + serde::de::DeserializeOwned + Send + Sync + 'static, + T: serde::Serialize + serde::de::DeserializeOwned + Eq + Send + Sync + 'static, { assert!( self.paused, @@ -483,7 +483,7 @@ impl DelayedWire { /// Drive one gossip session to completion, timing it in virtual time. fn reconcile(&mut self, a: Rumors, b: Rumors) -> ((Rumors, Rumors), Duration) where - T: serde::Serialize + serde::de::DeserializeOwned + Send + Sync + 'static, + T: serde::Serialize + serde::de::DeserializeOwned + Eq + Send + Sync + 'static, { let Self { runtime, @@ -514,7 +514,7 @@ impl DelayedWire { #[allow(dead_code)] pub fn session_hops(capacity: usize, delay: Duration, (a, b): (Rumors, Rumors)) -> u32 where - T: serde::Serialize + serde::de::DeserializeOwned + Send + Sync + 'static, + T: serde::Serialize + serde::de::DeserializeOwned + Eq + Send + Sync + 'static, { let mut wire = DelayedWire::new(capacity, delay); let (_pair, elapsed) = wire.round_trip_virtual(a, b); diff --git a/benches/support/wire.rs b/benches/support/wire.rs index c94585249..9e02f22e3 100644 --- a/benches/support/wire.rs +++ b/benches/support/wire.rs @@ -1,6 +1,6 @@ //! Runtime-free asynchronous wire harness shared by reconciliation benchmarks. //! -//! Benchmarks measure what ships: peers minted here run at the default +//! Benchmarks measure what ships: peers created here run at the default //! pipeline window, which is the production budget in every build shape. use rumors::link::MemoryLink; @@ -25,7 +25,7 @@ impl Wire { /// Reconcile one pair while driving both endpoints concurrently. pub fn round_trip(&mut self, a: Rumors, b: Rumors) -> (Rumors, Rumors) where - T: serde::Serialize + serde::de::DeserializeOwned + Send + Sync + 'static, + T: serde::Serialize + serde::de::DeserializeOwned + Eq + Send + Sync + 'static, { let (a_result, b_result) = pollster::block_on(async { tokio::join!(a.gossip(&mut self.a_link), b.gossip(&mut self.b_link)) @@ -36,10 +36,10 @@ impl Wire { } } -/// Mint one disjoint replica by serving a bootstrap over an ephemeral link. +/// Create one disjoint replica by serving a bootstrap over an ephemeral link. pub fn bootstrap_fork(parent: &Rumors, protocol: Protocol) -> Rumors where - T: serde::Serialize + serde::de::DeserializeOwned + Send + Sync + 'static, + T: serde::Serialize + serde::de::DeserializeOwned + Eq + Send + Sync + 'static, { pollster::block_on(async { let (mut parent_link, mut newcomer_link) = rumors::link::memory_with_capacity(CAPACITY); diff --git a/benches/window_wallclock.rs b/benches/window_wallclock.rs index 95dd12934..d6d01e7d0 100644 --- a/benches/window_wallclock.rs +++ b/benches/window_wallclock.rs @@ -89,10 +89,14 @@ fn diverged(budget: usize, divergent: usize) -> (Rumors, Rumors) { /// Commit `n` random payloads as one batch. fn send_random(rumors: &Rumors, n: usize, rng: &mut SmallRng) { - let mut batch = rumors.batch(); - for _ in 0..n { - batch.send(rng.next_u64()); - } + rumors + .batch(|batch| { + for _ in 0..n { + batch.send(rng.next_u64())?; + } + Ok::<(), rumors::EncodeError>(()) + }) + .expect("flat test payloads are within any depth limit"); } criterion_group!(benches, window_wallclock); diff --git a/crates/before-viz/Cargo.toml b/crates/before-viz/Cargo.toml index 6965899f8..7afe85d1b 100644 --- a/crates/before-viz/Cargo.toml +++ b/crates/before-viz/Cargo.toml @@ -10,18 +10,18 @@ publish = false crate-type = ["cdylib", "rlib"] [dependencies] -before = { path = "../before" } -wasm-bindgen = "0.2" -serde = { version = "1", features = ["derive"] } -serde_json = "1" -base64 = "0.22" -console_error_panic_hook = { version = "0.1", optional = true } +before = { workspace = true } +wasm-bindgen = { workspace = true } +serde = { workspace = true, default-features = true, features = ["derive"] } +serde_json = { workspace = true } +base64 = { workspace = true } +console_error_panic_hook = { workspace = true, optional = true } [features] default = ["console_error_panic_hook"] [dev-dependencies] -proptest = "1" +proptest = { workspace = true } # On release-profile builds wasm-pack otherwise downloads and executes a # binaryen `wasm-opt` fetched unpinned from GitHub releases at build time, diff --git a/crates/before/Cargo.toml b/crates/before/Cargo.toml index fe5a70461..7971fd309 100644 --- a/crates/before/Cargo.toml +++ b/crates/before/Cargo.toml @@ -18,36 +18,36 @@ rustdoc-args = ["--html-in-header", "docs/fuelscape-header.html"] # sections include from $OUT_DIR. serde_json only reads the committed # JSON; nothing is measured or computed at build time. [build-dependencies] -serde_json = "1" +serde_json = { workspace = true } [dependencies] -bytes = "1" -dashu-int = { version = "0.5", default-features = false, features = ["std"] } -suanpan = { path = "../suanpan" } -dsi-bitstream = { version = "0.10.1", default-features = false, features = ["alloc"] } -static_assertions = "1.1" -thiserror = "2" -serde = { version = "1", optional = true, default-features = false, features = ["derive", "alloc"] } -borsh = { version = "1", optional = true } +bytes = { workspace = true } +dashu-int = { workspace = true } +suanpan = { workspace = true } +dsi-bitstream = { workspace = true } +static_assertions = { workspace = true } +thiserror = { workspace = true } +serde = { workspace = true, optional = true, features = ["derive", "alloc"] } +borsh = { workspace = true, optional = true } [dev-dependencies] # The emit_probe example's external comparison baseline: the production # build buffer is the crate-owned BitsBuf, and nothing shipped links bitvec. -bitvec = "1" -proptest = "1" -serde_json = "1" -criterion = "0.5" -rand = "0.8" -rand_chacha = "0.3" -postcard = { version = "1", default-features = false, features = ["use-std"] } -ciborium = "0.2" -stacker = "0.1" -peak_alloc = "0.3" -insta = "1" -rayon = "1" -indicatif = "0.18" -before = { path = ".", features = ["oracle", "meter"] } -surface-scan = { path = "../surface-scan" } +bitvec = { workspace = true } +proptest = { workspace = true } +serde_json = { workspace = true } +criterion = { workspace = true } +rand = { workspace = true } +rand_chacha = { workspace = true } +postcard = { workspace = true } +ciborium = { workspace = true } +stacker = { workspace = true } +peak_alloc = { workspace = true } +insta = { workspace = true } +rayon = { workspace = true } +indicatif = { workspace = true } +before = { workspace = true, features = ["oracle", "meter"] } +surface-scan = { workspace = true } [features] default = [] diff --git a/crates/suanpan/Cargo.toml b/crates/suanpan/Cargo.toml index 08f77eb5b..a458ba23d 100644 --- a/crates/suanpan/Cargo.toml +++ b/crates/suanpan/Cargo.toml @@ -11,13 +11,13 @@ readme = "README.md" # stored words, so streaming a wide delta allocates nothing) and the final # normalized magnitude leaves as one. Default features stay off: `std` alone # covers this crate's use. -dashu-int = { version = "0.5", default-features = false, features = ["std"] } +dashu-int = { workspace = true } [dev-dependencies] -proptest = "1" +proptest = { workspace = true } # The workspace-shared source scanners behind the claims roster # (`src/claims.rs`): the public-surface extractor and the witness scanner. -surface-scan = { path = "../surface-scan" } +surface-scan = { workspace = true } [features] # Counts every digit read-modify-write (plus one per operand limb read by a diff --git a/design/cbor-legible-wire.md b/design/cbor-legible-wire.md deleted file mode 100644 index 4c64d8ac2..000000000 --- a/design/cbor-legible-wire.md +++ /dev/null @@ -1,266 +0,0 @@ -# A CBOR-legible wire protocol, and the observation hook - -Status: proposal, pre-implementation. Owner: Finch. Origin: design -conversation, 2026-08-19. Builds on the version-keying migration's -uniform-CBOR rulings (payloads and the wire's version atom are already -CBOR; each supply-record body is already a two-item CBOR sequence). - -## Goal - -Every directed stream of a session — data streams and the control stream -alike — parses as a CBOR sequence with standard tag unwrapping, so that a -tool knowing nothing about rumors can unfold a recorded session into a -legible tree, down to exactly the atoms that are honestly rumors-private. -The concrete payoff: a generic debugger for rumors sessions that needs no -knowledge of the internal format or of the application's message types -(which are the application's own CBOR, legible for free), and an -observation hook that feeds it — whose first consumer is a `tracing` -adapter with deep structural inspection of live sessions. - -## Why no stream length is needed - -The form is RFC 8742 *CBOR sequences*: concatenated data items, no count, -no total length, no terminator. That is exactly the shape of an unbounded -stream, and it degrades gracefully — a truncated capture is a valid-prefix -sequence. (CBOR's indefinite-length containers also need no length up -front but want a closing break code an aborted session never writes; -sequences are the right choice.) - -## The layers, current form → CBOR spelling → cost - -| Layer | Today | CBOR spelling | Recurring cost | -|---|---|---|---| -| Frame signal | one dense byte (stream × state, 17 × 10 codes) | unsigned int item | +1 byte for codes ≥ 24 (most); see the signal ruling below | -| Frame | signal ‖ raw body | small array `[signal, body…]` | +1 byte array header | -| Record framing | u32 BE record header | **tag 63** ("embedded CBOR sequence in a byte string"): `63(bstr(version ‖ payload))` | ≈ 0 (tag 2B + bstr header 1–5B vs flat 4B; often equal, −1 for small records) | -| Record body | CBOR bstr(version) ‖ CBOR(payload) — already a sequence | unchanged, now inside the tag-63 bstr | 0 | -| Run length | u32 BE | bstr header arithmetic | ≈ 0 | -| Query child listing | raw `(radix ‖ 24-byte hash)*` | alternating array `[radix, h'…', …]` (+2 B/child) or map `{radix: hash}` (+3 B/child) | the one hot cost: ballpark +3–4% on digest-dominated dispute traffic — **measure at the calibration cells before pinning** | -| Greeting | fixed-offset block + frames | text-keyed map (`{"network": …, "version": …, "listing": …, "set_len": …, "max_version_bytes": …}`) | few dozen bytes, once per session | -| Preamble magic | 6 raw bytes | CBOR self-described **tag 55799** (`0xd9d9f7`) opening the control stream, then version/intent as ints, network as bstr | once per session | -| Stream open label | epoch byte ‖ index byte | two leading int items | +0–2 bytes per stream | -| Epilogue marker | one byte | int item | 0 | - -Notes on the spellings: - -- **Tag 63 is the load-bearing find.** The u32 record header earns its - keep by giving O(1) record skip and budget pricing independent of - payload shape (nested CBOR containers are not O(1)-skippable — their - headers carry counts, not subtree byte lengths; only strings are). - Tag 63's byte string preserves both properties exactly, while telling a - generic tool "unwrap me and parse the inside as a sequence." The - ledger's charge-before-custody ordering is untouched. -- **The listing map's key order coincides with canonicality.** CBOR - deterministic encoding mandates ascending keys; the wire's canonical - form mandates strictly ascending radixes. If the map spelling is - chosen, the two disciplines are one discipline. -- **The wire is deterministic-encoding CBOR, as a stated contract**: - shortest-form headers everywhere, one spelling per value. This is what - keeps the byte-pinning snapshot discipline meaningful after the change. -- The existing `record_len` pricing pattern (exact header arithmetic, - pinned against an actual push) generalizes to every priced length - above. - -## Where the opaque boundary stays, and why - -Version and party atoms remain opaque byte strings. Their canonical -bit-level codings are the crate's semantics; re-spelling them as CBOR -structure on the wire would be true structural re-encoding — larger, -slower, and a second spelling of the exact thing the tree pins -byte-for-byte. The generic debugger shows "a 37-byte version atom"; -rendering the atom's *meaning* is the public skyline iterator's job -(`design/version-skyline-iterator.md`) — a rumors-aware lens over the -rumors-blind skeleton is one `Plateau` walk away, and the two designs -are deliberate complements. - -## The bookmark: fully CBOR-parseable on disk - -The stored bookmark follows the same property (ruled 2026-08-19): the -whole file parses as CBOR, not just its payload. Sketch: the file opens -with the self-described tag and carries the format version, the integrity -hash, and the payload as items — with the hashed region spelled as an -embedded byte string (tag 24, "encoded CBOR data item"), so "the bytes -the hash covers" is a well-defined CBOR-visible region rather than an -offset convention: - -``` -55799( [ format_version: int, integrity: bstr, payload: 24(bstr(map)) ] ) -``` - -`FormatError`'s taxonomy survives re-denominated: `BadMagic` becomes -"not self-described CBOR / wrong shape", `VersionMismatch` and -`HashMismatch` are unchanged in meaning, `Truncated` becomes a sequence -truncation. This is a format-version bump under the bookmark's own -convention. - -## Tagged atoms: context-free identity for the opaque byte strings - -The opaque atoms gain CBOR tags — selectively — so their identity travels -with them rather than living in protocol position (ruled 2026-08-19). A -tagged atom is self-describing anywhere it appears: a wire capture, a -bookmark, a log line, a pasted hex snippet. That turns the generic -debugger's "37-byte atom" into a dispatch point: a thin *rumors lens* -keyed on nothing but a tag table sends version atoms to the public -skyline iterator and renders them semantically, with zero -protocol-position knowledge. Tags are the bridge between the -rumors-blind skeleton and the rumors-aware lens. - -**Placement rule (the crux): tags belong to the transport codecs, never -to the serde impls.** A `Version` whose *serde* implementation emitted -tags would stop being format-agnostic — an application payload -containing a `Version`, serialized to JSON, would break on a -CBOR-specific concept tunneled through serde. Instead the wire and -bookmark codecs (already hand-written at the framing layer) write -`tag ‖ untagged-serde-bytes` and hand-read the tag before delegating -decode. `before`'s serde impls stay untagged and backend-agnostic; the -tags are protocol vocabulary, owned where the protocol is spelled. - -Consequences for parsing: no wholesale non-serde parser is required — -only the points that already hand-parse read tags. (Two library facts: -`ciborium::tag`'s `Required`/`Accepted` wrappers do tunnel tags through -serde, but as a ciborium-specific magic-newtype mechanism that would -format-lock `before`'s impls — deliberately not used; and -`ciborium::Value` preserves tags natively, so generic consumers get them -for free.) - -Tag / don't-tag: - -- **Tagged**: version atoms and party atoms wherever the protocol spells - them (supply records, the greeting, the bookmark's stored clocks). - Their contexts are diverse, and their per-instance cost (+3 bytes for - a first-come-first-served-range tag) lands on payload-dominated paths - or once-per-session surfaces. -- **Untagged**: hashes inside listings — one context, - position-determined, and +3 on a 25-byte child is ~12% on the - dispute-heavy path, the one place bytes are dear. Structure already - names them. Signals and counts likewise: position suffices. - -Tag numbers come from the IANA first-come-first-served range (256+; -3-byte encodings — the 1- and 2-byte ranges are assigned or -specification-required). The honest path is registering a small -contiguous block (FCFS registration is lightweight); squatting risks a -generic tool someday rendering these atoms with someone else's -semantics. Until registration lands, the numbers live in one pinned -constant table, and the capture renderer learns their names (the -sanctioned renderer-vocabulary re-accept class). - -## The one open wire ruling: signal redundancy - -Within one recorded directed stream the signal's stream component is -constant, so signals *could* re-base to state-only codes (≤ 9, always one -CBOR byte). But the dense code's redundant stream component is what the -`Mislabeled` check validates against the transport label — a conformance -bug detector with committed fault-matrix coverage. Recommendation: keep -the redundancy and pay the byte (codes ≥ 24 cost two). The ruling is the -charter's first decision. - -## The observation hook - -The capture path is a public hook, installed at `Peer` construction -(ruled 2026-08-19, shape below refined with the implementer's latitude): - -- **A handler attaches to the `Peer` when it is created.** For each - session the peer runs (gossip, and equally bootstrap and retire — a - capture that skips session kinds is a debugger with blind spots), the - handler is asked for a **per-session sub-handler**. The sub-handler's - creation call carries what identifies the session (intent, protocol, - role election, an ordinal): each captured session is uniquely - identifiable, and the sub-handler's lifetime is the session's. -- **The sub-handler is invoked once per protocol message on every - directed stream of that session, in observation order.** One serialized - invocation stream per session is what captures inter-stream ordering: - the call order *is* the observed interleaving. (Design note: the - streams pump concurrently, so this is a synchronization point — the - hook must never block on protocol progress, and a slow handler - back-pressures its session. That is acceptable for a debugger and must - be documented at the hook. If contention ever matters, the recorded - alternative is per-stream invocation plus a session-level atomic - ordinal, reconstructing total order without a lock.) -- **The per-message payload is the frame's wire bytes with minimal - identity, not parsed values.** Something of the shape - `fn message(&mut self, frame: Observed<'_>)` where `Observed` carries - the directed-stream identity (speaker + stream), the direction - (sent/received — both directions are captured), and `bytes: &[u8]` - which is **exactly one CBOR item**. Two deliberate choices here: - borrowing keeps the hot path zero-copy, and bytes-not-types keeps the - hook *itself* rumors-blind — no protocol type appears in its - signature, so the hook's API is stable across wire evolution and its - consumers parse with any CBOR library (or none). A bare - `FnMut(&[u8])` is the degenerate form; the small struct earns its - keep the moment a consumer wants to know which stream spoke. -- Attachment is dynamic (`Arc` held as an `Option`), not a - generic parameter on `Peer`: one branch per frame when unattached, - and the public type stays unparameterized. An observability surface - does not warrant monomorphization. - -## First consumer: the tracing adapter - -A separate crate (or feature-gated module) so the core keeps its -dependency surface: sessions open `tracing` spans (session identity as -span fields), every observed frame is an event within its span, and the -CBOR structure maps to structured fields — ints and text directly, maps -by key, atoms as lengths-plus-hex. Because the wire is CBOR all the way -down, the adapter is a *generic* CBOR-to-tracing bridge plus a thin -naming layer; deep inspection of application payloads comes free, since -they are the application's own CBOR. The adapter is also the dogfood -proof that the hook's bytes-only signature suffices. - -## The committed contract - -- **The rumors-blind render test**, built on the public hook (the - instrument enters through the public door): capture a full session in - tests, parse every directed stream with a generic RFC 8742 parser plus - standard tag unwrapping (55799, 63, 24), and assert everything parses - with no bytes outside CBOR items. This is the tamper-evident form of - the legibility promise; prose claims of legibility are decoration - without it. -- The full snapshot corpus re-accepts as one deliberate, owner-ruled - pre-release format change, named in the re-accepting commit. -- Re-derived (never transcribed) readings: the dispute-wire closed form - and crossover, the window-solve constants, digestshare, decode-alloc - meters, and the affected wasm32 wire-door pins. - -## Cost summary - -An M/L codec lane, comparable to the borsh→CBOR wire migration: the -codec layer (signal/frame/streams/greeting/bookmark format) rewritten, -hand-parsed as today (delegating to ciborium is *not* required — the -structural validation and exact pricing stay first-class), plus the hook -threading through the session drivers, plus the re-accept and re-pin -wave. Recurring wire cost: +1–2 bytes per frame and +2–3 bytes per listed -child on dispute-heavy traffic (low single-digit percent, measured before -pinning); essentially zero relative cost on bulk supply. What does not -change: session semantics, the deadlock-freedom argument (framing- -independent; the hook adds observation, never a protocol dependency), -and every validation property, re-denominated. - -## Sequencing - -After the version-keying branch merges: same review season, one format -era. Pre-release is the cheap moment for a wire change; once a release -ships, this is a new protocol version by the hard rules. The tracing -adapter can trail the codec lane as its own small unit; the render test -lands with the codec lane itself. - -## Decision record - -- 2026-08-19 (Finch): the +1 byte per record for CBOR-legible record - bodies is accepted; the record body is a CBOR *sequence*, deliberately - not an array-wrapped tuple. -- 2026-08-19 (Finch): pursue full stream legibility — structural CBOR - for listings and greetings; payloads legible via their containing - records; the generic-debugger use case is the design's purpose. -- 2026-08-19 (Finch): the on-disk bookmark format becomes fully - CBOR-parseable under the same property. -- 2026-08-19 (Finch, shape; Claude, refinements): the observation hook — - Peer-attached handler, per-session sub-handler capturing inter-stream - ordering and session identity, per-message bytes-level invocation; the - tracing adapter is the first consumer. -- 2026-08-19 (Finch): the opaque atoms gain CBOR tags, placed in the - transport codecs (never the serde impls), per the tag/don't-tag table - above. -- Open rulings for the charter: the signal-redundancy byte (recommend - paying it); listing spelling (alternating array vs map — recommend the - map, for the canonicality coincidence, unless the extra byte per child - reads as too dear at the calibration cells); the hook's final - signature; the tag-number block and its IANA registration. diff --git a/design/rumors-frame-fuzz.md b/design/rumors-frame-fuzz.md index 25e84554a..156da48b4 100644 --- a/design/rumors-frame-fuzz.md +++ b/design/rumors-frame-fuzz.md @@ -174,9 +174,9 @@ Every input, in one process, under libFuzzer: `Error::Epilogue`, `Error::Mirror` wrapping `MaterializedError`/`Violation` — `UnaskedReply`, `UncontainedSupply`, … — and `RemoteError` over the codec's - `DecodeError`/`DecodeErrorKind`, the stream layer's + `CodecDecodeError`/`CodecDecodeErrorKind`, the stream layer's `StreamError`/`AcceptError`/`SendError`, and the adapter's - `DecodeError` with `OversizedVersion`, `LeafOutsideScope`, …). What + `ReplyDecodeError` with `OversizedVersion`, `LeafOutsideScope`, …). What the harness asserts dynamically is the *link consequence* the contract attaches: on any `Err`, the link end's `SessionState::poisoned()` reads true (via `Link::into_parts`). diff --git a/examples/swarm.rs b/examples/swarm.rs index 15f619975..e5a132be6 100644 --- a/examples/swarm.rs +++ b/examples/swarm.rs @@ -73,7 +73,7 @@ //! thread — the only writer of the peer directory — watches the desired count //! against the live one and reconciles a step at a time: //! -//! - **Grow:** pick a random live party, hand it a *fork* command. It mints a +//! - **Grow:** pick a random live party, hand it a *fork* command. It creates a //! disjoint child of its own [`Rumors`] (via `bootstrap_fork`), ships the child //! back to the coordinator, and keeps running; the coordinator spawns a fresh //! thread for the child. The parent and child are disjoint sub-parties, so the @@ -152,10 +152,10 @@ use rumors::link::{Connector, Done, Link, LinkParts, MemoryAcceptor, MemoryConne use rumors::{Peer, Retire, Rumors, UnorderedMessages, Version}; use tokio::io::{AsyncRead, AsyncWrite, DuplexStream, ReadBuf}; -/// Mint a genuine party-disjoint peer that inherits `parent`'s content. +/// Create a genuine party-disjoint peer that inherits `parent`'s content. /// /// Every party in the swarm — which independently `send`s, `redact`s, and -/// `gossip`s — needs its own disjoint Interval Tree Clock region. We mint one +/// `gossip`s — needs its own disjoint Interval Tree Clock region. We create one /// by serving a bootstrap from `parent` over an in-memory link: the /// newcomer pulls `parent`'s whole tree through the ordinary mirror descent /// and is handed a fresh disjoint party, forked in the same critical section @@ -427,10 +427,13 @@ fn main() -> io::Result<()> { let seed: Rumors = Peer::seed().into_rumors(); { let mut rng = SmallRng::from_entropy(); - let mut batch = seed.batch(); - for _ in 0..args.seed_messages { - batch.send(random_message(&mut rng, args.message_size)); - } + seed.batch(|batch| { + for _ in 0..args.seed_messages { + batch.send(random_message(&mut rng, args.message_size))?; + } + Ok::<(), rumors::EncodeError>(()) + }) + .expect("flat test payloads are within any depth limit"); } // Every party starts as a disjoint fork of the seed: same observations, // its own party region. The seed party itself only serves the initial @@ -569,7 +572,7 @@ fn run_party( while let Ok(cmd) = control.try_recv() { match cmd { Command::Fork { reply } => { - // Mint a genuine disjoint child that inherits our content, + // Create a genuine disjoint child that inherits our content, // so it can independently churn and gossip; its thread // rebuilds the version pool by observer replay. We keep // running unchanged. @@ -834,9 +837,11 @@ fn steady_state_op( } } } - // The add arm — or a pool with no live message left in it. The minted + // The add arm — or a pool with no live message left in it. The send's // version reaches the pool through the observer's next drain. - rumors.send(random_message(rng, message_size)); + rumors + .send(random_message(rng, message_size)) + .expect("flat payload"); } /// Draw an exponential inter-arrival time with the current mean, so successive diff --git a/examples/swarm/tests.rs b/examples/swarm/tests.rs index 6104c1ebe..14a640483 100644 --- a/examples/swarm/tests.rs +++ b/examples/swarm/tests.rs @@ -88,10 +88,13 @@ fn controller_converges_through_retargeting() { let seed: Rumors = Peer::seed().into_rumors(); { let mut rng = SmallRng::seed_from_u64(0x5eed); - let mut batch = seed.batch(); - for _ in 0..100 { - batch.send(random_message(&mut rng, TEST_MESSAGE_SIZE)); - } + seed.batch(|batch| { + for _ in 0..100 { + batch.send(random_message(&mut rng, TEST_MESSAGE_SIZE))?; + } + Ok::<(), rumors::EncodeError>(()) + }) + .expect("flat test payloads are within any depth limit"); } let mut parties = [ Party::new( diff --git a/justfile b/justfile index 7ab401de9..648b68400 100644 --- a/justfile +++ b/justfile @@ -141,12 +141,18 @@ clippy: # features with warnings not denied, so without this leg that surface never # meets -D warnings anywhere). Each package is linted alone so workspace # feature unification cannot re-light the gated features. +# +# The bare-lib rumors line lints the shipped configuration on its own: an +# invocation that also builds test targets compiles the lib with +# cfg(test)-gated modules alive, so an item dead only in the default-feature +# lib — the artifact users actually build — never surfaces there. # Lint the default-feature library and test builds, warnings denied. clippy-default: cargo clippy -p suanpan --lib --tests -- -D warnings cargo clippy -p before --lib --tests -- -D warnings cargo clippy -p rumors --lib --tests -- -D warnings + cargo clippy -p rumors --lib -- -D warnings # Format the whole workspace. fmt: @@ -194,6 +200,17 @@ workflowlint: ./tools/workflowlint --self-test ./tools/workflowlint .github +# tools/digestshare reads the committed V2 wire captures and totals digest +# vs non-digest bytes. As a gate leg it checks the renderer-vocabulary +# contract, not a threshold: the tool exits nonzero when the corpus's +# byte-count headers or digest annotations stop matching its patterns (the +# renderer's vocabulary moved out from under the meter), never on the +# measured ratio. Build-free, so it rides the lint tier. + +# Check the wire-capture renderer vocabulary via the digest-share meter. +digestshare: + ./tools/digestshare + # tools/readme mirrors each crate's crate-level rustdoc into its README via # cargo-rdme, then strips the intra-doc links cargo-rdme can't resolve (the # public types are re-exported from private submodules, and the docs use @@ -375,7 +392,7 @@ fuzz-build: gate: gate-lints gate-streams # The build-free tier, sequential: a lint failure should cost seconds. -gate-lints: fmt-check doclint testdoc workflowlint mutants-list readme-check +gate-lints: fmt-check doclint testdoc workflowlint digestshare mutants-list readme-check # Each stream's output is captured rather than interleaved, and a failing # stream's log is replayed in full at the end, so a parallel failure reads @@ -973,7 +990,7 @@ worst-cases-pin: # tripwire, so the judge's red path rides every sweep. # Build everything (no fuzz run): the no-rot sweep as CI runs it. -ci: fmt-check doclint testdoc workflowlint readme-check fuelscape-claims mutants-list clippy clippy-default features wasm-check docs docs-internal test-all citecheck doctest bench-build fuzz-build fuelscape-verify viz +ci: fmt-check doclint testdoc workflowlint digestshare readme-check fuelscape-claims mutants-list clippy clippy-default features wasm-check docs docs-internal test-all citecheck doctest bench-build fuzz-build fuelscape-verify viz # Everything: the no-rot sweep, plus the fuzz smoke, the formal tier, and the bench judge. all: ci (fuzz fuzz_smoke_secs) lean eventdag muxprobe bench-judge bench-judge-tripwire diff --git a/src/batch.rs b/src/batch.rs index 8c648e7ff..fc0fd96ca 100644 --- a/src/batch.rs +++ b/src/batch.rs @@ -1,101 +1,98 @@ +use std::sync::Arc; + use tokio::sync::watch; -use crate::message::Message; +use crate::message::{EncodeError, PayloadCodec}; use crate::tree::Action; use crate::tree::typed::Path; use crate::{Inner, Version}; -use serde::Serialize; -/// A batch of insertions and redactions against a [`Rumors`](crate::Rumors), -/// applied in one commit. -/// -/// Returned by [`send`](crate::Rumors::send), -/// [`redact`](crate::Rumors::redact), and [`batch`](crate::Rumors::batch) on -/// [`Rumors`](crate::Rumors). Dropping the batch commits it: the single-action -/// case reads as a plain call (`rumors.send(message);` commits at the end of -/// the statement), and chaining accumulates -/// (`rumors.batch().send(a).send(b).redact(&version);`) into one commit. -/// -/// # A batch is a performance optimization, not an atomicity guarantee +/// The scope handle for a batch of insertions and redactions against a +/// [`Rumors`](crate::Rumors), applied in one all-or-nothing commit. /// -/// Batching coalesces several actions into one tree traversal, one commit -/// moment, and at most one internal gossip wakeup, instead of one per -/// action. When the batch drops: +/// Handed exclusively to the closure [`Rumors::batch`](crate::Rumors::batch) +/// runs: queue actions on it with [`send`](Self::send) and +/// [`redact`](Self::redact), and the batch commits — atomically, as one +/// commit — exactly when the closure returns `Ok`. Any other exit +/// (a returned `Err`, a panic) commits nothing; [`Rumors::batch`] states +/// the full lifecycle. /// -/// - **Dropped normally**, the batch commits everything queued so far, as -/// one commit: observers and concurrent gossip sessions see all of it -/// land at once, never a partially applied commit. -/// - **Dropped by a panic's unwind**, the batch commits nothing: the -/// caller never finished building it, so nothing it holds publishes. -/// - **Dropped by async cancellation** (the future holding it across an -/// `.await` is dropped), the batch commits the prefix queued before the -/// cancellation point. Cancellation runs no unwind, so this drop is -/// indistinguishable from an ordinary end-of-statement commit. +/// [`Rumors::batch`]: crate::Rumors::batch /// -/// An application that needs several pieces delivered all-or-nothing even -/// under panic or cancellation should not reach for a batch: bundle the -/// pieces into one application-level message in your definition of the -/// application's message type `T`. -/// -/// Building a [`Batch`] holds no lock; batches are serialized only upon -/// commit. Because building holds no lock, concurrent gossip rounds -/// can land between building and committing, and two batches carry no -/// guaranteed causal relationship to one another unless the application -/// synchronizes them itself. +/// Building a batch holds no lock; a batch is serialized against other +/// commits only at its own commit. Because building holds no lock, +/// concurrent gossip rounds can land between building and committing, and +/// two batches carry no guaranteed causal relationship to one another +/// unless the application synchronizes them itself. pub struct Batch<'a, T: Send + Sync> { inner: &'a watch::Sender>, + /// The peer's payload codec: every queued send serializes and + /// depth-checks through it. + codec: PayloadCodec, actions: Vec, } impl<'a, T: Send + Sync> Batch<'a, T> { - pub(crate) fn new(inner: &'a watch::Sender>) -> Self { + pub(crate) fn new(inner: &'a watch::Sender>, codec: PayloadCodec) -> Self { Self { inner, + codec, actions: Vec::new(), } } - /// Sends a message as part of this batch. + /// Queues a message for this batch's commit. + /// + /// Serialization and admission run here, not at commit: the message + /// is serialized through the peer's codec immediately, and a payload + /// a receiver would reject or misread — one nesting deeper than the + /// peer's + /// [`payload_depth_limit`](crate::Peer::payload_depth_limit), one + /// whose type does not survive its own serde round-trip, or one + /// whose encoding decodes to a different value — is the typed + /// [`EncodeError`], surfacing at the offending call + /// ([`Rumors::send`](crate::Rumors::send) states the admission + /// contract). Propagating the + /// error out of the closure cancels the whole batch + /// ([`Rumors::batch`](crate::Rumors::batch)'s commit-on-`Ok` + /// contract); handling it locally keeps the batch alive with the + /// offending message not queued. /// /// # Panics /// - /// If `message` fails to serialize. Serialization runs here, not at - /// commit: the failure surfaces at the offending call. - pub fn send(&mut self, message: T) -> &mut Self + /// If `message` fails to serialize: a violation of the payload + /// contract ([choosing a payload + /// type](crate#choosing-a-payload-type)), exactly as + /// [`Rumors::send`](crate::Rumors::send) treats it. + pub fn send(&mut self, message: T) -> Result<(), EncodeError> where - T: Serialize + 'static, + T: 'static, { - self.actions.push(Action::Insert(Message::new(message))); - self + let message = self.codec.message(Arc::new(message))?; + self.actions.push(Action::Insert(message)); + Ok(()) } - /// Redacts the message stamped with `version` as part of this batch. + /// Queues a redaction of the message stamped with `version` for this + /// batch's commit. /// /// Redacting a version not held at commit time is a no-op. - pub fn redact(&mut self, version: &Version) -> &mut Self { + pub fn redact(&mut self, version: &Version) { self.actions.push(Action::Forget(Path::for_leaf(version))); - self } -} -impl Drop for Batch<'_, T> { - fn drop(&mut self) { - // A drop reached by a panic's unwind commits nothing: the caller - // never finished building the batch, and nothing a half-built - // batch holds may publish. This also covers an unrelated panic - // unwinding over a held batch: RAII-transaction style, an unwound - // batch aborts. The guard sees only unwinds: a drop by async - // cancellation arrives outside any panic and commits the queued - // prefix, the documented hazard the type docs state, pinned by - // `a_cancelled_batch_commits_its_prefix` in `tests/single_peer.rs`. - if std::thread::panicking() { - return; - } - if self.actions.is_empty() { - return; - } - let actions = std::mem::take(&mut self.actions); - self.inner.send_if_modified(|inner| { + /// Commit everything queued, as one commit. + /// + /// Observers and concurrent gossip sessions see all of it land at + /// once, in at most one observer wakeup. Runs iff the caller's + /// closure returned `Ok` + /// ([`Rumors::batch`](crate::Rumors::batch) owns that decision). + pub(crate) fn commit(self) { + let Batch { inner, actions, .. } = self; + // An empty action list needs no special case: `Tree::act` + // documents an empty batch as a complete no-op, and its false + // changed flag suppresses the wakeup. + inner.send_if_modified(|inner| { // The party is present on every reachable handle: `retire` // consumes the `Peer`, and the `Peer`/`Rumors` XOR keeps a // retiring set's handles from coexisting with it. diff --git a/src/bookmark.rs b/src/bookmark.rs index 5435bdae4..7064889a7 100644 --- a/src/bookmark.rs +++ b/src/bookmark.rs @@ -21,7 +21,7 @@ use crate::Network; pub(crate) mod format; -pub use format::{BOOKMARK_FORMAT_VERSION, BOOKMARK_MAGIC, FormatError}; +pub use format::{BOOKMARK_FORMAT_VERSION, FormatError, FrameDefect, RecordDefect}; /// The error a [`Bookmark`] reports when persistence fails. pub trait BookmarkError { diff --git a/src/bookmark/format.rs b/src/bookmark/format.rs index 35454f3c2..55d1b3824 100644 --- a/src/bookmark/format.rs +++ b/src/bookmark/format.rs @@ -2,25 +2,47 @@ //! //! A [`Bookmark`](super::Bookmark) lends raw byte storage; *this* module owns //! what those bytes are. A stored record is a single self-describing, -//! self-checking frame: +//! self-checking frame, and the whole file parses as one CBOR item — a +//! generic CBOR tool with no rumors knowledge unfolds it completely: //! //! ```text -//! [ magic : 14 bytes = b"RUMORSBOOKMARK" -//! | version : 2 bytes (big-endian u16, BOOKMARK_FORMAT_VERSION) -//! | hash : 32 bytes BLAKE3(magic ‖ version ‖ payload) -//! | payload : N bytes CBOR(BTreeMap>) ] +//! 55799([ ; self-described CBOR (RFC 8949) +//! format_version : uint, ; BOOKMARK_FORMAT_VERSION +//! integrity : bstr .size 32, ; BLAKE3, coverage below +//! payload : 24(bstr .cbor map), ; the record, embedded +//! ]) //! ``` //! -//! The magic and version tag reject a foreign or future file *loudly* — a -//! non-bookmark or a format this build does not understand is an error, never a -//! misparse. The hash covers the whole frame body, so a truncated or bit-rotted -//! file is caught before its bytes are ever decoded into a [`Clock`] — the -//! silent-divergence failure mode this crate exists to prevent. +//! The opening self-described tag and the format version reject a foreign or +//! future file *loudly* — a non-bookmark or a format this build does not +//! understand is an error, never a misparse. The integrity hash covers the +//! encoded bytes of the format-version item and of the whole payload item +//! (tag 24 header included): every item of the frame array except the +//! integrity item itself. The frame's fixed opening (the self-described tag +//! and the array header) is outside the hash because any corruption there +//! already fails shape validation before the hash is consulted; a truncated +//! or bit-rotted file is caught before its bytes are ever decoded into a +//! [`Clock`] — the silent-divergence failure mode this crate exists to +//! prevent. +//! +//! The payload rides as an *embedded CBOR data item* (tag 24): the region the +//! hash covers is a CBOR-visible item, not an offset convention, and the +//! record inside is one CBOR map from each 16-byte network identifier to an +//! array of clocks, every clock a [`CLOCK_TAG`]-tagged byte string wrapping +//! its canonical encoding. The tags are written and read here, by the codec — +//! the atom types' serde implementations stay untagged and format-agnostic. //! //! The hash is a plain [`blake3`] digest, deliberately *not* the tree's //! path-identity hash: that type's contract is identity (a leaf's path), a //! different concern from this one's local, non-adversarial corruption check. //! +//! The frame is deterministic-encoding CBOR: shortest-form headers +//! everywhere, one spelling per value. The frame's own heads admit only +//! their canonical spelling; the embedded payload is decoded by a general +//! CBOR reader, so its one-spelling property is the encoder's (equal +//! records produce equal files, and the byte-for-byte format pins stay +//! meaningful), not an ingress check. +//! //! The framing ([`frame`]/[`unframe`]) is kept separate from the record codec //! ([`encode`]/[`decode`]) so the byte framing can be property-tested over //! arbitrary payloads, independent of the `!Clone` [`Clock`]s a real record @@ -29,35 +51,137 @@ use std::collections::BTreeMap; use before::Clock; +use ciborium::value::Value; use crate::Network; +use crate::tags::CLOCK_TAG; -/// Magic bytes that open every persisted bookmark frame. +/// On-disk bookmark format version, the first item of the frame array. /// -/// Distinct from the gossip wire's [`PROTOCOL_MAGIC`](crate::PROTOCOL_MAGIC): -/// the on-disk format and the wire protocol are versioned independently, so -/// bumping one never forces the other. -pub const BOOKMARK_MAGIC: [u8; 14] = *b"RUMORSBOOKMARK"; +/// Version 4 is the fully CBOR-parseable frame: the self-described tag, this +/// version, the integrity hash, and the record as an embedded CBOR item +/// (clocks as [`CLOCK_TAG`]-tagged byte strings wrapping their canonical +/// codec, skyline version coding inside). A file carrying any other version +/// is rejected with [`FormatError::VersionMismatch`] rather than misread; +/// there is no migration path. +pub const BOOKMARK_FORMAT_VERSION: u64 = 4; -/// On-disk bookmark format version, following [`BOOKMARK_MAGIC`]. +/// The three opening bytes of every frame: the CBOR self-described tag 55799. /// -/// Bumped whenever the frame layout or payload encoding changes — version 3 -/// carries the CBOR record encoding (clocks as byte strings wrapping their -/// canonical codec) with skyline version coding inside. A file carrying -/// any other version is rejected with [`FormatError::VersionMismatch`] -/// rather than misread; there is no migration path. -pub const BOOKMARK_FORMAT_VERSION: u16 = 3; - -/// Byte offset of the version field within a frame. -const VERSION_OFFSET: usize = BOOKMARK_MAGIC.len(); -/// Byte offset of the integrity hash within a frame. -const HASH_OFFSET: usize = VERSION_OFFSET + 2; +/// This is CBOR's own magic (RFC 8949 §3.4.6) — it says "CBOR", not +/// "rumors"; what makes the file a *bookmark* is the frame shape and the +/// format version behind it. +const SELF_DESCRIBED: [u8; 3] = [0xd9, 0xd9, 0xf7]; + +/// The frame array's header byte: a definite-length array of three items. +const FRAME_ARRAY: u8 = 0x83; + +/// The integrity item's header: a 32-byte byte string. +const INTEGRITY_HEAD: [u8; 2] = [0x58, 0x20]; + /// Width of the BLAKE3 integrity hash, in bytes. const HASH_LEN: usize = 32; -/// Byte offset of the payload within a frame: the end of the fixed header. -const PAYLOAD_OFFSET: usize = HASH_OFFSET + HASH_LEN; -/// Total fixed-header width: magic, version, and hash, before the payload. -const HEADER_LEN: usize = PAYLOAD_OFFSET; + +/// The payload item's tag header: tag 24, "encoded CBOR data item". +const EMBEDDED_CBOR: [u8; 2] = [0xd8, 0x18]; + +/// CBOR major types, pre-shifted into a header byte's high bits. +const MAJOR_UNSIGNED: u8 = 0 << 5; +const MAJOR_BYTES: u8 = 2 << 5; + +/// Which part of the frame's fixed shape failed to parse. +/// +/// Carried by [`FormatError::NotABookmark`]: the bytes are present but are not a +/// bookmark frame at the named position. +#[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)] +#[non_exhaustive] +pub enum FrameDefect { + /// The frame does not open with the CBOR self-described tag. + #[error("no self-described CBOR tag")] + SelfDescribedTag, + + /// The tagged item is not the three-item frame array. + #[error("not the three-item frame array")] + FrameArray, + + /// The format-version item is not a shortest-form unsigned integer. + #[error("malformed format-version item")] + FormatVersion, + + /// The integrity item is not a 32-byte byte string. + #[error("malformed integrity item")] + Integrity, + + /// The payload item does not carry the embedded-CBOR tag. + #[error("payload is not an embedded CBOR item")] + PayloadTag, + + /// The payload item's byte string header is malformed or non-canonical. + #[error("malformed payload byte string")] + PayloadByteString, + + /// Bytes continue past the end of the frame array. + #[error("trailing bytes after the frame")] + TrailingBytes, +} + +/// Why an intact frame's payload is not the record this codec writes. +/// +/// Carried by [`FormatError::Record`]. The frame passed its integrity check, +/// so every variant is a logic error — the bytes are the ones that were +/// written — never corruption. +#[derive(Debug, thiserror::Error)] +#[non_exhaustive] +pub enum RecordDefect { + /// The payload is not parseable as a CBOR item at all. + #[error("the payload does not parse as CBOR: {0}")] + Cbor(#[source] ciborium::de::Error), + + /// Bytes continue past the payload's single CBOR item. + #[error("{trailing} trailing bytes after the bookmark record")] + TrailingBytes { + /// How many bytes follow the record item. + trailing: usize, + }, + + /// The record item is not a map. + #[error("the bookmark record is not a map")] + NotAMap, + + /// A record key is not a byte string. + #[error("a record key is not a byte string")] + KeyNotBytes, + + /// A record key byte string is not a 16-byte network identifier. + #[error("a network identifier is exactly 16 bytes, found {len}")] + KeyWidth { + /// The width of the key actually found. + len: usize, + }, + + /// A record entry's value is not an array of clocks. + #[error("a record entry is not an array of clocks")] + ClocksNotArray, + + /// A stored clock carries no CBOR tag. + #[error("a stored clock is untagged")] + ClockUntagged, + + /// A stored clock carries a tag other than the clock tag. + #[error("a stored clock carries tag {found}, not the clock tag")] + ClockTag { + /// The tag number actually found. + found: u64, + }, + + /// A stored clock's tagged item is not a byte string. + #[error("a stored clock is not a byte string")] + ClockNotBytes, + + /// A stored clock byte string fails the strict clock decoder. + #[error("a stored clock would not decode")] + Clock(#[source] before::error::Decode), +} /// Why a stored bookmark could not be turned back into a record. /// @@ -67,32 +191,35 @@ const HEADER_LEN: usize = PAYLOAD_OFFSET; #[derive(Debug, thiserror::Error)] #[non_exhaustive] pub enum FormatError { - /// Fewer bytes than the fixed header: a truncated or empty file. (An + /// The bytes end inside the frame: a truncated or empty file. (An /// *absent* bookmark is reported by [`load`](super::Bookmark::load) /// returning `None`, never as an empty frame.) - #[error("bookmark too short: {len} bytes (need at least {})", HEADER_LEN)] + #[error("bookmark truncated: the {len} bytes present end inside the frame")] Truncated { /// How many bytes were actually present. len: usize, }, - /// The leading bytes are not [`BOOKMARK_MAGIC`]: this is not a bookmark. - #[error("not a rumors bookmark: unexpected magic bytes")] - BadMagic { - /// The magic bytes actually found. - found: [u8; BOOKMARK_MAGIC.len()], + /// The bytes are not a self-described CBOR bookmark frame: this is not a + /// bookmark. + #[error("not a rumors bookmark: {defect}")] + NotABookmark { + /// Which part of the frame shape failed. + #[source] + defect: FrameDefect, }, - /// A bookmark, but a format version this build does not understand. + /// A bookmark frame, but a format version this build does not understand. #[error( "unsupported bookmark format version {found} (this build writes {BOOKMARK_FORMAT_VERSION})" )] VersionMismatch { /// The format version the file declared. - found: u16, + found: u64, }, - /// The integrity hash does not match the body: the file is corrupt. + /// The integrity hash does not match the covered items: the file is + /// corrupt. #[error("bookmark integrity hash mismatch: stored record is corrupt")] HashMismatch, @@ -100,73 +227,184 @@ pub enum FormatError { #[error("reading the stored bookmark failed: {0}")] Read(#[source] std::io::Error), - /// The frame was well-formed and intact, but its payload would not decode — - /// a logic error, since a matching hash means the bytes are the ones that - /// were written. - #[error("decoding the bookmark payload failed: {0}")] - Decode(#[source] std::io::Error), + /// The frame was well-formed and intact, but its payload is not the + /// record this codec writes — a logic error, since a matching hash means + /// the bytes are the ones that were written. + #[error("the bookmark payload is not a record this codec writes: {0}")] + Record(#[source] RecordDefect), } -/// Wrap `payload` in a bookmark frame: prepend the magic and version tag and a -/// BLAKE3 hash over `magic ‖ version ‖ payload`. +/// Append the shortest-form CBOR header for `arg` under `major` (pre-shifted). +fn push_head(out: &mut Vec, major: u8, arg: u64) { + match arg { + 0..=23 => out.push(major | arg as u8), + 24..=0xff => { + out.push(major | 24); + out.push(arg as u8); + } + 0x100..=0xffff => { + out.push(major | 25); + out.extend_from_slice(&(arg as u16).to_be_bytes()); + } + 0x1_0000..=0xffff_ffff => { + out.push(major | 26); + out.extend_from_slice(&(arg as u32).to_be_bytes()); + } + _ => { + out.push(major | 27); + out.extend_from_slice(&arg.to_be_bytes()); + } + } +} + +/// Wrap `payload` in a bookmark frame declaring `version`. /// -/// The inverse of [`unframe`]. -pub(crate) fn frame(payload: &[u8]) -> Vec { - let version = BOOKMARK_FORMAT_VERSION.to_be_bytes(); +/// Split from [`frame`] so the tests can build otherwise-valid frames +/// carrying a rejected version: the hash is computed over whatever version is +/// written, so version rejection is exercised on its own, not shadowed by +/// [`FormatError::HashMismatch`]. +fn frame_as(version: u64, payload: &[u8]) -> Vec { + // The hash's covered region: the encoded format-version item followed by + // the encoded payload item (tag 24, byte-string header, payload bytes) — + // built first, exactly as it will appear in the frame. + let mut covered = Vec::with_capacity(9 + 2 + 9 + payload.len()); + push_head(&mut covered, MAJOR_UNSIGNED, version); + let version_item_len = covered.len(); + covered.extend_from_slice(&EMBEDDED_CBOR); + push_head(&mut covered, MAJOR_BYTES, payload.len() as u64); + covered.extend_from_slice(payload); + let hash = blake3::hash(&covered); - let mut hasher = blake3::Hasher::new(); - hasher.update(&BOOKMARK_MAGIC); - hasher.update(&version); - hasher.update(payload); - let hash = hasher.finalize(); - - let mut out = Vec::with_capacity(HEADER_LEN + payload.len()); - out.extend_from_slice(&BOOKMARK_MAGIC); - out.extend_from_slice(&version); + let mut out = Vec::with_capacity(SELF_DESCRIBED.len() + 1 + 2 + HASH_LEN + covered.len()); + out.extend_from_slice(&SELF_DESCRIBED); + out.push(FRAME_ARRAY); + out.extend_from_slice(&covered[..version_item_len]); + out.extend_from_slice(&INTEGRITY_HEAD); out.extend_from_slice(hash.as_bytes()); - out.extend_from_slice(payload); + out.extend_from_slice(&covered[version_item_len..]); out } +/// Wrap `payload` in a bookmark frame: the self-described tag, the frame +/// array, the format version, a BLAKE3 hash over the version and payload +/// items, and the payload as an embedded CBOR item. +/// +/// The inverse of [`unframe`]. +pub(crate) fn frame(payload: &[u8]) -> Vec { + frame_as(BOOKMARK_FORMAT_VERSION, payload) +} + +/// A strict, position-tracking reader over a candidate frame. +/// +/// Running out of bytes is [`FormatError::Truncated`]; bytes that differ from +/// the demanded spelling are [`FormatError::NotABookmark`] with the caller's +/// defect. Together the two carry the totality of the shape check: every byte +/// of the frame is either compared against a fixed spelling, parsed as a +/// shortest-form header, hashed, or payload. +struct Reader<'a> { + bytes: &'a [u8], + at: usize, +} + +impl<'a> Reader<'a> { + /// Take the next `n` bytes, or report the frame truncated. + fn take(&mut self, n: usize) -> Result<&'a [u8], FormatError> { + let end = self + .at + .checked_add(n) + .filter(|&end| end <= self.bytes.len()) + .ok_or(FormatError::Truncated { + len: self.bytes.len(), + })?; + let taken = &self.bytes[self.at..end]; + self.at = end; + Ok(taken) + } + + /// Demand the exact bytes `spelling` next, else the named `defect`. + fn expect(&mut self, spelling: &[u8], defect: FrameDefect) -> Result<(), FormatError> { + if self.take(spelling.len())? != spelling { + return Err(FormatError::NotABookmark { defect }); + } + Ok(()) + } + + /// Parse a shortest-form unsigned-int header of the expected `major` + /// (pre-shifted), returning its argument, else the named `defect`. + fn head(&mut self, major: u8, defect: FrameDefect) -> Result { + let initial = self.take(1)?[0]; + if initial & 0xe0 != major { + return Err(FormatError::NotABookmark { defect }); + } + let (arg, floor) = match initial & 0x1f { + small @ 0..=23 => return Ok(u64::from(small)), + 24 => (u64::from(self.take(1)?[0]), 24), + 25 => ( + u64::from(u16::from_be_bytes(self.take(2)?.try_into().expect("two"))), + 0x100, + ), + 26 => ( + u64::from(u32::from_be_bytes(self.take(4)?.try_into().expect("four"))), + 0x1_0000, + ), + 27 => ( + u64::from_be_bytes(self.take(8)?.try_into().expect("eight")), + 0x1_0000_0000, + ), + _ => return Err(FormatError::NotABookmark { defect }), + }; + // The frame is deterministic-encoding CBOR: a header wider than its + // argument needs is a spelling this codec never writes. + if arg < floor { + return Err(FormatError::NotABookmark { defect }); + } + Ok(arg) + } +} + /// Validate a bookmark frame and return its payload slice. /// -/// Checks, in order: length against the fixed header, magic, version, then the -/// integrity hash. The inverse of [`frame`]: `unframe(&frame(p)) == Ok(p)`. +/// Checks, in order: the self-described tag and frame shape, the format +/// version, then the integrity hash over the version and payload items. The +/// inverse of [`frame`]: `unframe(&frame(p)) == Ok(p)`. /// /// # Errors /// -/// [`FormatError::Truncated`], [`BadMagic`](FormatError::BadMagic), +/// [`FormatError::Truncated`], [`NotABookmark`](FormatError::NotABookmark), /// [`VersionMismatch`](FormatError::VersionMismatch), or -/// [`HashMismatch`](FormatError::HashMismatch) — each pinpointing how the bytes -/// failed to be a frame this build can trust. +/// [`HashMismatch`](FormatError::HashMismatch) — each pinpointing how the +/// bytes failed to be a frame this build can trust. pub(crate) fn unframe(bytes: &[u8]) -> Result<&[u8], FormatError> { - if bytes.len() < HEADER_LEN { - return Err(FormatError::Truncated { len: bytes.len() }); - } + let mut reader = Reader { bytes, at: 0 }; + reader.expect(&SELF_DESCRIBED, FrameDefect::SelfDescribedTag)?; + reader.expect(&[FRAME_ARRAY], FrameDefect::FrameArray)?; - let magic: [u8; BOOKMARK_MAGIC.len()] = bytes[..VERSION_OFFSET] - .try_into() - .expect("magic is checked"); - if magic != BOOKMARK_MAGIC { - return Err(FormatError::BadMagic { found: magic }); - } - - let version = u16::from_be_bytes( - bytes[VERSION_OFFSET..HASH_OFFSET] - .try_into() - .expect("two version bytes"), - ); + let version_start = reader.at; + let version = reader.head(MAJOR_UNSIGNED, FrameDefect::FormatVersion)?; + let version_end = reader.at; if version != BOOKMARK_FORMAT_VERSION { return Err(FormatError::VersionMismatch { found: version }); } - let stored_hash = &bytes[HASH_OFFSET..PAYLOAD_OFFSET]; - let payload = &bytes[PAYLOAD_OFFSET..]; + reader.expect(&INTEGRITY_HEAD, FrameDefect::Integrity)?; + let stored_hash = reader.take(HASH_LEN)?; + + let payload_start = reader.at; + reader.expect(&EMBEDDED_CBOR, FrameDefect::PayloadTag)?; + let declared = reader.head(MAJOR_BYTES, FrameDefect::PayloadByteString)?; + if declared > (bytes.len() - reader.at) as u64 { + return Err(FormatError::Truncated { len: bytes.len() }); + } + let payload = reader.take(declared as usize).expect("length checked"); + if reader.at != bytes.len() { + return Err(FormatError::NotABookmark { + defect: FrameDefect::TrailingBytes, + }); + } let mut hasher = blake3::Hasher::new(); - hasher.update(&BOOKMARK_MAGIC); - hasher.update(&version.to_be_bytes()); - hasher.update(payload); + hasher.update(&bytes[version_start..version_end]); + hasher.update(&bytes[payload_start..]); if hasher.finalize().as_bytes().as_slice() != stored_hash { return Err(FormatError::HashMismatch); } @@ -176,40 +414,92 @@ pub(crate) fn unframe(bytes: &[u8]) -> Result<&[u8], FormatError> { /// Serialize a record into a complete bookmark frame. /// -/// CBOR-encodes the record, then [`frame`]s it. The inverse of [`decode`]. +/// Spells the record as one CBOR map — network identifier byte strings to +/// arrays of [`CLOCK_TAG`]-tagged clock byte strings — then [`frame`]s it. +/// The inverse of [`decode`]. pub(crate) fn encode(record: &BTreeMap>) -> Vec { - // Encoding to a `Vec` cannot fail: every field's serde form is a plain - // byte string or container, and a `Vec` never fails to extend. + let map = Value::Map( + record + .iter() + .map(|(network, clocks)| { + ( + Value::Bytes(network.to_bytes().to_vec()), + Value::Array( + clocks + .iter() + .map(|clock| { + Value::Tag(CLOCK_TAG, Box::new(Value::Bytes(clock.encode()))) + }) + .collect(), + ), + ) + }) + .collect(), + ); + // Encoding to a `Vec` cannot fail: every value is a plain byte string or + // container, and a `Vec` never fails to extend. let mut payload = Vec::new(); - ciborium::ser::into_writer(record, &mut payload) + ciborium::ser::into_writer(&map, &mut payload) .expect("encoding a record to a Vec is infallible"); frame(&payload) } /// Validate a bookmark frame and deserialize its record. /// -/// [`unframe`]s, then CBOR-decodes the payload, which must be exactly one -/// CBOR value. The inverse of [`encode`]. +/// [`unframe`]s, then walks the payload, which must be exactly one CBOR map +/// of the shape [`encode`] writes — every clock carrying [`CLOCK_TAG`]. The +/// inverse of [`encode`]. /// /// # Errors /// -/// Any [`unframe`] error, or [`FormatError::Decode`] if a frame that passed its -/// integrity check nonetheless held an undecodable payload (a logic error, not -/// corruption). +/// Any [`unframe`] error, or [`FormatError::Record`] if a frame that passed +/// its integrity check nonetheless held an undecodable payload (a logic +/// error, not corruption). pub(crate) fn decode(bytes: &[u8]) -> Result>, FormatError> { let payload = unframe(bytes)?; + walk(payload).map_err(FormatError::Record) +} + +/// Walk an unframed payload into the record it spells. +fn walk(payload: &[u8]) -> Result>, RecordDefect> { let mut input = payload; - let record = ciborium::de::from_reader(&mut input).map_err(|e| { - FormatError::Decode(match e { - ciborium::de::Error::Io(e) => e, - e => std::io::Error::new(std::io::ErrorKind::InvalidData, e.to_string()), - }) - })?; + let value: Value = ciborium::de::from_reader(&mut input).map_err(RecordDefect::Cbor)?; if !input.is_empty() { - return Err(FormatError::Decode(std::io::Error::new( - std::io::ErrorKind::InvalidData, - format!("{} trailing bytes after the bookmark record", input.len()), - ))); + return Err(RecordDefect::TrailingBytes { + trailing: input.len(), + }); + } + + let Value::Map(entries) = value else { + return Err(RecordDefect::NotAMap); + }; + let mut record = BTreeMap::new(); + for (key, clocks) in entries { + let Value::Bytes(key) = key else { + return Err(RecordDefect::KeyNotBytes); + }; + let network = Network::from_bytes( + key.as_slice() + .try_into() + .map_err(|_| RecordDefect::KeyWidth { len: key.len() })?, + ); + let Value::Array(clocks) = clocks else { + return Err(RecordDefect::ClocksNotArray); + }; + let mut decoded = Vec::with_capacity(clocks.len()); + for clock in clocks { + let Value::Tag(tag, boxed) = clock else { + return Err(RecordDefect::ClockUntagged); + }; + if tag != CLOCK_TAG { + return Err(RecordDefect::ClockTag { found: tag }); + } + let Value::Bytes(clock) = *boxed else { + return Err(RecordDefect::ClockNotBytes); + }; + decoded.push(Clock::decode(clock.as_slice()).map_err(RecordDefect::Clock)?); + } + record.insert(network, decoded); } Ok(record) } diff --git a/src/bookmark/format/snapshots/rumors__bookmark__format__tests__frame_empty.snap b/src/bookmark/format/snapshots/rumors__bookmark__format__tests__frame_empty.snap index 37d312aac..7808780bb 100644 --- a/src/bookmark/format/snapshots/rumors__bookmark__format__tests__frame_empty.snap +++ b/src/bookmark/format/snapshots/rumors__bookmark__format__tests__frame_empty.snap @@ -1,5 +1,16 @@ --- source: src/bookmark/format/tests.rs -expression: "hex::encode(encode(&BTreeMap::new()))" +expression: "annotated(&encode(&BTreeMap::new()))" --- -52554d4f5253424f4f4b4d41524b000332608c879e1aedee21ce002c988d34a3e0d21205800a036c1a1c839266777695a0 +d9d9f78304582067c95c28df909f2e5e18657636011941814b36aa68617abfe7ced6793215a9efd81841a0 + +55799( / self-described CBOR / + [ + 4 / bookmark format version / + h'67c95c28df909f2e5e18657636011941814b36aa68617abfe7ced6793215a9ef' / integrity: BLAKE3 of the embedded record / + 24(<< / embedded record, 1 byte(s) / + { / 0 network(s) / + } + >>) + ] +) diff --git a/src/bookmark/format/snapshots/rumors__bookmark__format__tests__frame_non_trivial.snap b/src/bookmark/format/snapshots/rumors__bookmark__format__tests__frame_non_trivial.snap index f5ee05fe2..339083626 100644 --- a/src/bookmark/format/snapshots/rumors__bookmark__format__tests__frame_non_trivial.snap +++ b/src/bookmark/format/snapshots/rumors__bookmark__format__tests__frame_non_trivial.snap @@ -1,5 +1,22 @@ --- source: src/bookmark/format/tests.rs -expression: "hex::encode(encode(&sample_record()))" +expression: annotated(&encode(&sample_record())) --- -52554d4f5253424f4f4b4d41524b0003524403588d34607dc2d33e5ad6534c3a4266ca837990c8bc270311589dd50bc6a1505a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a8344a22aa58044482aa5804292e0 +d9d9f783045820516ff347a0f0cf7b370e5e36fb03272cf932360e39c3bbd7506f6eeeb72af793d8185829a1505a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a83d9d25744a22aa580d9d25744482aa580d9d2574292e0 + +55799( / self-described CBOR / + [ + 4 / bookmark format version / + h'516ff347a0f0cf7b370e5e36fb03272cf932360e39c3bbd7506f6eeeb72af793' / integrity: BLAKE3 of the embedded record / + 24(<< / embedded record, 41 byte(s) / + { / 1 network(s) / + h'5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a' / network / => + [ / 3 clock(s) / + 53847(h'a22aa580') / clock / + 53847(h'482aa580') / clock / + 53847(h'92e0') / clock / + ] + } + >>) + ] +) diff --git a/src/bookmark/format/tests.rs b/src/bookmark/format/tests.rs index 0e129ff63..0e7a128a0 100644 --- a/src/bookmark/format/tests.rs +++ b/src/bookmark/format/tests.rs @@ -1,5 +1,6 @@ //! The frame is self-inverse and self-checking: it round-trips any payload, -//! rejects every single-byte corruption, and pins byte-for-byte. +//! rejects every single-byte corruption and every truncation, parses whole +//! under a rumors-blind CBOR reader, and pins byte-for-byte. use std::collections::BTreeMap; @@ -52,18 +53,20 @@ proptest! { prop_assert_eq!(unframe(&framed).unwrap(), payload.as_slice()); } - /// A frame always carries the magic and version tag in its header, whatever - /// the payload. + /// A frame always opens with the self-described CBOR tag, the three-item + /// frame array, and the format-version item, whatever the payload. #[test] fn frame_carries_the_tag(payload: Vec) { let framed = frame(&payload); - let version = BOOKMARK_FORMAT_VERSION.to_be_bytes(); - prop_assert!(framed.starts_with(&BOOKMARK_MAGIC)); - prop_assert_eq!(&framed[VERSION_OFFSET..HASH_OFFSET], version.as_slice()); + let mut opening = SELF_DESCRIBED.to_vec(); + opening.push(FRAME_ARRAY); + push_head(&mut opening, MAJOR_UNSIGNED, BOOKMARK_FORMAT_VERSION); + prop_assert!(framed.starts_with(&opening)); } - /// Flipping any one byte of the frame body (magic, version, hash, or - /// payload) makes it fail to validate: nothing corrupt is ever accepted. + /// Flipping any one byte of the frame (opening, version, hash, payload + /// headers, or payload) makes it fail to validate: nothing corrupt is + /// ever accepted. #[test] fn any_single_byte_corruption_is_rejected( payload in prop::collection::vec(any::(), 1..64), @@ -75,6 +78,35 @@ proptest! { prop_assert!(unframe(&framed).is_err()); } + /// Cutting a frame anywhere before its end fails to validate — a partial + /// write is caught as [`FormatError::Truncated`], never misread. + #[test] + fn truncation_at_every_prefix_is_rejected( + payload in prop::collection::vec(any::(), 0..64), + index: prop::sample::Index, + ) { + let framed = frame(&payload); + let cut = index.index(framed.len()); + let truncated = matches!( + unframe(&framed[..cut]), + Err(FormatError::Truncated { len }) if len == cut, + ); + prop_assert!(truncated); + } + + /// Bytes appended after the frame array fail to validate: the frame is + /// exactly one CBOR item, so a follower is a shape defect. + #[test] + fn trailing_bytes_are_rejected(payload: Vec, extra: u8) { + let mut framed = frame(&payload); + framed.push(extra); + let rejected = matches!( + unframe(&framed), + Err(FormatError::NotABookmark { defect: FrameDefect::TrailingBytes }), + ); + prop_assert!(rejected); + } + /// A record survives a serialize/validate/deserialize round trip unchanged, /// for an arbitrary number of forked clocks under an arbitrary network id. #[test] @@ -100,39 +132,181 @@ fn empty_record_round_trips() { assert!(decoded.is_empty()); } -/// Foreign leading bytes are rejected as [`FormatError::BadMagic`], not misread. +/// Foreign leading bytes are rejected as [`FormatError::NotABookmark`], not +/// misread — including a file that opens with plain ASCII where the +/// self-described tag belongs. #[test] fn foreign_magic_is_rejected() { let mut framed = encode(&sample_record()); framed[0] ^= 0xff; assert!(matches!( unframe(&framed), - Err(FormatError::BadMagic { .. }) + Err(FormatError::NotABookmark { + defect: FrameDefect::SelfDescribedTag + }) )); + + let ascii = b"RUMORSBOOKMARKISH TEXT, NOT CBOR"; + assert!(matches!( + unframe(ascii), + Err(FormatError::NotABookmark { + defect: FrameDefect::SelfDescribedTag + }) + )); +} + +/// The bookmark's opening literal is the self-described tag's one +/// canonical spelling: the shared constant is the authority, and the +/// pinned bytes cannot drift from the head writer's rendering of it. +#[test] +fn opening_literal_is_the_self_described_tag() { + use crate::tree::mirror::cbor; + let mut rendered = Vec::new(); + cbor::write_tag(&mut rendered, cbor::TAG_SELF_DESCRIBED); + assert_eq!(rendered, SELF_DESCRIBED); } -/// A frame tagged with an unknown format version is rejected, never decoded -/// under this build's assumptions. +/// A frame declaring an unknown format version is rejected on the version +/// alone — its hash is valid, so the rejection is +/// [`FormatError::VersionMismatch`], never decoded under this build's +/// assumptions. #[test] fn unknown_version_is_rejected() { - let mut framed = encode(&sample_record()); - framed[VERSION_OFFSET..HASH_OFFSET].copy_from_slice(&0xbeef_u16.to_be_bytes()); + let framed = frame_as(0xbeef, b"payload"); assert!(matches!( unframe(&framed), Err(FormatError::VersionMismatch { found: 0xbeef }), )); } -/// A version-1 frame — the packed per-node payload coding — is strictly -/// rejected: the version-2 skyline payloads share no decoder with it, and -/// there is deliberately no migration path. +/// Every earlier format version is strictly rejected: the earlier frame +/// shapes share no decoder with this one, and there is deliberately no +/// migration path. #[test] -fn version_one_is_rejected() { - let mut framed = encode(&sample_record()); - framed[VERSION_OFFSET..HASH_OFFSET].copy_from_slice(&1u16.to_be_bytes()); +fn prior_versions_are_rejected() { + for prior in 0..BOOKMARK_FORMAT_VERSION { + let framed = frame_as(prior, b"payload"); + assert!(matches!( + unframe(&framed), + Err(FormatError::VersionMismatch { found }) if found == prior, + )); + } +} + +/// A non-shortest-form spelling of the format version is rejected as a shape +/// defect even though its value matches: the frame is deterministic-encoding +/// CBOR, and a wide header is a spelling this codec never writes. +#[test] +fn non_canonical_version_spelling_is_rejected() { + // Rebuild a frame exactly as `frame_as` would, but spell version 4 as + // the two-byte header 0x18 0x04, hashing over that spelling so only the + // spelling check can reject it. + let payload = b"payload"; + let mut covered = vec![0x18, u8::try_from(BOOKMARK_FORMAT_VERSION).unwrap()]; + let version_item_len = covered.len(); + covered.extend_from_slice(&EMBEDDED_CBOR); + push_head(&mut covered, MAJOR_BYTES, payload.len() as u64); + covered.extend_from_slice(payload); + let hash = blake3::hash(&covered); + + let mut framed = SELF_DESCRIBED.to_vec(); + framed.push(FRAME_ARRAY); + framed.extend_from_slice(&covered[..version_item_len]); + framed.extend_from_slice(&INTEGRITY_HEAD); + framed.extend_from_slice(hash.as_bytes()); + framed.extend_from_slice(&covered[version_item_len..]); + + assert!(matches!( + unframe(&framed), + Err(FormatError::NotABookmark { + defect: FrameDefect::FormatVersion + }), + )); +} + +/// The encoded length of the format-version item in a frame this codec +/// writes: the offset arithmetic below computes it rather than +/// hardcoding it, so a version bump cannot silently skew the flips. +fn version_item_len() -> usize { + let mut version_item = Vec::new(); + push_head(&mut version_item, MAJOR_UNSIGNED, BOOKMARK_FORMAT_VERSION); + version_item.len() +} + +/// Corrupting the integrity item's header is rejected as the typed +/// [`FrameDefect::Integrity`] shape defect, distinct from a hash +/// mismatch: the header bytes are part of the frame's fixed spelling. +#[test] +fn corrupt_integrity_head_is_an_integrity_defect() { + let mut framed = frame(b"payload"); + let integrity_at = SELF_DESCRIBED.len() + 1 + version_item_len(); + assert_eq!( + framed[integrity_at], INTEGRITY_HEAD[0], + "the computed offset lands on the integrity head" + ); + framed[integrity_at] ^= 0xff; + assert!(matches!( + unframe(&framed), + Err(FormatError::NotABookmark { + defect: FrameDefect::Integrity + }), + )); +} + +/// Corrupting the payload item's tag byte is rejected as the typed +/// [`FrameDefect::PayloadTag`] shape defect: the embedded-CBOR tag is +/// part of the frame's fixed spelling, checked before the hash. +#[test] +fn corrupt_payload_tag_is_a_payload_tag_defect() { + let mut framed = frame(b"payload"); + let payload_tag_at = + SELF_DESCRIBED.len() + 1 + version_item_len() + INTEGRITY_HEAD.len() + HASH_LEN; + assert_eq!( + framed[payload_tag_at], EMBEDDED_CBOR[0], + "the computed offset lands on the payload tag" + ); + framed[payload_tag_at] ^= 0xff; assert!(matches!( unframe(&framed), - Err(FormatError::VersionMismatch { found: 1 }), + Err(FormatError::NotABookmark { + defect: FrameDefect::PayloadTag + }), + )); +} + +/// A non-shortest-form spelling of the payload byte-string length is +/// rejected as the typed [`FrameDefect::PayloadByteString`] defect. +/// +/// The value matches; only the spelling is wrong: the frame is +/// deterministic-encoding CBOR, and a wide header is a spelling this +/// codec never writes. +#[test] +fn non_canonical_payload_spelling_is_rejected() { + // Rebuild a frame exactly as `frame_as` would, but spell the 7-byte + // payload's byte-string head as the widened two-byte form 0x58 0x07, + // hashing over that spelling so only the spelling check can reject + // it. + let payload = b"payload"; + let mut covered = Vec::new(); + push_head(&mut covered, MAJOR_UNSIGNED, BOOKMARK_FORMAT_VERSION); + let version_item_len = covered.len(); + covered.extend_from_slice(&EMBEDDED_CBOR); + covered.extend_from_slice(&[MAJOR_BYTES | 24, u8::try_from(payload.len()).unwrap()]); + covered.extend_from_slice(payload); + let hash = blake3::hash(&covered); + + let mut framed = SELF_DESCRIBED.to_vec(); + framed.push(FRAME_ARRAY); + framed.extend_from_slice(&covered[..version_item_len]); + framed.extend_from_slice(&INTEGRITY_HEAD); + framed.extend_from_slice(hash.as_bytes()); + framed.extend_from_slice(&covered[version_item_len..]); + + assert!(matches!( + unframe(&framed), + Err(FormatError::NotABookmark { + defect: FrameDefect::PayloadByteString + }), )); } @@ -146,33 +320,180 @@ fn payload_corruption_is_rejected() { assert!(matches!(unframe(&framed), Err(FormatError::HashMismatch))); } -/// Anything shorter than the fixed header — including an empty buffer — is -/// [`FormatError::Truncated`], never mistaken for an absent bookmark. +/// The empty input is [`FormatError::Truncated`], never mistaken for an +/// absent bookmark. #[test] fn short_input_is_truncated() { assert!(matches!( unframe(&[]), - Err(FormatError::Truncated { len: 0 }), + Err(FormatError::Truncated { len: 0 }) )); - let framed = encode(&sample_record()); +} + +/// An intact frame whose payload item holds an untagged clock is a +/// [`RecordDefect`], not corruption: the hash passed, so the defect class is +/// [`FormatError::Record`]. +#[test] +fn untagged_clock_is_a_record_defect() { + let clock = Clock::seed(); + // The map `encode` writes, minus the clock's tag. + let map = ciborium::value::Value::Map(vec![( + ciborium::value::Value::Bytes(vec![0x5a; 16]), + ciborium::value::Value::Array(vec![ciborium::value::Value::Bytes(clock.encode())]), + )]); + let mut payload = Vec::new(); + ciborium::ser::into_writer(&map, &mut payload).unwrap(); assert!(matches!( - unframe(&framed[..HEADER_LEN - 1]), - Err(FormatError::Truncated { .. }), + decode(&frame(&payload)), + Err(FormatError::Record(RecordDefect::ClockUntagged)), )); } -/// The encoded empty record pins byte-for-byte: a header (magic, version, -/// integrity hash) over the CBOR encoding of an empty map. A change here is a -/// deliberate on-disk format change, like the wire-format snapshots. +/// The whole file parses as exactly one CBOR item under a reader that knows +/// nothing of rumors. +/// +/// Unwrapping the standard tags (55799, then 24) and the clock tag exposes +/// the record's full structure, with no bytes outside CBOR items at either +/// level. This is the tamper-evident form of the "fully CBOR-parseable on +/// disk" promise. +#[test] +fn file_is_rumors_blind_cbor() { + use ciborium::value::Value; + + let file = encode(&sample_record()); + let mut input = file.as_slice(); + let item: Value = ciborium::de::from_reader(&mut input).expect("the file parses as CBOR"); + assert!(input.is_empty(), "no bytes outside the one CBOR item"); + + let Value::Tag(55799, frame) = item else { + panic!("the file is not self-described CBOR"); + }; + let Value::Array(items) = *frame else { + panic!("the frame is not an array"); + }; + let [version, integrity, payload]: [Value; 3] = + items.try_into().expect("the frame array has three items"); + assert_eq!(version, Value::from(BOOKMARK_FORMAT_VERSION)); + let Value::Bytes(integrity) = integrity else { + panic!("the integrity item is not a byte string"); + }; + assert_eq!(integrity.len(), HASH_LEN); + + let Value::Tag(24, embedded) = payload else { + panic!("the payload is not an embedded CBOR item"); + }; + let Value::Bytes(embedded) = *embedded else { + panic!("the embedded item is not a byte string"); + }; + let mut inner = embedded.as_slice(); + let record: Value = ciborium::de::from_reader(&mut inner).expect("the payload parses as CBOR"); + assert!(inner.is_empty(), "no bytes outside the record item"); + + let Value::Map(entries) = record else { + panic!("the record is not a map"); + }; + for (key, clocks) in entries { + assert!(matches!(key, Value::Bytes(bytes) if bytes.len() == 16)); + let Value::Array(clocks) = clocks else { + panic!("a record entry is not an array"); + }; + for clock in clocks { + assert!( + matches!(clock, Value::Tag(tag, inner) + if tag == crate::tags::CLOCK_TAG && matches!(*inner, Value::Bytes(_))), + "every stored clock is a clock-tagged byte string", + ); + } + } +} + +/// Render a bookmark frame for the byte pins: exact hex, then annotation. +/// +/// The frame's exact hex rides the first line — the pin itself, which an +/// annotation change leaves byte-identical — followed by a decoded, +/// annotated reading of the same bytes in the wire captures' idiom. +/// +/// The integrity digest is sliced from the frame's own bytes, never +/// recomputed, so the annotation reads what the pin holds; `unframe` has +/// already verified it against the payload. +fn annotated(frame: &[u8]) -> String { + use crate::tree::mirror::cbor::{TAG_EMBEDDED_ITEM, TAG_SELF_DESCRIBED}; + use std::fmt::Write; + let record = decode(frame).expect("the pinned frame decodes"); + let payload = unframe(frame).expect("the pinned frame unframes"); + // Fixed offsets: the version head is a single byte for any version + // below 24, and everything before the digest is a pinned spelling. + const { assert!(BOOKMARK_FORMAT_VERSION < 24, "the version head is one byte") }; + let hash_at = SELF_DESCRIBED.len() + 1 + 1 + INTEGRITY_HEAD.len(); + let integrity = &frame[hash_at..hash_at + HASH_LEN]; + let mut out = format!("{}\n\n", hex::encode(frame)); + writeln!(out, "{TAG_SELF_DESCRIBED}( / self-described CBOR /").unwrap(); + writeln!(out, " [").unwrap(); + writeln!( + out, + " {BOOKMARK_FORMAT_VERSION} / bookmark format version /" + ) + .unwrap(); + writeln!( + out, + " h'{}' / integrity: BLAKE3 of the embedded record /", + hex::encode(integrity) + ) + .unwrap(); + writeln!( + out, + " {TAG_EMBEDDED_ITEM}(<< / embedded record, {} byte(s) /", + payload.len() + ) + .unwrap(); + writeln!(out, " {{ / {} network(s) /", record.len()).unwrap(); + for (network, clocks) in &record { + writeln!( + out, + " h'{}' / network / =>", + hex::encode(network.to_bytes()) + ) + .unwrap(); + writeln!(out, " [ / {} clock(s) /", clocks.len()).unwrap(); + for clock in clocks { + writeln!( + out, + " {}(h'{}') / clock /", + crate::tags::CLOCK_TAG, + hex::encode(clock.encode()) + ) + .unwrap(); + } + writeln!(out, " ]").unwrap(); + } + writeln!(out, " }}").unwrap(); + writeln!(out, " >>)").unwrap(); + writeln!(out, " ]").unwrap(); + write!(out, ")").unwrap(); + out +} + +/// The encoded empty record pins byte-for-byte. +/// +/// The snapshot's first line is the frame's exact hex — the +/// self-described frame over the embedded CBOR encoding of an empty +/// map — followed by [`annotated`]'s decoded reading of the same bytes. +/// +/// A change to the hex line is a deliberate on-disk format change, like +/// the wire-format snapshots; an annotation-only change must preserve it +/// byte-identically. #[test] fn pins_the_empty_frame() { - insta::assert_snapshot!("frame_empty", hex::encode(encode(&BTreeMap::new()))); + insta::assert_snapshot!("frame_empty", annotated(&encode(&BTreeMap::new()))); } /// The encoded non-trivial record pins byte-for-byte, so format drift cannot /// hide in a populated payload (multiple clocks under a network id) the way it /// could in an empty one. /// +/// The snapshot's first line is the frame's exact hex (the pin); the rest +/// is [`annotated`]'s decoded reading of the same bytes. +/// /// The pinned bytes are fixture-derived: a re-accept whose only cause is a /// deliberate [`sample_record`] change — the format attested unchanged by the /// untouched `frame_empty` pin and the round-trip/corruption suite in the @@ -182,5 +503,18 @@ fn pins_the_empty_frame() { /// snapshot roster in `AGENTS.md`). #[test] fn pins_a_non_trivial_frame() { - insta::assert_snapshot!("frame_non_trivial", hex::encode(encode(&sample_record()))); + insta::assert_snapshot!("frame_non_trivial", annotated(&encode(&sample_record()))); +} + +/// Pins the stated ingress boundary: the embedded payload's spelling is +/// not ingress-judged — the hash binds bytes, the frame binds shape. +/// +/// A payload spelled as an indefinite-length map (a spelling this codec +/// never writes) decodes to the empty record. Flipping this to rejection +/// is a deliberate contract change, not drift. +#[test] +fn indefinite_length_payload_map_is_not_spelling_judged() { + let record = + decode(&frame(&[0xbf, 0xff])).expect("the payload's spelling is not ingress-judged"); + assert!(record.is_empty()); } diff --git a/src/conformance/backend.rs b/src/conformance/backend.rs index 488ddb78c..6ffd23610 100644 --- a/src/conformance/backend.rs +++ b/src/conformance/backend.rs @@ -677,8 +677,8 @@ where { let charged = Charged::new(backend); - // Two concurrent histories from one universe: the left clock mints the - // shared corpus and its own tail; the right fork mints the other tail. + // Two concurrent histories from one universe: the left clock produces the + // shared corpus and its own tail; the right fork produces the other tail. let mut left_clock = before::Clock::seed(); let mut right_clock = left_clock.fork(); @@ -741,7 +741,7 @@ where } /// Assemble one corpus through the charged backend, leaves in path order. -// The inline pair type is clearer than a name minted only to satisfy the +// The inline pair type is clearer than a name coined only to satisfy the // lint. #[allow(clippy::type_complexity)] async fn corpus( diff --git a/src/conformance/link.rs b/src/conformance/link.rs index 2fc459cba..1b36acc52 100644 --- a/src/conformance/link.rs +++ b/src/conformance/link.rs @@ -1043,18 +1043,21 @@ pub async fn check_sessions( .into_rumors(); // Divergence wide and deep enough to exercise many streams per side. - { - let mut batch = seed.batch(); + seed.batch(|batch| { for payload in 0..SESSION_PAYLOADS { - batch.send(payload); - } - } - { - let mut batch = newcomer.batch(); - for payload in SESSION_PAYLOADS..2 * SESSION_PAYLOADS { - batch.send(payload); + batch.send(payload)?; } - } + Ok::<(), crate::message::EncodeError>(()) + }) + .expect("flat payloads are within any depth limit"); + newcomer + .batch(|batch| { + for payload in SESSION_PAYLOADS..2 * SESSION_PAYLOADS { + batch.send(payload)?; + } + Ok::<(), crate::message::EncodeError>(()) + }) + .expect("flat payloads are within any depth limit"); // Session two: reconcile the divergence; session three: converge as a // no-op. Serialized on the same links, so the epoch counting and diff --git a/src/conformance/link/tests.rs b/src/conformance/link/tests.rs index a960fa26a..ce26bb087 100644 --- a/src/conformance/link/tests.rs +++ b/src/conformance/link/tests.rs @@ -668,7 +668,7 @@ struct SharedWindow { /// Handle to one direction's shared connection window. type Window = Arc>; -/// Mint one direction's window with `budget` bytes available. +/// Create one direction's window with `budget` bytes available. fn window(budget: usize) -> Window { Arc::new(Mutex::new(SharedWindow { available: budget, @@ -741,7 +741,7 @@ impl AsyncRead for WindowedRx { } } -/// The windowed connector: every minted stream's writes charge the shared +/// The windowed connector: every opened stream's writes charge the shared /// window. #[derive(Clone)] struct WindowedConnector { @@ -1047,16 +1047,23 @@ fn starved_pool_degrades_latency_not_liveness() { .sync_window_floor() .into_rumors(); { - let mut batch = seed.batch(); - for payload in 0..2048u64 { - batch.send(payload); - } + seed.batch(|batch| { + for payload in 0..2048u64 { + batch.send(payload)?; + } + Ok::<(), crate::message::EncodeError>(()) + }) + .expect("flat test payloads are within any depth limit"); } { - let mut batch = newcomer.batch(); - for payload in 2048..4096u64 { - batch.send(payload); - } + newcomer + .batch(|batch| { + for payload in 2048..4096u64 { + batch.send(payload)?; + } + Ok::<(), crate::message::EncodeError>(()) + }) + .expect("flat test payloads are within any depth limit"); } let (near, far) = futures::future::join(seed.gossip(&mut a), newcomer.gossip(&mut b)).await; near.expect("gossip completes over the starved pool"); diff --git a/src/error.rs b/src/error.rs index 76fade6fe..e4a47bf39 100644 --- a/src/error.rs +++ b/src/error.rs @@ -1,8 +1,9 @@ //! Public failures from transport sessions and durable identity handling. //! -//! You handle [`Error`]; everything else on this page is the diagnostic -//! taxonomy reachable through [`Error::Mirror`], for matching and bug -//! reports. Every session `Err` poisons its link (discard it and +//! You handle [`Error`], and a send can return the local admission +//! error [`EncodeError`] (also at the crate root); everything else on +//! this page is the diagnostic taxonomy reachable through +//! [`Error::Mirror`], for matching and bug reports. Every session `Err` poisons its link (discard it and //! reconnect, [`Error::LinkPoisoned`]), so the table below states what //! each variant means *beyond* that: //! @@ -12,9 +13,14 @@ //! | [`Error::MagicMismatch`] | unchanged | the counterparty is not speaking rumors: fix the dial target | //! | [`Error::VersionMismatch`] | unchanged | select the same [`Protocol`] at both ends; if both already do, the selected protocol's wire version differs across the two releases: align crate versions | //! | [`Error::NetworkMismatch`] | unchanged | unrelated universes: apply the dominance rule ([`Peer`](crate::Peer)'s "Bootstrapping without consensus") | +//! | [`Error::PayloadDepthMismatch`] | unchanged | fix the configuration: the payload depth limit is a fleet-wide parameter ([`Peer::payload_depth_limit`](crate::Peer::payload_depth_limit)); align it and reconnect | //! | [`Error::PartyOverlap`] | unchanged | nothing was absorbed: the retiring peer's identity overlaps ours | //! | [`Error::Epilogue`] | **committed** (a bootstrapping side instead applies nothing) | none locally: what was certainly lost is the peer's confirmation (a donor's identity may be lost with it: see the variant) | //! | [`Error::LinkPoisoned`] | unchanged | handle the first non-poisoned error; repeats mean the reconnect is not producing a fresh link | +//! | [`Error::PreambleMalformed`] | unchanged | counterparty bug: report it (the defect names the field) | +//! | [`Error::PreambleTruncated`] | unchanged | the peer or transport hung up mid-handshake: retry over a fresh link | +//! | [`Error::HandOffMalformed`] | unchanged | counterparty bug: report it (the defect names the fault) | +//! | [`Error::HandOffTruncated`] | unchanged | the peer or transport hung up before delivering its promised identity hand-off: retry over a fresh link | //! | [`Error::IntentInvalid`] | unchanged | counterparty bug: report it | //! | [`Error::BootstrapRetireConflict`] | unchanged | counterparty bug: report it | //! | [`Error::BootstrapHistoryConflict`] | unchanged | counterparty bug: report it | @@ -24,20 +30,23 @@ use std::convert::Infallible; use crate::{ - Network, Protocol, Ticks, + Network, PayloadDepthLimit, Protocol, Ticks, bookmark::{BookmarkError, BookmarkIo, NoBookmark}, tree::mirror::{self, handshake}, }; +pub use crate::message::EncodeError; +pub use crate::tree::mirror::handshake::PreambleDefect; +pub use crate::tree::mirror::party::HandOffDefect; pub use crate::tree::mirror::streaming::materialized::{ Error as MaterializedError, Violation as MaterializedViolation, }; pub use crate::tree::mirror::streaming::remote::{ AcceptError, CodecDecodeError, CodecDecodeErrorKind, CodecEncodeError, CodecEncodeErrorKind, - DecodeError, DecodeLeafError, DecodeSignalError, EncodeError, FramePart, - InvalidSignalPlacement, InvalidWireSignal, LeafRunError, LengthOverflow, OpeningError, Origin, - QueryOrderError, RemoteError, ReplyFrameError, ScopeError, SendError, Speaker, Stream, - StreamClass, StreamError, + DecodeLeafError, DecodeSignalError, FramePart, GreetingError, HeadError, + InvalidSignalPlacement, InvalidWireSignal, LeafRunError, LengthOverflow, ListingIssue, + OpeningError, Origin, QueryOrderError, RemoteError, ReplyDecodeError, ReplyEncodeError, + ReplyFrameError, ScopeError, SendError, Speaker, Stream, StreamClass, StreamError, }; /// The concrete production mirror failure, retaining its detecting side. @@ -57,15 +66,17 @@ pub enum Error { #[error(transparent)] Io(#[from] std::io::Error), - /// The peer's preamble did not begin with [`PROTOCOL_MAGIC`](crate::PROTOCOL_MAGIC). - #[error("peer is not a rumors stream (remote magic: {remote_magic:x?})")] + /// The peer is not speaking the rumors protocol: its preamble began + /// with neither the self-described CBOR opening of a + /// [`Protocol::V2`] session nor the legacy raw magic of a V1 one. + #[error("peer is not a rumors stream (leading bytes: {remote_magic:x?})")] MagicMismatch { remote_magic: [u8; 6] }, /// The peer speaks a different wire dialect. #[error("peer speaks rumors protocol version {remote_version}, we selected {local_protocol:?}")] VersionMismatch { local_protocol: Protocol, - remote_version: u16, + remote_version: u64, }, /// Both peers were gossiping but belong to unrelated causal universes. @@ -91,6 +102,25 @@ pub enum Error { #[error("retiring peer's party overlaps ours")] PartyOverlap, + /// The peer's configured payload depth limit differs from ours. + /// + /// The limit is a property of the shared set — every replica must be + /// able to hold and forward all content — so all peers of a fleet + /// must select the same [`Peer::payload_depth_limit`](crate::Peer::payload_depth_limit). + /// Both sides detect the mismatch symmetrically, after the greetings + /// are exchanged and before anything else (the converged-session + /// short-circuit included), so a mixed configuration is caught + /// deterministically at every pairing rather than mid-session on + /// particular content. Fix the configuration — align the limit + /// fleet-wide — and reconnect. + #[error("peer's payload depth limit ({remote}) differs from ours ({local})")] + PayloadDepthMismatch { + /// This side's configured limit. + local: PayloadDepthLimit, + /// The limit the peer's greeting declared. + remote: PayloadDepthLimit, + }, + /// The session's closing epilogue failed *after* the session's local /// work committed. /// @@ -146,6 +176,61 @@ pub enum Error { )] LinkPoisoned, + /// The peer opened as a rumors stream of the selected dialect, but a + /// field of its preamble is not spelled the way the wire demands. + /// + /// The preamble is deterministic-encoding CBOR — one spelling per + /// field — so this is always a counterparty bug, never an alternate + /// encoding; the defect names the offending field. + #[error("peer preamble is malformed: {defect}")] + PreambleMalformed { + /// Which preamble field failed, and how. + defect: PreambleDefect, + }, + + /// The peer closed the stream partway through its preamble. + /// + /// Distinct from [`Io`](Self::Io): the transport delivered a clean + /// close, not a failure — the counterparty (or something between) + /// hung up mid-handshake. Retry over a fresh link; persistent + /// zero-byte truncations from a live peer are a counterparty bug. + #[error("peer closed after sending {received} of its {expected} preamble bytes")] + PreambleTruncated { + /// Preamble bytes received before the close. + received: usize, + /// The selected dialect's full preamble width. + expected: usize, + }, + + /// The peer delivered its promised identity hand-off, but the item is + /// not spelled the way the wire demands, or its content is not one + /// canonical party encoding. + /// + /// The hand-off — the trailing party donation of a bootstrap or + /// retirement session — is deterministic-encoding CBOR wrapping a + /// canonical party encoding, one spelling per donation, so this is + /// always a counterparty bug, never an alternate encoding; the defect + /// names the fault. Nothing was absorbed: the local replica is + /// unchanged. Reachable only for [`Protocol::V2`]; the frozen V1 + /// dialect reports hand-off failures as [`Io`](Self::Io). + #[error("peer identity hand-off is malformed: {defect}")] + HandOffMalformed { + /// Which part of the hand-off failed, and how. + defect: HandOffDefect, + }, + + /// The peer closed the stream before delivering its promised identity + /// hand-off whole. + /// + /// Distinct from [`Io`](Self::Io): the transport delivered a clean + /// close, not a failure — the counterparty (or something between) + /// hung up after its preamble intent promised a donation. Nothing was + /// absorbed: the local replica is unchanged. Retry over a fresh link. + /// Reachable only for [`Protocol::V2`]; the frozen V1 dialect reports + /// hand-off failures as [`Io`](Self::Io). + #[error("peer closed before delivering its promised identity hand-off")] + HandOffTruncated, + /// The peer's intent byte had no defined meaning. #[error("peer sent an invalid intent byte ({byte:#04x})")] IntentInvalid { byte: u8 }, @@ -221,6 +306,10 @@ impl From for Error { local_protocol, remote_version, }, + handshake::Error::Malformed { defect } => Error::PreambleMalformed { defect }, + handshake::Error::Truncated { received, expected } => { + Error::PreambleTruncated { received, expected } + } handshake::Error::IntentInvalid { byte } => Error::IntentInvalid { byte }, handshake::Error::BootstrapRetireConflict => Error::BootstrapRetireConflict, } @@ -254,8 +343,17 @@ impl Error { local_min_events, }, Error::PartyOverlap => Error::PartyOverlap, + Error::PayloadDepthMismatch { local, remote } => { + Error::PayloadDepthMismatch { local, remote } + } Error::Epilogue(error) => Error::Epilogue(error), Error::LinkPoisoned => Error::LinkPoisoned, + Error::PreambleMalformed { defect } => Error::PreambleMalformed { defect }, + Error::PreambleTruncated { received, expected } => { + Error::PreambleTruncated { received, expected } + } + Error::HandOffMalformed { defect } => Error::HandOffMalformed { defect }, + Error::HandOffTruncated => Error::HandOffTruncated, Error::IntentInvalid { byte } => Error::IntentInvalid { byte }, Error::BootstrapRetireConflict => Error::BootstrapRetireConflict, Error::BootstrapHistoryConflict { claimed_min_events } => { diff --git a/src/lib.rs b/src/lib.rs index ed5d45e39..6540aa8b2 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -144,12 +144,12 @@ //! use rumors::Peer; //! //! #[tokio::main] -//! async fn main() -> Result<(), rumors::Error> { +//! async fn main() -> Result<(), Box> { //! // The universe's first peer creates it; every later peer bootstraps in. //! let alice = Peer::::seed().into_rumors(); //! -//! // A bare `send` statement commits when its `Batch` drops, right here. -//! alice.send("the meeting is at noon".to_string()); +//! // A send commits right here. +//! alice.send("the meeting is at noon".to_string())?; //! //! // A session runs over a `Link`: a control byte stream plus a supply //! // of independent data streams (see the `link` module); here, the @@ -208,6 +208,11 @@ //! to yield ([`Rumors::redact`] explains why none is needed); an application //! that needs deletion events sends them as ordinary messages of its own. //! +//! All of the above observe the *set*. To watch the *wire* instead — every +//! protocol message of a live session, as raw CBOR items, for debuggers, +//! recorders, and tracing adapters — attach a handler from the [`observe`] +//! module ([`Peer::observe`]). +//! //! # Transport: bring a [`Link`] //! //! A session's transport is a [`Link`]: one persistent bidirectional @@ -230,17 +235,41 @@ //! [`AsyncRead`](tokio::io::AsyncRead) and [`AsyncWrite`](tokio::io::AsyncWrite); //! no Tokio runtime, spawning, sockets, or timers are required by this crate. //! -//! # Message payloads and compatibility -//! -//! Your message type `T` needs [`serde::Serialize`] and -//! [`serde::de::DeserializeOwned`]; payloads are serialized as -//! CBOR ([RFC 8949](https://www.rfc-editor.org/rfc/rfc8949)). Because -//! CBOR carries field and variant *names*, reordering `struct` fields -//! or `enum` variants does not break compatibility with prior versions -//! of your type `T`; however, *renaming breaks compabitility*. It is worth -//! designing around this from the get-go: consider an outer `enum` indicating -//! the version of your application-level message type, even if it starts -//! out only having one variant, `V1`. +//! # Choosing a payload type +//! +//! Your message type `T` needs [`serde::Serialize`], +//! [`serde::de::DeserializeOwned`], [`Eq`], [`Send`], [`Sync`], and +//! `'static`, all demanded once, at peer construction. Payloads are +//! serialized as CBOR ([RFC 8949](https://www.rfc-editor.org/rfc/rfc8949)). +//! Each bound guards replication: +//! +//! - **`Serialize` must succeed on every value you send.** CBOR itself +//! imposes no format-driven failures, so a `Serialize` error is a bug +//! in the payload type: sending panics. Avoid types whose `Serialize` +//! is data-dependently fallible (for example `std::path::PathBuf`, +//! which errors on non-UTF-8 paths). +//! - **Every encoding must decode back equal to the value sent.** Each +//! send re-decodes its own encoding with the exact decoder receivers +//! run and compares by `Eq`; a lossy encoding (for example +//! `Some(None)` in a nested `Option`, which decodes as `None`) is the +//! typed [`EncodeError`], rejected at the author rather than silently +//! diverging at every replica. The bound is `Eq` rather than +//! `PartialEq` so the check is never spurious; this excludes +//! `f32`/`f64` fields (NaN compares unequal to itself). +//! - **Nesting depth is bounded.** Decoding a payload may recurse at +//! most [`Peer::payload_depth_limit`] steps (256 by default, ample +//! for ordinary types); an over-deep value is rejected at send. The +//! limit is held to exact equality fleet-wide at every handshake, so +//! an admitted payload is transferable everywhere; the knob's docs +//! carry the full contract. +//! +//! On compatibility across versions of your own type: because CBOR +//! carries field and variant *names*, reordering `struct` fields or +//! `enum` variants does not break compatibility with prior versions of +//! your type `T`; however, *renaming breaks compatibility*. It is worth +//! designing around this from the get-go: consider an outer `enum` +//! indicating the version of your application-level message type, even +//! if it starts out only having one variant, `V1`. //! //! # Cargo features //! @@ -282,11 +311,13 @@ pub mod error; pub mod link; mod message; mod network; +pub mod observe; mod peer; mod protocol; pub mod reconciliation; mod rumors; mod snapshot; +pub mod tags; #[cfg(any(test, feature = "test-internals"))] #[doc(hidden)] pub mod testing; @@ -296,21 +327,24 @@ pub mod tutorial; #[cfg(test)] mod tests; +#[cfg(feature = "protocol-v1")] pub use crate::peer::PROTOCOL_MAGIC; pub use ::before; pub use batch::Batch; pub use before::{Ticks, Version, causally}; pub use bookmark::{ - BOOKMARK_FORMAT_VERSION, BOOKMARK_MAGIC, Bookmark, BookmarkError, BookmarkIo, FormatError, - NoBookmark, Serialized, + BOOKMARK_FORMAT_VERSION, Bookmark, BookmarkError, BookmarkIo, FormatError, FrameDefect, + NoBookmark, RecordDefect, Serialized, }; pub use error::{Error, MirrorError}; pub use link::{Acceptor, Connector, Link}; +pub use message::EncodeError; pub use network::Network; pub(crate) use peer::Inner; pub use peer::{ - BookmarkedBootstrap, Bootstrap, DEFAULT_SYNC_MEMORY_BUDGET, DEFAULT_TARGET_MESSAGE_SIZE, - Gossiped, Joined, Led, Peer, Retire, Unbookmarked, + BookmarkedBootstrap, Bootstrap, DEFAULT_PAYLOAD_DEPTH_LIMIT, DEFAULT_SYNC_MEMORY_BUDGET, + DEFAULT_TARGET_MESSAGE_SIZE, Gossiped, Joined, Led, PayloadDepthLimit, Peer, Retire, + Unbookmarked, }; pub use protocol::Protocol; pub use rumors::{CausalMessages, Changes, Rumors, TryNext, TryTick, UnorderedMessages}; diff --git a/src/link.rs b/src/link.rs index ef2e742c9..098fd27c8 100644 --- a/src/link.rs +++ b/src/link.rs @@ -73,6 +73,10 @@ //! side, so an //! [`Acceptor`] may yield streams in any order and needs no routing logic. //! +//! Reads are exact and item-granular; on an unbuffered transport, wrap the +//! read half in `tokio::io::BufReader` — caller-owned buffering outlives a +//! session and is safe across session boundaries. +//! //! ## Pooled flow control //! //! Some transports do not give every stream its own private buffer. diff --git a/src/link/routed.rs b/src/link/routed.rs index 6cccad040..13288cd65 100644 --- a/src/link/routed.rs +++ b/src/link/routed.rs @@ -14,7 +14,7 @@ //! //! What makes that routable is a small connect header. Every dialed //! connection opens by naming the link it belongs to (a 16-byte random -//! *token* minted at link establishment), and one **router** per +//! *token* drawn at link establishment), and one **router** per //! [`Endpoint`] owns the listener: it reads each arriving connection's //! header and hands the connection — whole, never its bytes — to that //! link's bounded queue, where the link's [`Acceptor`] collects it. A diff --git a/src/link/routed/header.rs b/src/link/routed/header.rs index 3b232559d..330c5d43d 100644 --- a/src/link/routed/header.rs +++ b/src/link/routed/header.rs @@ -84,21 +84,21 @@ const TOKEN_LEN: usize = 16; /// bounded by construction. pub const MAX_ADDR_LEN: usize = u8::MAX as usize; -/// One link's routing identity: 16 random bytes minted at +/// One link's routing identity: 16 random bytes drawn at /// establishment. /// /// Both routers key the link's connection queue by its token, and every /// data-stream dial quotes it. The width makes collision handling a /// non-problem; the token is routing state, never a credential (the /// transport below is authenticated, per the [module docs](super)). -/// Tokens are minted by [`Endpoint::link`](super::Endpoint::link) and +/// Tokens are created by [`Endpoint::link`](super::Endpoint::link) and /// observed through [`LinkInfo`](super::LinkInfo); they cannot be /// constructed. #[derive(Clone, Copy, PartialEq, Eq, Hash)] pub struct Token([u8; TOKEN_LEN]); impl Token { - /// Mint a fresh random token. + /// Draw a fresh random token. pub(super) fn new() -> Self { Token(rand::random()) } diff --git a/src/link/routed/router.rs b/src/link/routed/router.rs index 75864a33e..44a9bd380 100644 --- a/src/link/routed/router.rs +++ b/src/link/routed/router.rs @@ -93,7 +93,7 @@ impl Drop for Registration { } } -/// Register a fresh outbound link: mint an unclaimed token and claim +/// Register a fresh outbound link: draw an unclaimed token and claim /// it. /// /// Collisions are astronomically unlikely at the token's width; the @@ -251,7 +251,7 @@ async fn deliver( let mut entries = entries(table); if entries.contains_key(&token) { // A duplicate establishment is a peer bug (tokens - // are minted fresh per link); dropping it leaves + // are drawn fresh per link); dropping it leaves // the live link undisturbed. return Ok(()); } diff --git a/src/message.rs b/src/message.rs index dec9a7e07..3c3a6e984 100644 --- a/src/message.rs +++ b/src/message.rs @@ -37,14 +37,10 @@ use serde::de::DeserializeOwned; /// /// Every payload value must serialize: methods that serialize /// ([`new`](Self::new), [`from_arc`](Self::from_arc)) panic if the -/// payload's [`serde::Serialize`] implementation reports an error. -/// Encoding runs into an in-memory buffer and CBOR imposes no -/// format-driven failures (any map key, any nesting), so the only trigger -/// is the implementation itself declining a value — which this crate -/// treats as a bug in the payload type, exactly as `Ord`'s totality is -/// trusted. Types whose `Serialize` is data-dependently fallible (for -/// example `std::path::PathBuf`, which errors on non-UTF-8 paths) violate -/// that obligation and must not be used as message types. +/// payload's [`serde::Serialize`] implementation reports an error — +/// always a bug in the payload type, since CBOR itself imposes no +/// format-driven failures. The crate docs' "choosing a payload type" +/// section is the contract of record for this obligation. /// /// The typed read panics on a payload type mismatch; see /// [`arc`](Self::arc). @@ -54,9 +50,220 @@ pub struct Message { serialized: Bytes, } +/// The default payload nesting-depth limit: 256 decode recursion steps. +/// +/// Exactly the CBOR decoder's own default recursion bound, so a fleet +/// upgrading together sees no acceptance change on existing content. +/// Wire interop across releases is governed by the greeting's format, +/// not by this constant. +pub const DEFAULT_PAYLOAD_DEPTH_LIMIT: PayloadDepthLimit = PayloadDepthLimit(256); + +/// A peer's payload nesting-depth limit, counted in the CBOR decode +/// engine's recursion steps. +/// +/// Selected by [`Peer::payload_depth_limit`](crate::Peer::payload_depth_limit) +/// (whose docs carry the full contract); defaults to +/// [`DEFAULT_PAYLOAD_DEPTH_LIMIT`]. A step is the engine's own +/// accounting (arrays, maps, tags, and type-driven wrappers such as an +/// enum's variant scope), not a structural property of the bytes — +/// which is why admission at send runs the decode itself rather than +/// counting anything. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub struct PayloadDepthLimit(u64); + +impl PayloadDepthLimit { + /// A limit of exactly `steps` decode recursion steps: a payload + /// value whose decode recurses deeper is rejected. + pub const fn new(steps: u64) -> Self { + PayloadDepthLimit(steps) + } + + /// The limit, in decode recursion steps. + pub const fn get(self) -> u64 { + self.0 + } + + /// The limit as the decoder's `usize` recursion bound. + /// + /// Saturating: a limit past `usize::MAX` admits every input that can + /// physically exist, because a value's nesting depth never exceeds its + /// encoding's byte length, which a slice caps well below `usize::MAX`. + pub(crate) fn recursion_limit(self) -> usize { + usize::try_from(self.0).unwrap_or(usize::MAX) + } +} + +impl Default for PayloadDepthLimit { + fn default() -> Self { + DEFAULT_PAYLOAD_DEPTH_LIMIT + } +} + +impl fmt::Display for PayloadDepthLimit { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + write!(f, "{} steps", self.0) + } +} + +/// A message payload failed admission at its author: the send is refused +/// before anything is stored or gossiped. +/// +/// Admission runs the exact decode every receiver's wire ingress runs, +/// so a payload this error rejects is one a receiver would have failed +/// to decode — surfaced at the author instead. A [`serde::Serialize`] +/// failure is never this error: it keeps the panic contract documented +/// at [`Rumors::send`](crate::Rumors::send). +#[derive(Debug, thiserror::Error)] +pub enum EncodeError { + /// The payload value's CBOR encoding nests deeper than the peer's + /// configured [`PayloadDepthLimit`]. + #[error("message payload nests deeper than the configured payload depth limit ({limit})")] + Depth { + /// The configured limit the payload's decode exceeded. + limit: PayloadDepthLimit, + }, + /// The payload type's [`serde::Deserialize`] implementation rejected + /// the bytes its own [`serde::Serialize`] implementation produced: + /// admitted, such a value would fail at every receiver instead. + #[error("message payload does not survive its own serde round-trip: {0}")] + Roundtrip(#[source] io::Error), + /// The payload value's encoding decodes to a different value (by + /// the payload type's own `Eq`): the serde pairing is lossy for + /// this value. + /// + /// The canonical example is a nested `Option` holding `Some(None)`, + /// which decodes as `None`. The check is send-side only: ingress + /// holds no original to compare against. + #[error("message payload's encoding decodes to a different value")] + Unfaithful, +} + +/// Why a payload decode failed: the crate-internal split between the +/// depth case and everything else. +/// +/// The depth case stays typed end to end so send-side admission can +/// surface it as [`EncodeError::Depth`] without string matching; wire +/// ingress folds both cases back into the `io::Error` its surface +/// speaks ([`Message::from_wire`]). The decode-side counterpart of +/// [`EncodeError`]. +#[derive(Debug)] +pub(crate) enum PayloadDecodeError { + /// The payload's decode recursed past the given limit (the decode + /// engine's recursion-limit error, preserved as a variant). + Depth(PayloadDepthLimit), + /// Truncation (the reader's own error, passed through) or invalid + /// data (corruption, a type mismatch, trailing bytes). + Io(io::Error), +} + +impl PayloadDecodeError { + /// Fold into `io::Error`, the wire-ingress surface: the depth case + /// becomes invalid data naming the exceeded limit. + fn into_io(self) -> io::Error { + match self { + PayloadDecodeError::Depth(limit) => io::Error::new( + io::ErrorKind::InvalidData, + format!("message payload nests deeper than the payload depth limit ({limit})"), + ), + PayloadDecodeError::Io(error) => error, + } + } +} + +/// Serializes one type-erased payload value into an admission-checked +/// [`Message`]; the serializing half of a [`PayloadCodec`]. +pub(crate) type PayloadSerializer = + fn(Arc, PayloadDepthLimit) -> Result; + /// Deserializes one exact CBOR payload encoding into a type-erased payload -/// value; see [`Message::deserializer`]. -pub(crate) type PayloadDeserializer = fn(&[u8]) -> io::Result>; +/// value, bounding the decode's recursion at the given depth limit; the +/// deserializing half of a [`PayloadCodec`]. +pub(crate) type PayloadDeserializer = + fn(&[u8], PayloadDepthLimit) -> Result, PayloadDecodeError>; + +/// A peer's payload codec: the typed payload boundary created once at +/// [`Peer`](crate::Peer) construction and carried by every session. +/// +/// The payload type's serde obligations concentrate at construction; the fn +/// pointer inside is the type's only residue afterwards, so everything +/// that carries a codec stays non-generic. The configured +/// [`PayloadDepthLimit`] rides beside the pointer as data (a plain fn +/// pointer cannot capture it), which is what makes the limit unmissable: +/// every ingress parse in the peer's orbit goes through this one value. +#[derive(Clone, Copy)] +pub(crate) struct PayloadCodec { + serialize: PayloadSerializer, + deserialize: PayloadDeserializer, + limit: PayloadDepthLimit, +} + +impl PayloadCodec { + /// Construct the codec for payloads of type `T` at the given depth limit. + /// + /// All of `T`'s payload obligations land here, at construction: a peer + /// demands `Serialize` at construction even if it never sends, + /// symmetric with demanding `DeserializeOwned` even if it never + /// receives (forwarding needs neither bound, since gossip re-supplies + /// cached bytes), and `Eq` so send-side admission can hold every + /// encoding to decode back equal to the value sent. + pub(crate) fn new(limit: PayloadDepthLimit) -> Self + where + T: Serialize + DeserializeOwned + Eq + Send + Sync + 'static, + { + fn serialize_payload( + payload: Arc, + limit: PayloadDepthLimit, + ) -> Result { + let payload: Arc = payload.downcast().unwrap_or_else(|_| { + panic!("a codec serializes exactly the payload type it was built for") + }); + Message::try_from_arc(payload, limit) + } + PayloadCodec { + serialize: serialize_payload::, + deserialize: Message::deserializer::(), + limit, + } + } + + /// Serialize one payload value of the codec's payload type into an + /// admission-checked [`Message`] at the carried limit + /// ([`Message::try_new`]'s contract). + /// + /// # Panics + /// + /// If the value is not the payload type the codec was built for: a crate bug, + /// never an input — every caller hands in the `T` its own typed + /// signature names. A `Serialize` failure keeps [`Message`]'s + /// documented panic contract. + pub(crate) fn message( + &self, + payload: Arc, + ) -> Result { + (self.serialize)(payload, self.limit) + } + + /// The configured payload depth limit this codec enforces: what the + /// greeting declares and the handshake holds to equality. + pub(crate) fn limit(&self) -> PayloadDepthLimit { + self.limit + } + + /// Replace the carried depth limit, keeping the codec's fn pointers. + #[must_use] + pub(crate) fn with_limit(self, limit: PayloadDepthLimit) -> Self { + PayloadCodec { limit, ..self } + } + + /// Decode one exact CBOR payload encoding into a type-erased payload + /// value, bounded at the carried depth limit. + pub(crate) fn decode( + &self, + bytes: &[u8], + ) -> Result, PayloadDecodeError> { + (self.deserialize)(bytes, self.limit) + } +} /// Map a ciborium deserialization failure into `io::Error`, keeping the /// truncation/corruption split callers classify by: a reader's own error @@ -82,9 +289,46 @@ fn to_vec(value: &T) -> Vec { buf } +/// Decode exactly one CBOR value of type `T` from `bytes`, bounding the +/// decode's recursion at `limit`: the one payload parse behind every +/// typed constructor and the codec's deserializer. +/// +/// Trailing bytes are rejected as invalid data, so a cache built from the +/// input is always the value's exact encoding. A value whose decode +/// recurses past the limit is the typed depth case, kept distinct so +/// send-side admission can classify it without string matching. +fn decode_exact( + bytes: &[u8], + limit: PayloadDepthLimit, +) -> Result { + let mut input = bytes; + let message: T = + ciborium::de::from_reader_with_recursion_limit(&mut input, limit.recursion_limit()) + .map_err(|error| match error { + ciborium::de::Error::RecursionLimitExceeded => PayloadDecodeError::Depth(limit), + error => PayloadDecodeError::Io(de_error(error)), + })?; + if !input.is_empty() { + return Err(PayloadDecodeError::Io(io::Error::new( + io::ErrorKind::InvalidData, + format!("{} trailing bytes after the message payload", input.len()), + ))); + } + Ok(message) +} + impl Message { /// Creates a `Message` pairing the given object with its cached - /// serialization. + /// serialization, with no admission check. + /// + /// No unchecked constructor can reach a peer's set: insertion happens + /// only through [`Rumors::send`](crate::Rumors::send) and + /// [`Batch::send`](crate::Batch::send), which create admission-checked + /// messages through the peer's codec ([`try_new`](Self::try_new)), + /// and through wire ingress, which runs the same decode admission + /// runs. `new` and [`from_arc`](Self::from_arc) construct + /// free-standing messages (trees built outside any peer, fixtures, + /// size probes). /// /// # Panics /// @@ -99,89 +343,138 @@ impl Message { } } + /// Creates an admission-checked `Message`: the constructor behind + /// [`Rumors::send`](crate::Rumors::send) and + /// [`Batch::send`](crate::Batch::send). + /// + /// Serializes `message` and admits it only if the exact decode + /// every receiver's wire ingress runs for `T` reads the encoding + /// back within `limit`, to a value equal (by `T`'s own `Eq`) to the + /// one sent. Because admission is the receiving computation itself, + /// there is no second accounting to drift: a payload a receiver + /// would reject or misread fails here instead, at its author, as + /// the typed [`EncodeError`] (its variants name the causes). A + /// [`Serialize`] failure keeps [`Message`]'s documented panic + /// contract. + pub fn try_new(message: T, limit: PayloadDepthLimit) -> Result + where + T: Serialize + DeserializeOwned + Eq + Send + Sync + 'static, + { + Self::try_from_arc(Arc::new(message), limit) + } + + /// [`try_new`](Self::try_new) from an existing [`Arc`], without + /// copying: the same allocation, unsized in place. + pub(crate) fn try_from_arc( + arc: Arc, + limit: PayloadDepthLimit, + ) -> Result + where + T: Serialize + DeserializeOwned + Eq + Send + Sync + 'static, + { + let serialized = to_vec(&*arc); + // Admission is the receiver's computation: the payload deserializer + // — the same fn every receiver's wire ingress runs for this + // payload type — reads the just-serialized bytes back at the same + // limit. + let decoded = match Self::deserializer::()(&serialized, limit) { + Ok(decoded) => decoded, + Err(PayloadDecodeError::Depth(limit)) => return Err(EncodeError::Depth { limit }), + Err(PayloadDecodeError::Io(source)) => return Err(EncodeError::Roundtrip(source)), + }; + // Faithfulness: what a receiver reads must be the value that was + // sent, judged by the payload type's own equality. + let decoded: Arc = decoded + .downcast() + .unwrap_or_else(|_| panic!("a payload decodes to its own type")); + if *decoded != *arc { + return Err(EncodeError::Unfaithful); + } + Ok(Message { + serialized: Bytes::from(serialized), + message: arc, + }) + } + /// Creates a `Message` pairing the given serialized bytes with the - /// object derived by deserializing them as a `T`. + /// object derived by deserializing them as a `T`, bounding the + /// decode's recursion at `limit`. /// - /// The bytes must be exactly one CBOR value: trailing bytes are - /// rejected as invalid data, so the cache is always the value's exact - /// encoding. - pub fn from_slice(bytes: &[u8]) -> io::Result + /// Crate-internal rehydration over bytes that arrive outside any + /// peer's orbit (fixtures, capture tooling), so the limit is an + /// explicit parameter rather than a codec's: a caller rehydrating + /// bytes written under a raised limit passes that limit. The bytes + /// must be exactly one CBOR value: trailing bytes are rejected as + /// invalid data, so the cache is always the value's exact encoding, + /// and a value whose decode recurses past the limit is invalid data + /// too. + pub fn from_slice(bytes: &[u8], limit: PayloadDepthLimit) -> io::Result where T: DeserializeOwned + Send + Sync + 'static, { - let mut input = bytes; - let message: T = ciborium::de::from_reader(&mut input).map_err(de_error)?; - if !input.is_empty() { - return Err(io::Error::new( - io::ErrorKind::InvalidData, - format!("{} trailing bytes after the message payload", input.len()), - )); - } + let message: T = decode_exact(bytes, limit).map_err(PayloadDecodeError::into_io)?; Ok(Message { message: Arc::new(message), serialized: Bytes::copy_from_slice(bytes), }) } - /// Decodes wire payload bytes into a `Message` through a - /// [`PayloadDeserializer`]: the one deserialization every gossip - /// ingress performs, with the deserializer carrying the payload type - /// the peer was constructed with. + /// Decodes wire payload bytes into a `Message` through the peer's + /// [`PayloadCodec`]: the one deserialization every gossip ingress + /// performs. /// - /// The deserializer validates the bytes are exactly one CBOR value of - /// its type ([`from_slice`](Self::from_slice)'s contract), so the - /// cache is always the payload's exact encoding and a malformed - /// payload fails here, at the wire boundary. - pub(crate) fn from_wire(bytes: Bytes, deserializer: PayloadDeserializer) -> io::Result { + /// The codec carries the payload type the peer was constructed with + /// and the depth limit it was configured with. + /// + /// The codec validates the bytes are exactly one CBOR value of its + /// type, decoded within its limit, so the cache is always the + /// payload's exact encoding and a malformed or over-deep payload + /// fails here, at the wire boundary, as invalid data. + pub(crate) fn from_wire(bytes: Bytes, codec: PayloadCodec) -> io::Result { Ok(Message { - message: deserializer(&bytes)?, + message: codec.decode(&bytes).map_err(PayloadDecodeError::into_io)?, serialized: bytes, }) } - /// The deserializer for payloads of type `T`: what a - /// [`Peer`](crate::Peer) mints at construction and threads to every - /// session's wire ingress ([`from_wire`](Self::from_wire)). + /// The deserializer for payloads of type `T`: the deserializing half + /// of the [`PayloadCodec`] a [`Peer`](crate::Peer) builds at + /// construction, applied at every session's wire ingress + /// ([`from_wire`](Self::from_wire)). /// /// A plain function pointer, so everything that carries it stays /// non-generic: the payload type's only residue in a running session. + /// The depth limit arrives as an argument because a fn pointer cannot + /// capture one; the codec pairs the two. Send-side admission + /// ([`try_new`](Self::try_new)) runs this same fn over its own + /// output, which is what makes admission and ingress one computation. pub(crate) fn deserializer() -> PayloadDeserializer where T: DeserializeOwned + Send + Sync + 'static, { fn deserialize( bytes: &[u8], - ) -> io::Result> { - let mut input = bytes; - let message: T = ciborium::de::from_reader(&mut input).map_err(de_error)?; - if !input.is_empty() { - return Err(io::Error::new( - io::ErrorKind::InvalidData, - format!("{} trailing bytes after the message payload", input.len()), - )); - } + limit: PayloadDepthLimit, + ) -> Result, PayloadDecodeError> { + let message: T = decode_exact(bytes, limit)?; Ok(Arc::new(message)) } deserialize:: } /// Creates a `Message` from already-shared serialized bytes, without - /// copying. + /// copying, bounding the decode's recursion at `limit`. /// /// The bytes are deserialized as a `T` to produce the paired object, - /// under [`from_slice`](Self::from_slice)'s exactly-one-value contract. - pub fn from_bytes(bytes: Bytes) -> io::Result + /// under [`from_slice`](Self::from_slice)'s exactly-one-value, + /// within-limit contract (its docs state why the caller supplies the + /// limit). + pub fn from_bytes(bytes: Bytes, limit: PayloadDepthLimit) -> io::Result where T: DeserializeOwned + Send + Sync + 'static, { - let mut input = bytes.as_ref(); - let message: T = ciborium::de::from_reader(&mut input).map_err(de_error)?; - if !input.is_empty() { - return Err(io::Error::new( - io::ErrorKind::InvalidData, - format!("{} trailing bytes after the message payload", input.len()), - )); - } + let message: T = + decode_exact(bytes.as_ref(), limit).map_err(PayloadDecodeError::into_io)?; Ok(Message { message: Arc::new(message), serialized: bytes, @@ -191,6 +484,9 @@ impl Message { /// Creates a `Message` from an existing [`Arc`], without copying: the /// same allocation, unsized in place. /// + /// Like [`new`](Self::new), no depth admission: `new`'s docs state + /// why the unlimited constructors cannot reach a peer's set. + /// /// # Panics /// /// If the message cannot be serialized (see [`Message`]). @@ -208,17 +504,19 @@ impl Message { /// /// The shape is one CBOR byte string wrapping the payload's own CBOR /// encoding (the same shape [`Serialize`] writes), decoded through - /// the peer's payload deserializer. Trailing data after the byte - /// string survives for the next field: the property the wire codec's + /// the peer's payload codec. The outer parse is a flat byte string, + /// so it runs at the decoder's default recursion bound; the codec + /// bounds the payload inside. Trailing data after the byte string + /// survives for the next field: the property the wire codec's /// mid-stream decodes rest on. Gated to the alternating protocol's /// codec, its only production consumer. #[cfg(any(test, feature = "protocol-v1"))] - pub(crate) fn from_reader(reader: R, deserializer: PayloadDeserializer) -> io::Result + pub(crate) fn from_reader(reader: R, codec: PayloadCodec) -> io::Result where R: io::Read, { let bytes: Vec = ciborium::de::from_reader(reader).map_err(de_error)?; - Self::from_wire(Bytes::from(bytes), deserializer) + Self::from_wire(Bytes::from(bytes), codec) } /// Clones out an owned handle to the payload: a reference bump on the diff --git a/src/message/tests.rs b/src/message/tests.rs index c857eb242..aed00b2bb 100644 --- a/src/message/tests.rs +++ b/src/message/tests.rs @@ -1,3 +1,4 @@ +use crate::message::{PayloadCodec, PayloadDepthLimit}; use std::collections::hash_map::DefaultHasher; use std::hash::{Hash, Hasher}; @@ -55,7 +56,7 @@ proptest! { #[test] fn from_slice_roundtrips(p in payload()) { let bytes = cbor_vec(&p); - let m = Message::from_slice::(&bytes).unwrap(); + let m = Message::from_slice::(&bytes, PayloadDepthLimit::default()).unwrap(); prop_assert_eq!(&*m.arc::(), &p); prop_assert_eq!(m.bytes(), bytes.as_slice()); } @@ -65,8 +66,8 @@ proptest! { #[test] fn from_bytes_matches_from_slice(p in payload()) { let bytes = cbor_vec(&p); - let a = Message::from_slice::(&bytes).unwrap(); - let b = Message::from_bytes::(Bytes::from(bytes.clone())).unwrap(); + let a = Message::from_slice::(&bytes, PayloadDepthLimit::default()).unwrap(); + let b = Message::from_bytes::(Bytes::from(bytes.clone()), PayloadDepthLimit::default()).unwrap(); prop_assert_eq!(&a, &b); prop_assert_eq!(a.bytes(), b.bytes()); } @@ -77,8 +78,8 @@ proptest! { fn trailing_bytes_are_rejected(p in payload(), trailer in proptest::collection::vec(any::(), 1..8)) { let mut bytes = cbor_vec(&p); bytes.extend_from_slice(&trailer); - prop_assert!(Message::from_slice::(&bytes).is_err()); - prop_assert!(Message::from_bytes::(Bytes::from(bytes)).is_err()); + prop_assert!(Message::from_slice::(&bytes, PayloadDepthLimit::default()).is_err()); + prop_assert!(Message::from_bytes::(Bytes::from(bytes), PayloadDepthLimit::default()).is_err()); } /// The serde form of a `Message` is one CBOR byte string wrapping @@ -104,7 +105,7 @@ proptest! { fn serde_roundtrip(p in payload()) { let m = Message::new(p); let bytes = cbor_vec(&m); - let back = Message::from_reader(bytes.as_slice(), Message::deserializer::()).unwrap(); + let back = Message::from_reader(bytes.as_slice(), PayloadCodec::new::(PayloadDepthLimit::default())).unwrap(); prop_assert_eq!(&m, &back); prop_assert_eq!(m.bytes(), back.bytes()); } @@ -120,7 +121,7 @@ proptest! { combined.extend_from_slice(&trailer); let mut slice: &[u8] = &combined; - let back = Message::from_reader(&mut slice, Message::deserializer::()).unwrap(); + let back = Message::from_reader(&mut slice, PayloadCodec::new::(PayloadDepthLimit::default())).unwrap(); prop_assert_eq!(back.bytes(), m.bytes()); prop_assert_eq!(slice, trailer.as_slice()); prop_assert_eq!(combined.len() - slice.len(), expected.len()); @@ -154,3 +155,189 @@ fn mismatched_downcast_panics() { let m = Message::new(0u64); let _ = m.arc::(); } + +/// Nested-array CBOR bytes at exactly `depth` scopes: `depth` array heads +/// around one integer, the minimal encoding whose nesting depth is chosen +/// freely by the test. +fn nested_arrays(depth: usize) -> Vec { + let mut bytes = vec![0x81; depth]; + bytes.push(0x00); + bytes +} + +/// The rehydration constructors take the limit explicitly, so an +/// application on a raised fleet limit can rehydrate its own stored +/// deep messages. +/// +/// A payload past the default depth fails `from_slice` and `from_bytes` +/// at the default limit (as invalid data) and succeeds at a raised one: +/// both directions, so the parameter is proven live in each. +#[test] +fn rehydration_honors_the_explicit_limit() { + let default = PayloadDepthLimit::default(); + let deep = nested_arrays((default.get() + 1) as usize); + + let rejected = Message::from_slice::(&deep, default); + assert_eq!( + rejected.unwrap_err().kind(), + std::io::ErrorKind::InvalidData, + "the default limit must reject a payload one scope past it" + ); + let rejected = Message::from_bytes::(Bytes::from(deep.clone()), default); + assert_eq!( + rejected.unwrap_err().kind(), + std::io::ErrorKind::InvalidData + ); + + let raised = PayloadDepthLimit::new(default.get() + 1); + let m = Message::from_slice::(&deep, raised) + .expect("a raised limit must rehydrate the deep message"); + assert_eq!(m.as_slice(), deep.as_slice()); + let m = Message::from_bytes::(Bytes::from(deep.clone()), raised) + .expect("a raised limit must rehydrate the deep message"); + assert_eq!(m.as_slice(), deep.as_slice()); +} + +/// Pure CBOR array nesting from a type satisfying the payload contract: +/// each layer serializes as a one-element array, the innermost empty. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +struct Arr(Vec); + +/// A value of exactly `depth` nested array scopes (`depth` >= 1). +fn nested_arr(depth: u64) -> Arr { + (1..depth).fold(Arr(vec![]), |a, _| Arr(vec![a])) +} + +/// The admission boundary is exact and typed. +/// +/// A value whose decode needs exactly the configured limit constructs, +/// one more recursion step is `EncodeError::Depth` carrying the +/// configured limit, and wire-style rehydration draws the same line — +/// admission and ingress are the same decode, so the two verdicts +/// cannot differ. +#[test] +fn try_new_admits_exactly_the_limit() { + let limit = super::PayloadDepthLimit::new(8); + let at = nested_arr(8); + let m = Message::try_new(at.clone(), limit).expect("at the limit is admitted"); + assert_eq!(m.bytes(), cbor_vec(&at).as_slice()); + + let over = nested_arr(9); + let error = Message::try_new(over.clone(), limit).unwrap_err(); + assert!( + matches!(error, super::EncodeError::Depth { limit: l } if l == limit), + "one step past the limit is the typed depth case: {error:?}" + ); + + // Rehydration rejects the same bytes admission rejects. + assert!( + Message::from_slice::(&cbor_vec(&over), limit).is_err(), + "the decoder rejects what admission rejects" + ); + let raised = super::PayloadDepthLimit::new(9); + Message::try_new(over, raised).expect("one more step of limit admits it"); +} + +/// A recursive enum whose spine is `serde`'s newtype-variant shape. +/// +/// Each `N` wrapper is one map scope on the wire, and decoding it as +/// `E` prices the innermost unit variant one further recursion step — +/// the type-dependent accounting only the type's own decode can price. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +enum E { + A, + N(Box), +} + +/// `E::A` under `wrappers` layers of `E::N`. +fn nested_enum(wrappers: u64) -> E { + (0..wrappers).fold(E::A, |e, _| E::N(Box::new(e))) +} + +/// The admission boundary is exact for an enum payload. +/// +/// The enum's decode recursion is type-dependent (the innermost unit +/// variant costs a step no structural count of the bytes would find): +/// the deepest value whose decode fits the limit is admitted, one more +/// wrapper is `EncodeError::Depth` — at the author, never at a +/// receiver. +#[test] +fn try_new_prices_an_enums_own_decode() { + let limit = super::PayloadDepthLimit::new(8); + // 7 map scopes + the unit-variant step = 8: exactly the limit. + let at = nested_enum(limit.get() - 1); + let m = Message::try_new(at, limit).expect("a decode at exactly the limit is admitted"); + assert_eq!( + &*Message::from_slice::(m.as_slice(), limit) + .expect("the admitted encoding decodes at an equally-configured receiver") + .arc::(), + &nested_enum(limit.get() - 1), + ); + + // 8 map scopes + the unit-variant step = 9: one past the limit. + let error = Message::try_new(nested_enum(limit.get()), limit).unwrap_err(); + assert!( + matches!(error, super::EncodeError::Depth { limit: l } if l == limit), + "a decode needing limit + 1 is the typed depth case: {error:?}" + ); +} + +/// A payload type violating the round-trip obligation: it serializes as +/// an integer but deserializes expecting text. +#[derive(Debug, PartialEq, Eq)] +struct Lopsided; + +impl Serialize for Lopsided { + fn serialize(&self, serializer: S) -> Result { + serializer.serialize_u64(0) + } +} + +impl<'de> serde::Deserialize<'de> for Lopsided { + fn deserialize>(deserializer: D) -> Result { + String::deserialize(deserializer).map(|_| Lopsided) + } +} + +/// A payload type whose `Deserialize` rejects its own `Serialize` output +/// is the typed `EncodeError::Roundtrip` at the author — the value would +/// have failed at every receiver, and admission is that decode. +#[test] +fn a_type_that_cannot_read_its_own_output_fails_admission() { + let error = Message::try_new(Lopsided, super::PayloadDepthLimit::default()).unwrap_err(); + assert!( + matches!(error, super::EncodeError::Roundtrip(_)), + "the round-trip violation is its own typed case: {error:?}" + ); +} + +/// The constructed codec's serializing half applies the carried limit and +/// reuses the caller's allocation. +/// +/// The codec is `Message::try_new` with the peer's configured limit +/// riding along; the spine here is the enum's map scopes, so the codec +/// path is exercised on the type-dependent accounting. +#[test] +fn codec_serializes_through_the_carried_limit() { + use std::sync::Arc; + let limit = super::PayloadDepthLimit::new(4); + let codec = super::PayloadCodec::new::(limit); + + // 4 map scopes + the unit-variant step: one past the limit. + let deep = nested_enum(4); + let error = codec.message(Arc::new(deep)).unwrap_err(); + assert!( + matches!(error, super::EncodeError::Depth { limit: l } if l == limit), + "the codec surfaces the carried limit: {error:?}" + ); + + // 3 map scopes + the unit-variant step: exactly the limit. + let shallow = nested_enum(3); + let stored: Arc = Arc::new(shallow.clone()); + let m = codec.message(stored.clone()).expect("within the limit"); + assert_eq!(m.bytes(), cbor_vec(&shallow).as_slice()); + assert!( + std::sync::Arc::ptr_eq(&stored, &m.arc::()), + "the codec stores the caller's own allocation" + ); +} diff --git a/src/observe.rs b/src/observe.rs new file mode 100644 index 000000000..565679d55 --- /dev/null +++ b/src/observe.rs @@ -0,0 +1,394 @@ +//! Bytes-level observation of live wire sessions. +//! +//! The hook a debugger, session recorder, or tracing adapter attaches +//! to a [`Peer`](crate::Peer) ([`Peer::observe`](crate::Peer::observe)) +//! or a [`Bootstrap`](crate::Bootstrap) builder +//! ([`Bootstrap::observe`](crate::Bootstrap::observe)) to watch every +//! protocol message the peer exchanges, as raw wire bytes. The hook is +//! rumors-blind: no protocol type appears in its signature, each +//! invocation carries exactly one whole CBOR item with its stream +//! identity, and a consumer parses with any CBOR library — or none. +//! +//! Attachment has three levels, one handler per level, each supplied +//! by the level above; every level can return `None` to skip what it +//! does not care about: +//! +//! - **Peer**: an [`Observer`] attaches once, follows the peer through +//! cloning, bookmarking, and reunion, and is asked for a session +//! handler for every session the peer enters — gossip, bootstrap, +//! and retire alike. +//! - **Session**: a [`SessionObserver`] lives exactly as long as its +//! session and is asked for a stream handler for each directed +//! stream as it opens. +//! - **Stream**: a [`StreamObserver`] receives that one directed +//! stream's messages, in stream order, one CBOR item per +//! [`message`](StreamObserver::message) call. +//! +//! The contract: +//! +//! - **Ordering**: within one directed stream, invocations arrive in +//! the stream's byte order; across streams there is no ordering at +//! all (a session's streams pump concurrently). To recover the +//! observed interleaving, stamp each message from a session-scoped +//! atomic counter shared by the stream handlers. +//! - **Never block**: handlers run synchronously inside the session's +//! own stream tasks. Blocking in +//! [`message`](StreamObserver::message) stalls that directed stream, +//! and waiting on protocol progress deadlocks; hand bytes off to a +//! channel if the consumer is slow. +//! - **Coverage**: every directed stream of a `Protocol::V2` session, +//! both directions, control and data streams alike. The stream-open +//! label is stream *addressing*, not an item, and is not delivered. +//! Only complete items are observed: a session that dies mid-frame +//! does not deliver the fragment, and an aborted session may have +//! observed fewer items than crossed the wire. `Protocol::V1` +//! sessions are not observed: the frozen legacy wire is not a CBOR +//! sequence, so it cannot honor the one-item contract. +//! - **Cost**: unattached (or a level declined), one branch per frame; +//! attached, one extra contiguous copy of each observed frame. The +//! wire bytes themselves are unchanged either way. +//! +//! The hook watches the **wire**, synchronously, from inside the +//! session's tasks; the content observers +//! ([`UnorderedMessages`](crate::UnorderedMessages), +//! [`CausalMessages`](crate::CausalMessages), and +//! [`Changes`](crate::Changes)) watch the **set**, asynchronously, +//! from outside. + +use std::sync::{Arc, Mutex, PoisonError}; + +use crate::Protocol; + +/// A peer-level observation handler: attaches once, yields one +/// [`SessionObserver`] per session the peer enters. +/// +/// Attach with [`Peer::observe`](crate::Peer::observe) or +/// [`Bootstrap::observe`](crate::Bootstrap::observe). The handler is +/// shared by every clone of the peer's [`Rumors`](crate::Rumors) +/// handle, and sessions run concurrently, so it is asked for session +/// handlers from concurrent tasks. +pub trait Observer: Send + Sync { + /// Begin observing one session, or return `None` to skip it. + /// + /// Called once per session, before the session's first byte + /// crosses the wire — for sessions of an observable dialect; see + /// the module docs' `Protocol::V1` exclusion. `session` identifies + /// it; the returned handler's lifetime is the session's. + fn session(&self, session: &SessionInfo) -> Option>; +} + +/// A session-level observation handler: yields one [`StreamObserver`] +/// per directed stream, as each opens. +/// +/// A session's streams open and pump concurrently, so +/// [`stream`](Self::stream) is called from concurrent tasks. +pub trait SessionObserver: Send + Sync { + /// Learn which role this side won in the session's role election. + /// + /// Called at most once, when the election is decided — after the + /// greetings are exchanged and before any data stream opens. Not + /// called when the greetings carried equal versions (no election + /// happens: the session ends over the control stream alone). The + /// role is not part of [`SessionInfo`] because it does not exist + /// yet when the session begins; a consumer that needs it before + /// data frames arrive records it here. The default does nothing. + fn elected(&self, role: Role) { + let _ = role; + } + + /// Begin observing one directed stream, or return `None` to skip + /// it. + /// + /// Called once per directed stream, when it opens: for the control + /// stream's two directions at session start, and for each data + /// stream when this side first writes (sent) or first reads + /// (received) it. A data stream the session never speaks yields no + /// handler. The returned handler's lifetime is the stream's. + fn stream(&self, stream: &StreamInfo) -> Option>; +} + +/// A stream-level observation handler: receives one directed stream's +/// messages, in stream order. +pub trait StreamObserver: Send { + /// Observe one protocol message: exactly one CBOR item of the + /// wire, as sent or received on this handler's directed stream. + /// + /// Invoked synchronously from the stream's own task, after the + /// item was written and flushed (sent) or completely read and + /// accepted (received). Blocking here stalls this directed stream; + /// see the module docs' back-pressure contract. + fn message(&mut self, bytes: &[u8]); +} + +/// What identifies one observed session. +/// +/// Deliberately carries no session number: numbering is the observer's +/// own concern, exactly like message interleaving (see the module +/// docs' ordering section). An [`Observer`] that wants "the peer's Nth +/// observed session" counts inside its own +/// [`session`](Observer::session) — the method is `&self`, so it +/// synchronizes internally (an `AtomicU64` suffices), and the count +/// means precisely what that observer defines it to mean. +#[non_exhaustive] +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct SessionInfo { + /// Which lifecycle operation entered the session. + pub kind: SessionKind, + /// The wire dialect the session speaks. + pub protocol: Protocol, +} + +/// The lifecycle operation that entered an observed session, on this +/// side. +/// +/// The counterparty's role in the same session may differ: a peer +/// serving a bootstrap or absorbing a retirement observes an ordinary +/// [`Gossip`](Self::Gossip) session, and learns what the remote wants +/// from the remote's preamble — which its control-stream handler sees +/// as bytes. +#[non_exhaustive] +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum SessionKind { + /// This side is joining the universe ([`Bootstrap::join`](crate::Bootstrap::join)). + Bootstrap, + /// This side is gossiping ([`Rumors::gossip`](crate::Rumors::gossip) + /// and [`Rumors::gossip_when`](crate::Rumors::gossip_when)). + Gossip, + /// This side is retiring ([`Peer::retire`](crate::Peer::retire)). + Retire, +} + +/// What identifies one observed directed stream. +#[non_exhaustive] +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct StreamInfo { + /// Which of the session's streams this is. + pub id: StreamId, + /// Whether this side sent or received the stream's messages. + pub direction: Direction, +} + +/// One session stream's identity. +#[non_exhaustive] +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum StreamId { + /// The session's control stream: preamble, greeting, identity + /// hand-off, epilogue. + Control, + /// One reconciliation data stream. + Data { + /// The elected role that speaks this stream's frames. + /// + /// Sent data streams are spoken by this side's elected role; + /// received ones by the counterparty's. + speaker: Role, + /// The stream's wire index, `0..`[`STREAM_COUNT`](crate::link::STREAM_COUNT): + /// the same index the stream's on-wire open label carries. + index: u8, + }, +} + +/// The direction of one observed stream, from this peer's perspective. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Direction { + /// This side wrote the stream's messages. + Sent, + /// This side read the stream's messages. + Received, +} + +/// One side's elected role in a session's reconciliation descent. +/// +/// Decided after the greetings are exchanged (the smaller advertised +/// set initiates; see [`SessionObserver::elected`]). +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Role { + /// This role asks the opening question and absorbs the final + /// leaves. + Initiator, + /// This role answers the opening question. + Responder, +} + +/// The observation state a peer carries: the attached handler, if any +/// — shared, like the replica state, by every handle to one peer +/// identity. +#[derive(Clone, Default)] +pub(crate) struct Attachment { + handler: Option>, +} + +impl std::fmt::Debug for Attachment { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("Attachment") + .field("attached", &self.handler.is_some()) + .finish() + } +} + +impl Attachment { + /// Attach `observer`; later sessions ask it for session handlers. + pub(crate) fn attach(&mut self, observer: Arc) { + self.handler = Some(observer); + } + + /// Enter one session: create its handle. + /// + /// The handle is inert — every invocation a no-op branch — when no + /// observer is attached, when the observer declines the session, + /// or when the dialect is not observable (`Protocol::V1`'s frozen + /// wire is not a CBOR sequence; the module docs state the + /// exclusion). + pub(crate) fn begin(&self, kind: SessionKind, protocol: Protocol) -> SessionHandle { + let Some(handler) = &self.handler else { + return SessionHandle::default(); + }; + if protocol != Protocol::V2 { + return SessionHandle::default(); + } + let info = SessionInfo { kind, protocol }; + let Some(session) = handler.session(&info) else { + return SessionHandle::default(); + }; + // The control stream's two directions open with the session + // itself: create both handlers now, ahead of the preamble. + let sent = session.stream(&StreamInfo { + id: StreamId::Control, + direction: Direction::Sent, + }); + let received = session.stream(&StreamInfo { + id: StreamId::Control, + direction: Direction::Received, + }); + SessionHandle { + inner: Some(Arc::new(HandleInner { + session, + control_sent: Mutex::new(sent), + control_received: Mutex::new(received), + })), + } + } +} + +/// One session's observation handle: cheap to clone, inert when no +/// handler observes the session. +/// +/// The session machinery threads a clone to every layer that emits or +/// accepts wire items (the pattern the stats recorder set). Data +/// streams create their own owned [`StreamObserver`]s through +/// [`data`](Self::data) when they open; the control stream's two +/// handlers live here, behind mutexes, because the control stream's +/// items are written from several protocol layers in sequence — the +/// locks are uncontended by construction (each direction's items are +/// protocol-ordered) and absent entirely from the unattached path. +#[derive(Clone, Default)] +pub(crate) struct SessionHandle { + inner: Option>, +} + +struct HandleInner { + session: Box, + control_sent: Mutex>>, + control_received: Mutex>>, +} + +impl SessionHandle { + /// Whether any handler observes this session. + pub(crate) fn attached(&self) -> bool { + self.inner.is_some() + } + + /// Observe one item sent on the control stream. + pub(crate) fn control_sent(&self, bytes: &[u8]) { + if let Some(inner) = &self.inner { + observe_control(&inner.control_sent, bytes); + } + } + + /// Observe one item received on the control stream. + pub(crate) fn control_received(&self, bytes: &[u8]) { + if let Some(inner) = &self.inner { + observe_control(&inner.control_received, bytes); + } + } + + /// Report the session's decided role election. + pub(crate) fn elected(&self, role: Role) { + if let Some(inner) = &self.inner { + inner.session.elected(role); + } + } + + /// Create the handler for one opening data stream, if the session + /// handler wants it. + pub(crate) fn data( + &self, + speaker: Role, + index: u8, + direction: Direction, + ) -> Option> { + let inner = self.inner.as_ref()?; + inner.session.stream(&StreamInfo { + id: StreamId::Data { speaker, index }, + direction, + }) + } +} + +/// Invoke one control-direction handler under its lock. +/// +/// A poisoned lock means an earlier invocation panicked (an +/// application handler's panic, already propagating through the +/// session); keep delivering to the handler rather than silently +/// dropping the direction. +fn observe_control(slot: &Mutex>>, bytes: &[u8]) { + let mut guard = slot.lock().unwrap_or_else(PoisonError::into_inner); + if let Some(observer) = guard.as_mut() { + observer.message(bytes); + } +} + +/// A reader adapter that retains a copy of every delivered byte, so an +/// exact item-shaped read (a frame, a greeting, a hand-off) can hand +/// its observer the item's true wire bytes rather than a re-encoding. +pub(crate) struct CaptureRead<'a, R: ?Sized> { + captured: Vec, + inner: &'a mut R, +} + +impl<'a, R: ?Sized> CaptureRead<'a, R> { + /// Capture everything the wrapped reader delivers from here on. + pub(crate) fn new(inner: &'a mut R) -> Self { + Self { + captured: Vec::new(), + inner, + } + } + + /// The bytes delivered through this adapter so far. + pub(crate) fn bytes(&self) -> &[u8] { + &self.captured + } +} + +impl tokio::io::AsyncRead for CaptureRead<'_, R> +where + R: tokio::io::AsyncRead + Unpin + ?Sized, +{ + fn poll_read( + self: std::pin::Pin<&mut Self>, + cx: &mut std::task::Context<'_>, + buf: &mut tokio::io::ReadBuf<'_>, + ) -> std::task::Poll> { + let this = self.get_mut(); + let before = buf.filled().len(); + let poll = std::pin::Pin::new(&mut *this.inner).poll_read(cx, buf); + if let std::task::Poll::Ready(Ok(())) = &poll { + this.captured.extend_from_slice(&buf.filled()[before..]); + } + poll + } +} + +#[cfg(test)] +mod tests; diff --git a/src/observe/tests.rs b/src/observe/tests.rs new file mode 100644 index 000000000..d21548f74 --- /dev/null +++ b/src/observe/tests.rs @@ -0,0 +1,105 @@ +use std::sync::atomic::{AtomicUsize, Ordering}; +use std::sync::{Arc, Mutex}; + +use tokio::io::AsyncReadExt; + +use super::*; + +/// A handler that counts what it is asked for and records stream +/// identities, so the plumbing tests can see exactly which levels were +/// created. +#[derive(Default)] +struct Counting { + sessions: AtomicUsize, + infos: Mutex>, + streams: Arc>>, +} + +impl Observer for Counting { + fn session(&self, session: &SessionInfo) -> Option> { + self.sessions.fetch_add(1, Ordering::Relaxed); + self.infos.lock().unwrap().push(*session); + Some(Box::new(CountingSession { + streams: Arc::clone(&self.streams), + })) + } +} + +struct CountingSession { + streams: Arc>>, +} + +impl SessionObserver for CountingSession { + fn stream(&self, stream: &StreamInfo) -> Option> { + self.streams.lock().unwrap().push(*stream); + Some(Box::new(Sink)) + } +} + +struct Sink; + +impl StreamObserver for Sink { + fn message(&mut self, _: &[u8]) {} +} + +/// An unattached peer's session handle is inert: nothing is created and +/// every invocation is a no-op, whatever the session kind. +#[test] +fn unattached_handles_are_inert() { + let attachment = Attachment::default(); + let handle = attachment.begin(SessionKind::Gossip, Protocol::V2); + assert!(!handle.attached()); + handle.control_sent(b"x"); + handle.control_received(b"x"); + handle.elected(Role::Initiator); + assert!(handle.data(Role::Initiator, 0, Direction::Sent).is_none()); +} + +/// Beginning an observed V2 session creates the control stream's two +/// directed handlers immediately, ahead of any wire traffic. +#[test] +fn begin_creates_the_control_handlers() { + let observer = Arc::new(Counting::default()); + let mut attachment = Attachment::default(); + attachment.attach(observer.clone()); + + let handle = attachment.begin(SessionKind::Bootstrap, Protocol::V2); + assert!(handle.attached()); + assert_eq!(observer.sessions.load(Ordering::Relaxed), 1); + let infos = observer.infos.lock().unwrap(); + assert_eq!( + *infos, + vec![SessionInfo { + kind: SessionKind::Bootstrap, + protocol: Protocol::V2, + }] + ); + let streams = observer.streams.lock().unwrap(); + assert_eq!( + *streams, + vec![ + StreamInfo { + id: StreamId::Control, + direction: Direction::Sent, + }, + StreamInfo { + id: StreamId::Control, + direction: Direction::Received, + }, + ] + ); +} + +/// The capture adapter retains exactly the bytes it delivered, across +/// split reads, so an observed exact read hands its handler the true +/// wire bytes. +#[tokio::test] +async fn capture_read_retains_delivered_bytes() { + let mut source: &[u8] = b"one item"; + let mut capture = CaptureRead::new(&mut source); + let mut first = [0u8; 3]; + capture.read_exact(&mut first).await.unwrap(); + let mut rest = Vec::new(); + capture.read_to_end(&mut rest).await.unwrap(); + assert_eq!(capture.bytes(), b"one item"); +} diff --git a/src/peer.rs b/src/peer.rs index f5270c49d..d799c3aed 100644 --- a/src/peer.rs +++ b/src/peer.rs @@ -11,7 +11,9 @@ use tokio::sync::{Mutex, watch}; use crate::bookmark::{BookmarkError, Bookmarked, NoBookmark}; use crate::link::{Acceptor, Connector, Link}; -use crate::message::{Message, PayloadDeserializer}; +pub use crate::message::{DEFAULT_PAYLOAD_DEPTH_LIMIT, PayloadDepthLimit}; +use crate::message::{EncodeError, PayloadCodec}; +use crate::observe::{Attachment, Observer}; use crate::tree::Tree; pub use crate::tree::mirror::streaming::remote::DEFAULT_TARGET_MESSAGE_SIZE; use crate::tree::mirror::streaming::remote::RunBudget; @@ -28,7 +30,9 @@ mod bootstrap; mod gossip; pub use bootstrap::{BookmarkedBootstrap, Bootstrap, Joined}; -pub use gossip::{Gossiped, Led, PROTOCOL_MAGIC, Retire, Unbookmarked}; +#[cfg(feature = "protocol-v1")] +pub use gossip::PROTOCOL_MAGIC; +pub use gossip::{Gossiped, Led, Retire, Unbookmarked}; /// The start and end of a [`Rumors`]'s lifecycle. /// @@ -158,10 +162,14 @@ pub struct Peer { /// Separate from `inner` because persisting is `async` and the record is /// `!Clone`; see [`Bookmarked`]. pub(crate) bookmark: Arc>>, - /// The payload deserializer minted at construction: the typed ingress - /// every gossip session's supplied leaf records decode through (see - /// [`Message::deserializer`](crate::message::Message::deserializer)). - pub(crate) deserializer: PayloadDeserializer, + /// The payload codec built at construction: the typed ingress every + /// gossip session's supplied leaf records decode through. + /// + /// Carries the [`payload_depth_limit`](Self::payload_depth_limit) + /// beside the codec's fn pointers (see [`PayloadCodec`]). + pub(crate) codec: PayloadCodec, + /// The wire-observation handler selected by [`observe`](Self::observe). + pub(crate) observe: Attachment, } /// The replica's shared mutable state, behind the `watch` channel every @@ -189,16 +197,17 @@ impl std::fmt::Debug for Peer { } } -impl Peer { +impl Peer { /// Create the distinguished seed rumor set: the single root from which /// every other participant must [`bootstrap`](Peer::bootstrap). /// /// Call this exactly once per universe of cooperating peers. /// - /// The payload type's [`DeserializeOwned`] lives here, at - /// construction: the peer mints its payload deserializer once, and - /// every gossip session decodes through it, so the gossip entry - /// points themselves carry no serde bounds. + /// The payload type's serde obligations — [`Serialize`] and + /// [`DeserializeOwned`] both — live here, at construction: the peer + /// builds its payload codec once, every send serializes through it, + /// and every gossip session decodes through it, so neither the send + /// paths nor the gossip entry points carry serde bounds of their own. pub fn seed() -> Self { Self::seed_rng(&mut OsRng) } @@ -217,7 +226,8 @@ impl Peer { tree: Tree::new(), }), bookmark: Arc::new(Mutex::new(Bookmarked::new(NoBookmark))), - deserializer: Message::deserializer::(), + codec: PayloadCodec::new::(PayloadDepthLimit::default()), + observe: Attachment::default(), } } } @@ -400,7 +410,7 @@ impl Peer { /// pinned: each in-flight dispute (one disputed subtree, the unit /// the table below counts as a disputed scope) charges the budget /// a 5431 B envelope (recomputed exactly by test), and each disputed - /// message costs 35 B of wire overhead on top of its record + /// message costs 43 B of wire overhead on top of its record /// (calibrated by deterministic byte counts, /// `tests/dispute_wire.rs`). /// @@ -408,50 +418,50 @@ impl Peer { /// trade. A session's worst-case slowdown, relative to a session /// limited only by wire time, is about /// - /// > `slowdown ≈ max(1, BDP × 5431 / (budget × (35 + m)))` + /// > `slowdown ≈ max(1, BDP × 5431 / (budget × (43 + m)))` /// /// Read it as a ratio of two message counts: how many disputed - /// messages the wire holds, `BDP / (35 + m)`, against how many the + /// messages the wire holds, `BDP / (43 + m)`, against how many the /// budget keeps in flight, `budget / 5431`. Slowdown 1 is /// wire-time-optimal: bandwidth-bound stays bandwidth-bound. /// /// The estimate has a stated accuracy band. It overstates the /// window by roughly `F / budget`, where `F` is the corpus-fixed /// component of the real charge, so the slowdown it returns runs - /// ~2.3× low at a 10 MB budget, ~1.6× low at 16 MiB, and within a + /// ~2–3× low at ~10 MB budgets, ~1.4× low at ~26 MB, and within a /// few percent past ~300 MB. It also prices no population ceiling, /// so where windows reach corpus scale, the exact solve's numbers /// (the table below, and the pinned crossover) replace it. /// Measured: sessions whose serialized one-way trips are counted - /// exactly on a virtual clock, at 10–31 MB budgets on the design - /// corpus, ran 1.3–1.65× the form's figure + /// exactly on a virtual clock, at 8–26 MB budgets on the minimal + /// and design corpora, ran 1.35–1.96× the form's figure /// (`tests/tradeoff_probe.rs`). /// /// The ballpark answers, at the specification BDP: /// /// - **Is the default enough?** For any corpus whose mean encoded - /// record size is at least 60 B, yes: the default imposes no + /// record size is at least 52 B, yes: the default imposes no /// window-induced serialization at all, because the in-flight /// disputes' own transfer time covers the round trip. That - /// 60 B crossover comes from the exact solve, evaluated + /// 52 B crossover comes from the exact solve, evaluated /// self-consistently (each record size at its own BDP-scale /// corpus: the specification BDP in `m`-sized records, per side) /// and pinned by `default_crossover_matches_the_solve`; - /// the closed form's safe-side estimate is ~91 B. + /// the closed form's safe-side estimate is ~84 B. /// - **What budget removes the wait entirely?** About - /// `BDP × 5431 / (35 + m)` bytes. The design record (`m = 172`) - /// needs ~330 MB, where the solve agrees with the form to three + /// `BDP × 5431 / (43 + m)` bytes. The design record (`m = 172`) + /// needs ~316 MB, where the solve agrees with the form to three /// digits (this is the design point the envelope is pinned at). - /// A minimal `u64`-record corpus (9 B encoded) needs ~1.5 GB by - /// the form, ~1.1 GB by the solve: population caps thin the deep + /// A minimal `u64`-record corpus (9 B encoded) needs ~1.3 GB by + /// the form, ~0.8 GB by the solve: population caps thin the deep /// charge at BDP-scale corpora, so the estimate is conservative /// there. /// - **What does a smaller budget cost?** Smooth latency, never /// memory, and only on the interleaved dispute walk (bulk supply /// runs stream outside the window). `u64` records at the default - /// run at ~4.3× wire time for a BDP-scale corpus, and the factor + /// run at ~2.6× wire time for a BDP-scale corpus, and the factor /// grows slowly with set size as the derived window narrows: - /// ~13.6× at 10⁷ messages, ~25.3× at 10¹⁰ (all derived from the + /// ~11.5× at 10⁷ messages, ~21.4× at 10¹⁰ (all derived from the /// solve). `tests/window_operator.rs` holds the wave model /// against measured sessions on a bandwidth-limited link. /// @@ -463,7 +473,7 @@ impl Peer { /// session of 62500-message corpora a side; larger corpora derive /// narrower windows. Each cell then applies the measured wave form /// `slowdown = max(1, BDP_messages / K)`, with - /// `BDP_messages = BDP / (35 + m)` evaluated at the specification + /// `BDP_messages = BDP / (43 + m)` evaluated at the specification /// BDP of 12.5 MB (the wave form is measured: /// `tests/window_knee.rs`, `tests/window_operator.rs`). One /// caution when reading it: in rows whose window reaches the @@ -496,6 +506,29 @@ impl Peer { self } + /// Attach a wire-observation handler to this peer's future sessions. + /// + /// For every session the peer enters — gossip, bootstrap serving, + /// and retirement alike — the handler is asked for a per-session + /// observer, which sees each directed stream's protocol messages + /// as raw CBOR items. The full contract (the three handler levels, + /// the ordering and back-pressure rules, what exactly is observed) + /// is the [`observe`](crate::observe) module's. + /// + /// Observation never changes the wire: an observed session's bytes + /// are identical to an unobserved one's. Like + /// [`protocol`](Self::protocol), the choice follows the peer + /// through [`into_rumors`](Self::into_rumors), cloning and + /// reunion, bookmarking, and retirement; every [`Rumors`] clone + /// shares the one handler. To observe a joining peer's own + /// bootstrap session, attach on the builder instead + /// ([`Bootstrap::observe`]). + #[must_use] + pub fn observe(mut self, observer: Arc) -> Self { + self.observe.attach(observer); + self + } + /// Bound the encoded size of the batched messages this peer sends. /// /// When the default protocol supplies a subtree the counterparty lacks, @@ -527,9 +560,9 @@ impl Peer { /// the wire's maximally disputed reply (the decode side's documented /// per-reply memory unit), so default batching never raises the wire's /// established memory ceiling. Any value is safe: zero degrades to one - /// leaf per message, and values above the wire's framing ceiling + /// leaf per message, and values above the wire's run byte cap /// (`u32::MAX` less the frame envelope) saturate to it, so a run built - /// within the target always fits its length header. + /// within the target always fits the cap. /// /// Like [`protocol`](Self::protocol), the choice follows the peer /// through [`into_rumors`](Self::into_rumors), cloning and reunion, @@ -542,6 +575,72 @@ impl Peer { self } + /// Bound the nesting depth of the message payloads this peer sends + /// and accepts. + /// + /// A payload value is accepted only if decoding its CBOR encoding as + /// the peer's payload type recurses at most `limit` steps. What + /// consumes one step is the decode engine's own accounting for that + /// type — arrays, maps, and tags each do, and so can type-driven + /// wrappers such as an enum's variant scope — so the bound is + /// engine-defined, not a structural count of the bytes. The default, + /// [`DEFAULT_PAYLOAD_DEPTH_LIMIT`], is 256 steps: exactly the bound + /// the decoder applies by default, so a fleet at the default sees no + /// acceptance change on existing content. + /// + /// Three points enforce the one bound: + /// + /// - **Send** ([`Rumors::send`](crate::Rumors::send), + /// [`Batch::send`](crate::Batch::send)): admission runs the exact + /// decode every receiver's wire ingress runs — same payload type, + /// same limit, same engine — so an over-deep value is rejected at + /// its author, at the moment of choice, with a typed + /// [`EncodeError`]. + /// - **Handshake**: the greeting carries each side's configured + /// limit, and a session proceeds only if the two are exactly equal; + /// a mismatch in either direction aborts both sides with + /// [`Error::PayloadDepthMismatch`](crate::Error::PayloadDepthMismatch) + /// before anything else — the converged-session short-circuit + /// included — so a mixed configuration is caught at every pairing. + /// - **Wire ingress**: every payload decode runs under this same + /// limit, so over-deep *content* supplied by a nonconforming + /// implementation fails its session with a typed decode error. + /// The bound governs the decode's recursion, not the bytes' shape: + /// deep byte patterns the engine consumes without recursing (a tag + /// chain in a scalar position, say) decode fine and are harmless. + /// + /// Together those establish the invariant this setting exists for: + /// between conforming peers, no session can fail on payload depth at + /// all — true by construction within a decode-engine version, because + /// admission and ingress are one computation, not two accountings + /// held in agreement. Over-deep values are rejected at their author, + /// and mismatched fleets are rejected at the handshake. (A fleet + /// mixing builds whose CBOR engine versions account recursion + /// differently could still diverge; upgrading the engine is a + /// fleet-coordination event in the same register as changing this + /// limit.) The limit is a property of the *shared set* — every + /// replica must be able to hold and forward all content — which is + /// why the handshake demands equality rather than negotiating: a + /// peer whose session bound dropped below its own configured limit + /// could already hold messages deeper than the negotiated bound, + /// which it would then not be allowed to gossip. Changing the limit + /// is therefore a fleet-coordinated configuration event, like + /// changing the selected [`Protocol`], never a per-peer tuning + /// parameter. + /// + /// The frozen `Protocol::V1` greeting cannot carry the parameter, so + /// V1 sessions enforce only at decode, and a mixed-limit V1 fleet can + /// still fail mid-session, conditional on content. + /// + /// Like [`protocol`](Self::protocol), the choice follows the peer + /// through [`into_rumors`](Self::into_rumors), cloning and reunion, + /// bookmarking, and retirement. + #[must_use] + pub fn payload_depth_limit(mut self, limit: PayloadDepthLimit) -> Self { + self.codec = self.codec.with_limit(limit); + self + } + /// Convert the [`Peer`] into a [`Rumors`] so it can [`send`](Rumors::send), /// [`redact`](Rumors::redact), and [`gossip`](Rumors::gossip). /// @@ -553,29 +652,34 @@ impl Peer { Rumors::new(self) } - pub(crate) fn send(&self, message: T) -> Batch<'_, T> + pub(crate) fn send(&self, message: T) -> Result<(), EncodeError> where - T: Serialize + Send + Sync + 'static, + T: Send + Sync + 'static, { - let mut batch = self.batch(); - batch.send(message); - batch + let mut batch = Batch::new(&self.inner, self.codec); + batch.send(message)?; + batch.commit(); + Ok(()) } - pub(crate) fn redact(&self, version: &Version) -> Batch<'_, T> + pub(crate) fn redact(&self, version: &Version) where T: Send + Sync, { - let mut batch = self.batch(); + let mut batch = Batch::new(&self.inner, self.codec); batch.redact(version); - batch + batch.commit(); } - pub(crate) fn batch(&self) -> Batch<'_, T> + pub(crate) fn batch(&self, f: F) -> Result where T: Send + Sync, + F: for<'s> FnOnce(&'s mut Batch<'_, T>) -> Result, { - Batch::new(&self.inner) + let mut batch = Batch::new(&self.inner, self.codec); + let result = f(&mut batch)?; + batch.commit(); + Ok(result) } pub(crate) fn snapshot(&self) -> Snapshot { diff --git a/src/peer/bootstrap.rs b/src/peer/bootstrap.rs index da574953c..00cc2dd1a 100644 --- a/src/peer/bootstrap.rs +++ b/src/peer/bootstrap.rs @@ -6,14 +6,19 @@ use std::marker::PhantomData; use tokio::io::{AsyncRead, AsyncWrite}; +use std::sync::Arc; + use crate::bookmark::{Bookmark, BookmarkError}; use crate::link::{Acceptor, Connector, Link}; +use crate::message::PayloadDepthLimit; +use crate::observe::{Attachment, Observer}; use crate::tree::mirror::streaming::remote::RunBudget; use crate::tree::mirror::streaming::window::WindowConfig; use crate::{Error, Peer, Protocol}; use super::gossip::Unbookmarked; +use serde::Serialize; use serde::de::DeserializeOwned; /// Configuration for joining an existing universe: the builder behind /// [`Peer::bootstrap`]. @@ -27,8 +32,9 @@ use serde::de::DeserializeOwned; /// /// Every setting here is the new peer's own, selected one session /// early: [`protocol`](Self::protocol), -/// [`sync_memory_budget`](Self::sync_memory_budget), and -/// [`target_message_size`](Self::target_message_size) each state what they +/// [`sync_memory_budget`](Self::sync_memory_budget), +/// [`target_message_size`](Self::target_message_size), and +/// [`payload_depth_limit`](Self::payload_depth_limit) each state what they /// change about the bootstrap session itself, and the joined peer keeps /// the choice exactly as if selected through the matching [`Peer`] method. /// [`bookmark`](Self::bookmark) additionally persists the received @@ -36,9 +42,9 @@ use serde::de::DeserializeOwned; /// [`BookmarkedBootstrap`] state (whose `join` reports outcomes as a /// [`Joined`], since a persist can fail while the peer lives). /// -/// The builder is `Copy`: after a mutual-bootstrap bail -/// ([`join`](Self::join)'s `Ok(None)`) or a failed session, the same -/// configuration retries against another provider as-is. +/// The builder is `Clone`: after a mutual-bootstrap bail +/// ([`join`](Self::join)'s `Ok(None)`) or a failed session, a clone of +/// the same configuration retries against another provider as-is. /// /// # The provider's side /// @@ -52,25 +58,46 @@ use serde::de::DeserializeOwned; /// one, never by both sides. #[must_use = "a `Bootstrap` does nothing until `join` runs it against a link"] pub struct Bootstrap { + /// The wire protocol selected by [`protocol`](Self::protocol): spoken + /// by the join session, carried into the joined peer. pub(crate) protocol: Protocol, + /// The window policy selected by + /// [`sync_memory_budget`](Self::sync_memory_budget): the join + /// session's reconciliation memory bound, carried into the joined + /// peer. pub(crate) window: WindowConfig, + /// The supply-run sizing budget selected by + /// [`target_message_size`](Self::target_message_size): the join + /// session's byte target, carried into the joined peer. pub(crate) run_budget: RunBudget, + /// The payload depth limit selected by + /// [`payload_depth_limit`](Self::payload_depth_limit): the join + /// session's ingress bound, carried into the joined peer's codec. + pub(crate) payload_depth_limit: PayloadDepthLimit, + /// The wire-observation handler selected by + /// [`observe`](Self::observe), carried into the joined peer. + pub(crate) observe: Attachment, /// Covariant, `Send`/`Sync`-neutral marker for the payload type the /// new [`Peer`] will carry. marker: PhantomData T>, } -// Manual, unbounded impls: the payload type is phantom (the builder holds -// configuration only), so the `T: Clone`/`T: Copy` bounds `derive` would -// add have nothing to constrain. +// A manual, unbounded impl: the payload type is phantom (the builder +// holds configuration only), so the `T: Clone` bound `derive` would add +// has nothing to constrain. impl Clone for Bootstrap { fn clone(&self) -> Self { - *self + Self { + protocol: self.protocol, + window: self.window, + run_budget: self.run_budget, + payload_depth_limit: self.payload_depth_limit, + observe: self.observe.clone(), + marker: PhantomData, + } } } -impl Copy for Bootstrap {} - /// The configuration only; the payload type parameter carries no state. impl std::fmt::Debug for Bootstrap { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { @@ -78,6 +105,7 @@ impl std::fmt::Debug for Bootstrap { .field("protocol", &self.protocol) .field("window", &self.window) .field("run_budget", &self.run_budget) + .field("payload_depth_limit", &self.payload_depth_limit) .finish() } } @@ -90,6 +118,8 @@ impl Bootstrap { protocol: Protocol::default(), window: WindowConfig::default(), run_budget: RunBudget::default(), + payload_depth_limit: PayloadDepthLimit::default(), + observe: Attachment::default(), marker: PhantomData, } } @@ -149,6 +179,33 @@ impl Bootstrap { self } + /// Bound the nesting depth of the message payloads the bootstrap + /// session, and every later session, accepts. + /// + /// The join session decodes the provider's supplied records before a + /// [`Peer`] exists, so the bound is selected here, one session early. + /// The default, the scope accounting, and the fleet-coordination + /// contract are [`Peer::payload_depth_limit`]'s; the joined peer + /// behaves exactly as if it had called it. + pub fn payload_depth_limit(mut self, limit: PayloadDepthLimit) -> Self { + self.payload_depth_limit = limit; + self + } + + /// Attach a wire-observation handler, starting with the bootstrap + /// session itself. + /// + /// The join is the one session that runs before the peer exists, + /// so observing it means selecting the handler here; the joined + /// peer then keeps the handler exactly as [`Peer::observe`] would + /// attach it; an observer that numbers sessions will count the join + /// as the first session it sees. The observation contract is the + /// [`observe`](crate::observe) module's. + pub fn observe(mut self, observer: Arc) -> Self { + self.observe.attach(observer); + self + } + /// Persist the received identity as part of joining: the peer comes /// back already [`bookmark`](Peer::bookmark)ed. /// @@ -190,7 +247,7 @@ impl Bootstrap { /// `Ok(None)` means the counterparty was itself still bootstrapping, /// so neither side had anything to share and no identity moved. It is /// a clean session boundary: the link remains usable. Connect to - /// another peer and try again (the builder is `Copy`, so the same + /// another peer and try again (the builder is `Clone`, so the same /// configuration retries as-is). /// /// On `Ok(Some(peer))` the provider has confirmed committing its side @@ -216,7 +273,7 @@ impl Bootstrap { link: &mut Link, ) -> Result>, Error> where - T: DeserializeOwned + Send + Sync + 'static, + T: Serialize + DeserializeOwned + Eq + Send + Sync + 'static, CR: AsyncRead + Unpin + Send, CW: AsyncWrite + Unpin + Send, C: Connector, @@ -283,6 +340,20 @@ impl BookmarkedBootstrap { self } + /// Bound payload nesting depth; the contract is + /// [`Bootstrap::payload_depth_limit`]'s. + pub fn payload_depth_limit(mut self, limit: PayloadDepthLimit) -> Self { + self.config = self.config.payload_depth_limit(limit); + self + } + + /// Attach a wire-observation handler; the contract is + /// [`Bootstrap::observe`]'s. + pub fn observe(mut self, observer: Arc) -> Self { + self.config = self.config.observe(observer); + self + } + /// Join the provider's universe and durably record the received /// identity, reporting what survived as a [`Joined`]. /// @@ -296,7 +367,7 @@ impl BookmarkedBootstrap { /// in every outcome that never used it. pub async fn join(self, link: &mut Link) -> Joined where - T: DeserializeOwned + Send + Sync + 'static, + T: Serialize + DeserializeOwned + Eq + Send + Sync + 'static, CR: AsyncRead + Unpin + Send, CW: AsyncWrite + Unpin + Send, C: Connector, diff --git a/src/peer/bootstrap/tests.rs b/src/peer/bootstrap/tests.rs index 08166ff44..9a600ed52 100644 --- a/src/peer/bootstrap/tests.rs +++ b/src/peer/bootstrap/tests.rs @@ -1,12 +1,12 @@ //! The [`Bootstrap`] builder's configuration plumbing: what the builder -//! holds, and what the minted [`Peer`] retains. +//! holds, and what the joined [`Peer`] retains. //! //! The knobs' *behavioral* contracts are pinned end to end elsewhere — //! run sizing under the exchanged minimum in `tests/target_message_size.rs`, //! protocol persistence in `tests/bootstrap.rs`, the session bytes in //! `tests/bootstrap_snapshot.rs`. This suite pins the plumbing those tests //! rest on: every builder knob reaches the builder's state, and every -//! stored choice reaches the minted peer unchanged. +//! stored choice reaches the joined peer unchanged. use super::{Bootstrap, Joined}; use crate::bookmark::NoBookmark; @@ -29,7 +29,7 @@ fn budget_bytes(window: WindowConfig) -> usize { } } -/// Serve one bootstrap from `provider` and hand back the minted peer. +/// Serve one bootstrap from `provider` and hand back the joined peer. fn join_from_seed(config: Bootstrap) -> Peer { pollster::block_on(async { let provider = Peer::::seed().into_rumors(); @@ -79,13 +79,13 @@ fn knobs_store_the_selected_values() { assert_ne!(saturated.run_budget.bytes(), usize::MAX); } -/// The minted peer retains every builder choice for its later sessions. +/// The joined peer retains every builder choice for its later sessions. /// /// The configured budget and run target arrive on the [`Peer`] exactly /// as if selected through [`Peer::sync_memory_budget`] and /// [`Peer::target_message_size`]. #[test] -fn minted_peer_retains_the_configuration() { +fn joined_peer_retains_the_configuration() { let peer = join_from_seed( Peer::::bootstrap() .sync_memory_budget(CUSTOM_BUDGET) @@ -96,12 +96,12 @@ fn minted_peer_retains_the_configuration() { assert_eq!(peer.run_budget, RunBudget::from_bytes(CUSTOM_TARGET)); } -/// Negative control for [`minted_peer_retains_the_configuration`]: an -/// unconfigured join mints a peer at the crate defaults, so the retention +/// Negative control for [`joined_peer_retains_the_configuration`]: an +/// unconfigured join produces a peer at the crate defaults, so the retention /// test above cannot pass by the defaults happening to equal the custom /// values. #[test] -fn unconfigured_join_mints_the_defaults() { +fn unconfigured_join_produces_the_defaults() { let peer = join_from_seed(Peer::::bootstrap()); assert_eq!(peer.protocol, Protocol::default()); assert_eq!(budget_bytes(peer.window), DEFAULT_SYNC_MEMORY_BUDGET); @@ -132,7 +132,7 @@ fn bookmark_transition_preserves_and_accepts_knobs() { } } -/// A bookmarked join drives the same session plumbing: the minted peer +/// A bookmarked join drives the same session plumbing: the joined peer /// retains the session knobs exactly as an unbookmarked join's would, /// arriving through the [`Joined::Joined`] arm. #[test] @@ -149,7 +149,7 @@ fn bookmarked_join_retains_the_configuration() { joined }); let Joined::Joined { peer } = outcome else { - panic!("an established provider and infallible bookmark must mint a joined peer"); + panic!("an established provider and infallible bookmark must produce a joined peer"); }; assert_eq!(budget_bytes(peer.window), CUSTOM_BUDGET); assert_eq!(peer.run_budget, RunBudget::from_bytes(CUSTOM_TARGET)); diff --git a/src/peer/gossip.rs b/src/peer/gossip.rs index 1194055ef..f8f4dd39d 100644 --- a/src/peer/gossip.rs +++ b/src/peer/gossip.rs @@ -20,7 +20,8 @@ use crate::link::{ Acceptor, Connector, Link, SessionState, erased::{DynAcceptor, DynConnector}, }; -use crate::message::{Message, PayloadDeserializer}; +use crate::message::PayloadCodec; +use crate::observe::{SessionHandle, SessionKind}; #[cfg(any(test, feature = "protocol-v1"))] use crate::tree::mirror::{ alternating::{self, local as alternating_local, remote as alternating_remote}, @@ -44,19 +45,29 @@ use crate::{ use super::{Inner, Peer, bootstrap::Bootstrap}; +use serde::Serialize; use serde::de::DeserializeOwned; -/// Magic bytes that open every `rumors` gossip session's preamble frame. -pub const PROTOCOL_MAGIC: [u8; 6] = *b"RUMORS"; +/// Magic bytes that open a V1 gossip session's preamble frame. +/// +/// A [`Protocol::V2`] session opens with the self-described CBOR tag +/// instead, and carries its protocol magic in the preamble's opening +/// array; this raw marker belongs to the V1 wire dialect alone, so it +/// is exposed only with the `protocol-v1` feature. (A V2 endpoint +/// still recognizes these bytes internally, to diagnose a legacy peer +/// as the version mismatch it is.) +#[cfg(feature = "protocol-v1")] +pub const PROTOCOL_MAGIC: [u8; 6] = crate::tree::mirror::handshake::LEGACY_MAGIC; -/// The one epilogue marker byte each side writes on the control stream after -/// all of its session work, under [`Protocol::V2`]. +/// The epilogue marker each side writes on the control stream after all +/// of its session work, under [`Protocol::V2`]: the CBOR text item `"."`. /// /// Reading the peer's marker is what lets `Ok` certify that the peer -/// completed and committed too. Deliberately distinct from -/// [`PROTOCOL_MAGIC`]'s first byte (`b'R'`): a desynchronized peer that -/// starts its next preamble where an epilogue belongs is diagnosed as a -/// protocol violation, not mistaken for completion. -const EPILOGUE_MARKER: u8 = b'.'; +/// completed and committed too. As an item, the marker keeps the control +/// stream a pure CBOR sequence; its leading byte is deliberately distinct +/// from the self-described tag opening a V2 preamble, so a desynchronized +/// peer that starts its next preamble where an epilogue belongs is +/// diagnosed as a protocol violation, not mistaken for completion. +const EPILOGUE_MARKER: [u8; 2] = [0x61, b'.']; /// A session's control read half with its concrete transport type erased. /// @@ -215,7 +226,7 @@ impl Peer { link: &'a mut Link, ) -> BoxFuture<'a, Result, Error>> where - T: DeserializeOwned + Send + Sync + 'static, + T: Serialize + DeserializeOwned + Eq + Send + Sync + 'static, CR: AsyncRead + Unpin + Send, CW: AsyncWrite + Unpin + Send, C: Connector, @@ -242,17 +253,20 @@ impl Peer { link: DynLinkParts<'a>, ) -> BoxFuture<'a, Result, Error>> where - T: DeserializeOwned + Send + Sync + 'static, + T: Serialize + DeserializeOwned + Eq + Send + Sync + 'static, { Box::pin(async move { let (read, write, connector, acceptor, epoch) = link; - // The peer-to-be's payload deserializer, minted before it - // exists: the bootstrap session's ingress decodes through it, - // and the constructed peer inherits it. - let deserializer = Message::deserializer::(); + // The peer-to-be's payload codec, built before it exists: + // the bootstrap session's ingress decodes through it, and the + // constructed peer inherits it. + let codec = PayloadCodec::new::(config.payload_depth_limit); + let observe = config + .observe + .begin(SessionKind::Bootstrap, config.protocol); // Magic/version/network/intent preamble first, before either protocol // is allowed to trust peer-declared frame lengths. - let mut staged = handshake::Staged::new(); + let mut staged = handshake::Staged::new(config.protocol); let remote = handshake::preamble( config.protocol, Network::BOOTSTRAP, @@ -260,6 +274,7 @@ impl Peer { &mut staged, read, write, + &observe, ) .await .map_err(Error::from)?; @@ -279,18 +294,19 @@ impl Peer { let reconcile = match config.protocol { Protocol::V2 => bootstrap_v2( (read, write, connector, acceptor, epoch), - deserializer, + codec, config.window, config.run_budget, both_bootstrapping, + observe.clone(), ), #[cfg(any(test, feature = "protocol-v1"))] - Protocol::V1 => bootstrap_v1(read, write, deserializer, both_bootstrapping), + Protocol::V1 => bootstrap_v1(read, write, codec, both_bootstrapping), }; let Some((root, mut read, mut write)) = reconcile.await? else { return Ok(None); }; - let party = party::receive(&mut read).await?; + let party = party::receive(config.protocol, &mut read, &observe).await?; // Our absorption of the received identity completes with the // in-memory `Peer` construction below, which cannot fail: certify // completion now, and require the provider's certificate so `Ok` @@ -298,7 +314,7 @@ impl Peer { // frozen.) On `Err` the received fork is dropped — its region // leaks, benignly, like any fork lost in flight. if config.protocol == Protocol::V2 { - epilogue(&mut read, &mut write).await?; + epilogue(&mut read, &mut write, &observe).await?; } let peer = Self { network: remote.network, @@ -310,7 +326,8 @@ impl Peer { tree: Tree::from_root(root), }), bookmark: Arc::new(Mutex::new(Bookmarked::new(NoBookmark))), - deserializer, + codec, + observe: config.observe, }; Ok(Some(peer)) }) @@ -327,7 +344,8 @@ impl Peer { window, run_budget, inner, - deserializer, + codec, + observe, .. } = self; let peer = Peer { @@ -337,7 +355,8 @@ impl Peer { run_budget, inner, bookmark: Arc::new(Mutex::new(Bookmarked::new(bookmark))), - deserializer, + codec, + observe, }; // A pristine seed has no identity worth recording yet; persisting it @@ -366,7 +385,8 @@ impl Peer { run_budget: peer.run_budget, inner: peer.inner, bookmark: Arc::new(Mutex::new(Bookmarked::new(NoBookmark))), - deserializer: peer.deserializer, + codec: peer.codec, + observe: peer.observe, }, error, }), @@ -403,7 +423,7 @@ impl Peer { C: Connector, A: Acceptor, { - let mut staged = handshake::Staged::new(); + let mut staged = handshake::Staged::new(self.protocol); let parts = match erase(link) { Ok(parts) => parts, // The fail-fast happened before any wire traffic: nothing of @@ -442,7 +462,7 @@ impl Peer { C: Connector, A: Acceptor, { - let mut staged = handshake::Staged::new(); + let mut staged = handshake::Staged::new(self.protocol); let parts = erase(link).map_err(Error::widen)?; let (_intent, result) = self.gossip_inner(Intent::Remain, &mut staged, parts).await; // Un-poison on clean completion: the session's own `Ok` under V2 is @@ -580,22 +600,37 @@ impl Peer { T: Send + Sync + 'static, { let (read, write, connector, acceptor, epoch) = link; - let deserializer = self.deserializer; + let codec = self.codec; // The session's stats recorder: under V2, both protocol // participants below share it (the walk counts disputes, gains, // sheds, and the window grant; the proxy's codec seam counts // bytes), and its snapshot rides the `Ok`. A session that ends // before reconciliation, and every V1 session, reports zeros. let stats = Recorder::default(); + // The session's observation handle: inert unless a handler is + // attached and the dialect is observable, and shared, like the + // recorder, by every layer that moves a wire item. + let kind = match intent { + Intent::Remain => SessionKind::Gossip, + Intent::Retire => SessionKind::Retire, + }; + let observe = self.observe.begin(kind, self.protocol); // Magic/version preamble: reject a non-rumors or incompatible peer // before the framing trusts any peer-supplied frame length. - let remote = - match handshake::preamble(self.protocol, self.network, intent, staged, read, write) - .await - { - Err(error) => return (Intent::Remain, Err(Error::from(error).widen())), - Ok(remote) => remote, - }; + let remote = match handshake::preamble( + self.protocol, + self.network, + intent, + staged, + read, + write, + &observe, + ) + .await + { + Err(error) => return (Intent::Remain, Err(Error::from(error).widen())), + Ok(remote) => remote, + }; let peer_bootstrapping = remote.network.is_bootstrap(); let self_retiring = intent == Intent::Retire; let peer_retiring = remote.intent == Intent::Retire; @@ -605,7 +640,7 @@ impl Peer { // epilogue markers pair up with no session body between them. if self_retiring && peer_retiring { if self.protocol == Protocol::V2 - && let Err(e) = epilogue(read, write).await + && let Err(e) = epilogue(read, write, &observe).await { return (Intent::Remain, Err(e.widen())); } @@ -621,7 +656,7 @@ impl Peer { // - The persisted record's own-party projection dominates the // snapshot's own-party version, so every own event this session can // transmit is durably accounted for before it crosses the wire, and - // a crash-and-reclaim can never remint a causal coordinate some + // a crash-and-reclaim can never reuse a causal coordinate some // replica already holds. A `send` committed while the record's // write is in flight lands *after* the snapshot: it stays out of // this session and the next session's update covers it. @@ -701,10 +736,11 @@ impl Peer { let reconciliation = Reconciliation { root: prior_tree.root, link: (read, write, connector, acceptor, epoch), - deserializer, + codec, window: self.window, run_budget: self.run_budget, stats: stats.clone(), + observe: observe.clone(), peer_bootstrapping, remote_network: remote.network, network: self.network, @@ -732,7 +768,7 @@ impl Peer { // The preamble rejects a peer that claims to both bootstrap and // retire, and we bailed early if we were retiring too, so no // party of ours is in flight here: `guarded.party` is `None`. - absorbed = match party::receive(read).await { + absorbed = match party::receive(self.protocol, read, &observe).await { Err(e) => return (Intent::Remain, Err(e.widen())), Ok(donated_party) => Some(donated_party), }; @@ -753,7 +789,7 @@ impl Peer { // the peer may hold the party even if the send errors, so it can // never be safely re-joined. let donated = guarded.party.take().expect("is_some"); - match party::send(donated, write).await { + match party::send(self.protocol, donated, write, &observe).await { Err(e) => { // A retiring donation in limbo must be assumed received: // report `Intent::Retire` alongside the error so that the @@ -864,7 +900,7 @@ impl Peer { // crossed the wire but whose epilogue failed is post-hand-off, and // mapping it back to `Intent::Remain` would duplicate the identity. if self.protocol == Protocol::V2 - && let Err(e) = epilogue(read, write).await + && let Err(e) = epilogue(read, write, &observe).await { return (outcome, Err(e.widen())); } @@ -905,7 +941,7 @@ impl Peer { acceptor: &mut link.acceptor as DynAcceptor<'a>, state: &mut link.session, when: Box::pin(when), - staged: handshake::Staged::new(), + staged: handshake::Staged::new(self.protocol), converged: None, done: false, }; @@ -1010,7 +1046,7 @@ impl Peer { // fresh staging buffer (this preamble is // consumed), and the new suppression token. drive.state.finish(); - drive.staged = handshake::Staged::new(); + drive.staged = handshake::Staged::new(drive.peer.protocol); drive.converged = Some(converged.clone()); Some(( Ok(Gossiped { @@ -1047,14 +1083,23 @@ struct Reconciliation<'a> { root: tree::Root, /// The session's erased link. link: DynLinkParts<'a>, - /// The peer's payload deserializer, applied at wire ingress. - deserializer: PayloadDeserializer, + /// The peer's payload codec: the payload boundary in both directions. + /// + /// Supplied leaves decode through it at wire ingress, and egress + /// replays only bytes it previously admitted (at send, or at an + /// earlier session's ingress). Its depth limit rides the greeting, + /// where the counterparty's must match. + codec: PayloadCodec, /// The window policy the V2 session negotiates under. window: WindowConfig, /// The V2 supply-run sizing budget; its byte target rides the greeting. run_budget: RunBudget, /// The session's stats recorder, shared by both V2 participants. stats: Recorder, + /// The session's observation handle: inert unless a handler is + /// attached and the dialect is observable, and shared, like the + /// recorder, by every layer that moves a wire item. + observe: SessionHandle, /// Whether the remote's preamble declared it a bootstrap claimant. peer_bootstrapping: bool, /// The network the remote's preamble declared. @@ -1084,10 +1129,11 @@ impl<'a> Reconciliation<'a> { let Self { root, link, - deserializer, + codec, window, run_budget, stats, + observe, peer_bootstrapping, remote_network, network, @@ -1099,9 +1145,10 @@ impl<'a> Reconciliation<'a> { .target_message_size(run_budget.bytes() as u64) .stats(stats.clone()); let carrier = Link::for_session(read, write, connector, acceptor, epoch); - let proxy = streaming_remote::Handshaking::start(Local, carrier, deserializer) + let proxy = streaming_remote::Handshaking::start(Local, carrier, codec) .window(window) - .stats(stats); + .stats(stats) + .observe(observe); let handshaken = streaming::handshake(local, proxy) .await .map_err(streaming_error)?; @@ -1134,7 +1181,7 @@ impl<'a> Reconciliation<'a> { let Self { root, link, - deserializer, + codec, peer_bootstrapping, remote_network, network, @@ -1146,7 +1193,7 @@ impl<'a> Reconciliation<'a> { let proxy = alternating_remote::Exchange::start( FrameRead::new(read), FrameWrite::new(write), - deserializer, + codec, ); let handshaken = alternating::handshake(local, proxy) .await @@ -1184,10 +1231,11 @@ impl<'a> Reconciliation<'a> { #[allow(clippy::type_complexity)] fn bootstrap_v2<'a>( link: DynLinkParts<'a>, - deserializer: PayloadDeserializer, + codec: PayloadCodec, window: WindowConfig, run_budget: RunBudget, both_bootstrapping: bool, + observe: SessionHandle, ) -> BoxFuture<'a, Result, DynWrite<'a>)>, Error>> { Box::pin(async move { let (read, write, connector, acceptor, epoch) = link; @@ -1202,8 +1250,9 @@ fn bootstrap_v2<'a>( .window(window) .target_message_size(run_budget.bytes() as u64); let carrier = Link::for_session(read, write, connector, acceptor, epoch); - let proxy = - streaming_remote::Handshaking::start(Local, carrier, deserializer).window(window); + let proxy = streaming_remote::Handshaking::start(Local, carrier, codec) + .window(window) + .observe(observe.clone()); let handshaken = streaming::handshake(local, proxy) .await .map_err(streaming_error)?; @@ -1220,7 +1269,7 @@ fn bootstrap_v2<'a>( let descent: BoxFuture<'_, _> = Box::pin(handshaken.reconcile()); let (root, (mut read, mut write)) = descent.await.map_err(streaming_error)?; if both_bootstrapping { - epilogue(&mut read, &mut write).await?; + epilogue(&mut read, &mut write, &observe).await?; return Ok(None); } Ok(Some((root.into(), read, write))) @@ -1240,7 +1289,7 @@ fn bootstrap_v2<'a>( fn bootstrap_v1<'a>( read: DynRead<'a>, write: DynWrite<'a>, - deserializer: PayloadDeserializer, + codec: PayloadCodec, both_bootstrapping: bool, ) -> BoxFuture<'a, Result, DynWrite<'a>)>, Error>> { Box::pin(async move { @@ -1248,7 +1297,7 @@ fn bootstrap_v1<'a>( let proxy = alternating_remote::Exchange::start( FrameRead::new(read), FrameWrite::new(write), - deserializer, + codec, ); let handshaken = alternating::handshake(local, proxy) .await @@ -1318,23 +1367,29 @@ fn bootstrap_claimant_is_newborn(claimed: &Version) -> Result<(), Error> { async fn epilogue( read: &mut (dyn AsyncRead + Unpin + Send + '_), write: &mut (dyn AsyncWrite + Unpin + Send + '_), + observe: &SessionHandle, ) -> Result<(), Error> { let send = async { - write.write_all(&[EPILOGUE_MARKER]).await?; - write.flush().await + write.write_all(&EPILOGUE_MARKER).await?; + write.flush().await?; + observe.control_sent(&EPILOGUE_MARKER); + Ok(()) }; let receive = async { - let mut marker = [0u8; 1]; + let mut marker = [0u8; EPILOGUE_MARKER.len()]; read.read_exact(&mut marker).await?; - if marker[0] != EPILOGUE_MARKER { + if marker != EPILOGUE_MARKER { return Err(std::io::Error::new( std::io::ErrorKind::InvalidData, format!( - "peer wrote {:#04x} where the epilogue marker belongs", - marker[0] + "peer wrote {:#04x} {:#04x} where the epilogue marker belongs", + marker[0], marker[1] ), )); } + // The validated marker is byte-equal to the local constant, so + // the constant is the received item. + observe.control_received(&EPILOGUE_MARKER); Ok(()) }; futures_util::future::try_join(send, receive) @@ -1442,6 +1497,17 @@ fn streaming_error( streaming_remote::Error, >, ) -> Error { + // The depth-limit mismatch is a configuration diagnosis, not a + // reconciliation failure: surface it as its own top-level variant. + // Only the proxy (the server side of every production handshake) + // detects it; the materialized participant has no wire. + if let tree::mirror::Error::Server(streaming_remote::Error::PayloadDepthMismatch { + local, + remote, + }) = error + { + return Error::PayloadDepthMismatch { local, remote }; + } Error::Mirror(error) } diff --git a/src/peer/gossip/tests.rs b/src/peer/gossip/tests.rs index 5cc72fed0..91019b57d 100644 --- a/src/peer/gossip/tests.rs +++ b/src/peer/gossip/tests.rs @@ -30,13 +30,14 @@ //! [`gossip_inner`]: super::Peer::gossip_inner //! [`Network::BOOTSTRAP`]: Network -use crate::message::Message; +use crate::message::{PayloadCodec, PayloadDepthLimit}; use before::Party; use futures::future::BoxFuture; use tokio::io::{duplex, split}; use super::{EPILOGUE_MARKER, alternating_error, epilogue, erase, streaming_error}; use crate::link::{Link, MemoryLink, memory}; +use crate::observe::SessionHandle; use crate::tree::mirror::{ alternating::{self, local as alternating_local, remote as alternating_remote}, framing::{FrameRead, FrameWrite}, @@ -67,9 +68,10 @@ fn concurrent_exchange_is_symmetric() { let (mut right_read, mut right_write) = split(right_io); let (left, right) = pollster::block_on(async { + let observe = SessionHandle::default(); tokio::join!( - epilogue(&mut left_read, &mut left_write), - epilogue(&mut right_read, &mut right_write), + epilogue(&mut left_read, &mut left_write, &observe), + epilogue(&mut right_read, &mut right_write, &observe), ) }); left.expect("left epilogue completes"); @@ -86,11 +88,15 @@ fn concurrent_exchange_is_symmetric() { #[test] fn marker_byte_space_is_exhaustive() { for byte in u8::MIN..=u8::MAX { - let bytes = [byte]; + let bytes = [EPILOGUE_MARKER[0], byte]; let mut reader = &bytes[..]; let mut writer = tokio::io::sink(); - let result = pollster::block_on(epilogue(&mut reader, &mut writer)); - if byte == EPILOGUE_MARKER { + let result = pollster::block_on(epilogue( + &mut reader, + &mut writer, + &SessionHandle::default(), + )); + if byte == EPILOGUE_MARKER[1] { result.expect("the marker byte completes the epilogue"); } else { let error = epilogue_error(result); @@ -113,23 +119,32 @@ fn marker_byte_space_is_exhaustive() { fn close_before_the_marker_is_a_typed_eof() { let mut reader: &[u8] = &[]; let mut writer = tokio::io::sink(); - let result = pollster::block_on(epilogue(&mut reader, &mut writer)); + let result = pollster::block_on(epilogue( + &mut reader, + &mut writer, + &SessionHandle::default(), + )); let error = epilogue_error(result); assert_eq!(error.kind(), std::io::ErrorKind::UnexpectedEof); } -/// Reading the marker consumes exactly one byte, leaving later bytes -/// untouched. +/// Reading the marker consumes exactly the marker's bytes, leaving later +/// bytes untouched. /// /// A next session's preamble may already sit behind the marker on a reused /// link; the epilogue must not slurp it. After a clean exchange the /// following bytes remain unread in the transport. #[test] fn bytes_after_the_marker_stay_untouched() { - let bytes = [EPILOGUE_MARKER, b'R', b'U']; + let bytes = [EPILOGUE_MARKER[0], EPILOGUE_MARKER[1], b'R', b'U']; let mut reader = &bytes[..]; let mut writer = tokio::io::sink(); - pollster::block_on(epilogue(&mut reader, &mut writer)).expect("the marker completes"); + pollster::block_on(epilogue( + &mut reader, + &mut writer, + &SessionHandle::default(), + )) + .expect("the marker completes"); assert_eq!(reader, b"RU", "the next session's bytes were consumed"); } @@ -147,10 +162,14 @@ fn bytes_after_the_marker_stay_untouched() { fn redacted_history_root(events: u64) -> tree::Root { let donor = Peer::::seed(); { - let mut batch = donor.batch(); - for v in 0..events { - batch.send(v); - } + donor + .batch(|batch| { + for v in 0..events { + batch.send(v)?; + } + Ok::<(), crate::message::EncodeError>(()) + }) + .expect("flat test payloads are within any depth limit"); } let versions: Vec<_> = donor .snapshot() @@ -158,10 +177,14 @@ fn redacted_history_root(events: u64) -> tree::Root { .map(|(version, _)| version.clone()) .collect(); { - let mut batch = donor.batch(); - for version in &versions { - batch.redact(version); - } + donor + .batch(|batch| { + for version in &versions { + batch.redact(version); + } + Ok::<(), crate::message::EncodeError>(()) + }) + .expect("flat test payloads are within any depth limit"); } let snapshot = donor.snapshot(); assert!(snapshot.is_empty(), "every message was redacted"); @@ -184,7 +207,7 @@ async fn claim_bootstrap_v2( root: tree::Root, ) -> Result<(Party, Tree), Error> { let (read, write, connector, acceptor, epoch) = erase(link)?; - let mut staged = handshake::Staged::new(); + let mut staged = handshake::Staged::new(Protocol::V2); handshake::preamble( Protocol::V2, Network::BOOTSTRAP, @@ -192,21 +215,25 @@ async fn claim_bootstrap_v2( &mut staged, read, write, + &SessionHandle::default(), ) .await .map_err(Error::from)?; let local_root: streaming::Root = root.into(); let local = materialized::Handshaking::start(Local, local_root); let carrier = Link::for_session(read, write, connector, acceptor, epoch); - let proxy = - streaming_remote::Handshaking::start(Local, carrier, Message::deserializer::()); + let proxy = streaming_remote::Handshaking::start( + Local, + carrier, + PayloadCodec::new::(PayloadDepthLimit::default()), + ); let handshaken = streaming::handshake(local, proxy) .await .map_err(streaming_error)?; let descent: BoxFuture<'_, _> = Box::pin(handshaken.reconcile()); let (root, (mut read, mut write)) = descent.await.map_err(streaming_error)?; - let party = party::receive(&mut read).await?; - epilogue(&mut read, &mut write).await?; + let party = party::receive(Protocol::V2, &mut read, &SessionHandle::default()).await?; + epilogue(&mut read, &mut write, &SessionHandle::default()).await?; Ok((party, Tree::from_root(root.into()))) } @@ -220,7 +247,7 @@ async fn claim_bootstrap_v1( root: tree::Root, ) -> Result<(Party, Tree), Error> { let (read, write, _connector, _acceptor, _epoch) = erase(link)?; - let mut staged = handshake::Staged::new(); + let mut staged = handshake::Staged::new(Protocol::V1); handshake::preamble( Protocol::V1, Network::BOOTSTRAP, @@ -228,6 +255,7 @@ async fn claim_bootstrap_v1( &mut staged, read, write, + &SessionHandle::default(), ) .await .map_err(Error::from)?; @@ -235,7 +263,7 @@ async fn claim_bootstrap_v1( let proxy = alternating_remote::Exchange::start( FrameRead::new(read), FrameWrite::new(write), - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), ); let handshaken = alternating::handshake(local, proxy) .await @@ -243,7 +271,7 @@ async fn claim_bootstrap_v1( let descent: BoxFuture<'_, _> = Box::pin(handshaken.reconcile()); let (root, (read, _write)) = descent.await.map_err(alternating_error)?; let mut read = read.into_inner(); - let party = party::receive(&mut read).await?; + let party = party::receive(Protocol::V1, &mut read, &SessionHandle::default()).await?; Ok((party, Tree::from_root(root))) } @@ -251,10 +279,14 @@ async fn claim_bootstrap_v1( fn provider_with(values: &[u64]) -> Peer { let provider = Peer::::seed(); { - let mut batch = provider.batch(); - for &v in values { - batch.send(v); - } + provider + .batch(|batch| { + for &v in values { + batch.send(v)?; + } + Ok::<(), crate::message::EncodeError>(()) + }) + .expect("flat test payloads are within any depth limit"); } provider } diff --git a/src/reconciliation.rs b/src/reconciliation.rs index c9d6040e7..163ec138b 100644 --- a/src/reconciliation.rs +++ b/src/reconciliation.rs @@ -13,11 +13,11 @@ //! [`Version`](crate::Version) stamped on it at send time. Nothing else //! enters the address; it rests on the invariant the protocol already //! requires everywhere — no two sends ever share a version (a replica's -//! version advances on every send, and disjoint parties can never mint the +//! version advances on every send, and disjoint parties can never produce the //! same one). Two consequences are deliberate. //! //! - **Every send is a distinct message.** Sending byte-identical content -//! twice mints two versions, hence two leaves; redacting one never +//! twice creates two versions, hence two leaves; redacting one never //! touches the other, and re-sending redacted content is a new message, //! neither resurrected nor suppressed by the redaction that came before //! it. @@ -29,7 +29,7 @@ //! [Twenty-four-byte digests](#twenty-four-byte-digests). //! //! Version reuse — the only way two messages could claim one address — -//! cannot arise: every send mints a fresh version (a tick strictly above +//! cannot arise: every send creates a fresh version (a tick strictly above //! everything the replica has ever held), and the linearity of parties //! keeps replicas' versions disjoint. Producing a reused version at all //! requires violating the linearity invariant the crate docs' safety @@ -166,7 +166,7 @@ //! content therefore contributes zero bits to any compared quantity — the //! offline content-grinding route to a collision is structurally gone, not //! merely priced. What could still contribute bits is influence over which -//! versions get minted (an actor steering gossip schedules steers the +//! versions get created (an actor steering gossip schedules steers the //! version set); against any such actor, the 24-byte width keeps the //! offline birthday floor at 2⁹⁶ evaluations, an unconditional bound that //! rests on no premise about capabilities. Hostile *peers* remain diff --git a/src/rumors.rs b/src/rumors.rs index 504fea473..544673f3b 100644 --- a/src/rumors.rs +++ b/src/rumors.rs @@ -8,6 +8,7 @@ pub use unordered::{TryNext, UnorderedMessages}; use crate::bookmark::{Bookmark, BookmarkError, NoBookmark}; use crate::link::{Acceptor, Connector, Link}; +use crate::message::EncodeError; use crate::{Batch, Error, Gossiped, Network, Peer, Snapshot, Version}; use futures::Stream; use std::sync::Arc; @@ -17,7 +18,6 @@ use tokio::{ sync::watch, }; -use serde::Serialize; /// A handle for [`send`](Rumors::send)ing and [`redact`](Rumors::redact)ing /// messages, and [`gossip`](Rumors::gossip)ing the result with peers. /// @@ -69,7 +69,8 @@ impl Clone for Rumors { run_budget: self.peer.run_budget, inner: self.peer.inner.clone(), bookmark: Arc::clone(&self.peer.bookmark), - deserializer: self.peer.deserializer, + codec: self.peer.codec, + observe: self.peer.observe.clone(), }, extant: self.extant.clone(), } @@ -131,52 +132,52 @@ impl Rumors { } } - /// Send a message. + /// Send a message, committing it immediately. /// - /// Returns a [`Batch`] that commits when dropped: a bare - /// `rumors.send(message);` commits at the end of the statement, and - /// chaining further [`send`](Batch::send)s and [`redact`](Batch::redact)s - /// accumulates them into one commit. A batch dropped by async - /// cancellation commits its queued prefix, so never hold one across an - /// `.await` in a cancellable task ([`Batch`] states the drop semantics). + /// The message is serialized and admitted here, at the call: + /// admission runs the exact decode every receiver's wire ingress + /// runs, so a payload a receiver would reject or misread is the + /// typed [`EncodeError`] instead (its variants name the causes), + /// and nothing commits. To apply several changes in one commit, + /// use [`batch`](Self::batch). /// /// `send` does not return the message's [`Version`]. Versions come back /// through observation: the observers and [`Snapshot`] attach every - /// message to the version its send minted, unique across the universe's + /// message to the version its send created, unique across the universe's /// whole history, so even byte-identical re-sends are distinct messages /// under distinct versions. [`redact`](Self::redact) states the intended - /// observe-then-redact pattern and why the write path returns nothing. + /// observe-then-redact pattern and why the write path returns no + /// version. /// /// # Observe-then-send is domination /// - /// Every message this replica observed before a batch commits is in - /// the causal past of that batch's sends, which is the supersession + /// Every message this replica observed before a commit is in the + /// causal past of that commit's sends, which is the supersession /// contract last-write-wins patterns lean on. The boundary: sends from /// different threads or different batches carry **no** guaranteed /// causal relationship to one another unless the application - /// synchronizes them itself (building a batch holds no lock, so - /// concurrent synchronization can land before the batch commits). + /// synchronizes them itself. /// /// # Panics /// - /// If `message` fails to serialize (see [`Batch::send`]). - pub fn send(&self, message: T) -> Batch<'_, T> + /// If `message` fails to serialize: a violation of the payload + /// contract ([choosing a payload + /// type](crate#choosing-a-payload-type)). + pub fn send(&self, message: T) -> Result<(), EncodeError> where - T: Serialize + Send + Sync + 'static, + T: Send + Sync + 'static, { self.peer.send(message) } - /// Redact a message: remove the live message stamped with `version` from - /// the set, here and, through gossip, everywhere. Redacting a version not - /// currently held is a no-op. + /// Redact a message: remove the live message stamped with `version` + /// from the set, here and, through gossip, everywhere, committing + /// immediately. /// - /// Returns a [`Batch`] that commits when dropped: a bare - /// `rumors.redact(&version);` commits at the end of the statement, and chaining - /// further [`send`](Batch::send)s and [`redact`](Batch::redact)s - /// accumulates them into one commit. A batch dropped by async - /// cancellation commits its queued prefix, so never hold one across an - /// `.await` in a cancellable task ([`Batch`] states the drop semantics). + /// Redacting a version not currently held is a no-op, and redaction + /// is infallible: no payload is created, so no depth admission + /// applies. To bundle redactions and sends into one commit, use + /// [`batch`](Self::batch). /// /// # Deletion is honored /// @@ -186,7 +187,7 @@ impl Rumors { /// deletions from the causal frontiers the two sides exchange. A /// message the counterparty's version shows it must already have seen, /// yet it no longer holds, was deleted there, so the holder drops its - /// own copy instead of transmitting it. And because every send mints a + /// own copy instead of transmitting it. And because every send creates a /// fresh version, re-sending byte-identical content after a redaction /// is a *new* message: no resurrection, no suppression. For the same /// reason, two identical sends are two messages, and redacting one @@ -204,41 +205,69 @@ impl Rumors { /// anyway: a batch inserts all its messages at once, so sends are not /// 1:1 with insertions and a message's version is not knowable until /// insertion. - pub fn redact(&self, version: &Version) -> Batch<'_, T> + pub fn redact(&self, version: &Version) where T: Send + Sync, { self.peer.redact(version) } - /// Start an empty [`Batch`], for applying several changes in one - /// commit: observers and concurrent gossip sessions see all of them - /// land at once, in at most one observer wakeup. - /// - /// A batch is a performance optimization, not an atomicity guarantee: - /// it coalesces its actions into one tree traversal and one commit, - /// but, outside of a panic, whatever the batch holds when it drops is - /// committed automatically. Do not rely on batch atomicity for - /// correctness, *especially* in the presence of async cancellation. + /// Apply several changes in one all-or-nothing commit. + /// + /// Runs `f` with a [`Batch`] scope handle for queueing + /// [`send`](Batch::send)s and [`redact`](Batch::redact)s, and + /// commits everything queued **iff `f` returns `Ok`**: observers + /// and concurrent gossip sessions see it all land as one commit, + /// one tree traversal, and at most one observer wakeup. Any other + /// exit — a returned `Err` (how a caller abandons a batch) or a + /// panic — commits nothing. + /// + /// The closure is synchronous and the scope handle cannot leave it + /// (the examples below show both escape routes failing to compile), + /// so async cancellation cannot observe a half-built batch. The + /// closure may use the same `Rumors` handle — a + /// [`send`](Self::send), or a nested `batch` — and such nested + /// operations commit first, as their own separate commits. /// /// # Examples /// /// ``` - /// use rumors::Peer; + /// use rumors::{EncodeError, Peer}; /// /// let rumors = Peer::::seed().into_rumors(); - /// rumors - /// .batch() - /// .send("a".to_string()) - /// .send("b".to_string()); - /// // The batch committed, as one commit, when the statement ended. + /// rumors.batch(|batch| { + /// batch.send("a".to_string())?; + /// batch.send("b".to_string())?; + /// Ok::<(), EncodeError>(()) + /// })?; + /// // Both landed, in one commit. /// assert_eq!(rumors.snapshot().len(), 2); + /// # Ok::<(), EncodeError>(()) + /// ``` + /// + /// The scope handle cannot be stashed outside the closure: + /// + /// ```compile_fail + /// let rumors = rumors::Peer::::seed().into_rumors(); + /// let mut stash = None; + /// let _ = rumors.batch::<_, (), _>(|batch| { + /// stash = Some(batch); + /// Ok(()) + /// }); + /// ``` + /// + /// ...and cannot be returned out of it: + /// + /// ```compile_fail + /// let rumors = rumors::Peer::::seed().into_rumors(); + /// let escaped = rumors.batch::<_, (), _>(|batch| Ok(batch)); /// ``` - pub fn batch(&self) -> Batch<'_, T> + pub fn batch(&self, f: F) -> Result where T: Send + Sync, + F: for<'s> FnOnce(&'s mut Batch<'_, T>) -> Result, { - self.peer.batch() + self.peer.batch(f) } /// The identifier shared by every peer that descends from the same @@ -487,7 +516,7 @@ impl Rumors { /// // A long-lived link between them, one driver per end. /// let (mut alice_side, mut bob_side) = rumors::link::memory(); /// - /// alice.send("psst".to_string()); + /// alice.send("psst".to_string())?; /// /// let mut alice_drive = alice.gossip_when(alice.changes(), &mut alice_side); /// let mut bob_drive = bob.gossip_when(bob.changes(), &mut bob_side); @@ -498,9 +527,9 @@ impl Rumors { /// pushed.expect("driver running")?; /// served.expect("driver running")?; /// assert_eq!(bob.snapshot().len(), 1); - /// # Ok::<(), rumors::Error>(()) + /// # Ok::<(), Box>(()) /// # })?; - /// # Ok::<(), rumors::Error>(()) + /// # Ok::<(), Box>(()) /// ``` #[must_use = "the driver does nothing until the returned stream is polled"] pub fn gossip_when<'a, CR, CW, C, A, S>( diff --git a/src/tags.rs b/src/tags.rs new file mode 100644 index 000000000..64812e91b --- /dev/null +++ b/src/tags.rs @@ -0,0 +1,37 @@ +//! CBOR tag numbers identifying the crate's opaque atoms on its +//! serialized surfaces. +//! +//! The wire protocol and the stored bookmark spell party and version +//! atoms as CBOR byte strings wrapping their canonical bit-level +//! codings. Each such byte string is preceded by one of the tags +//! below, so the atom's identity travels with it: a generic CBOR tool +//! holding nothing but this table can pick the atoms out of a capture, +//! a bookmark, or a pasted snippet, with no knowledge of where in the +//! protocol they appeared. +//! +//! Two rules govern the tags: +//! +//! - **They are protocol vocabulary, written and read only by the +//! transport and bookmark codecs.** The serde implementations of the +//! underlying types stay untagged and format-agnostic: an +//! application payload containing a version serializes identically +//! to JSON, CBOR, or any other backend, and never carries a +//! CBOR-specific concept. +//! - **The numbers are provisional, pending IANA registration.** They +//! are drawn from the first-come-first-served range (32768 and up +//! per RFC 8949 §9.2) and based at `0xD255` — the ASCII bytes `RU` +//! with the range's high bit set — in currently unassigned space. +//! Should registration assign a different block, the constants here +//! move in a deliberate, versioned format change; nothing else in +//! the crate hard-codes them. + +/// Tags a byte string holding a party atom's canonical encoding. +pub const PARTY_TAG: u64 = 0xD255; + +/// Tags a byte string holding a version atom's canonical encoding. +pub const VERSION_TAG: u64 = 0xD256; + +/// Tags a byte string holding a clock's canonical encoding: a party +/// atom's bytes immediately followed by a version atom's bytes, as the +/// bookmark stores them. +pub const CLOCK_TAG: u64 = 0xD257; diff --git a/src/testing.rs b/src/testing.rs index 048724d7a..bd1335a63 100644 --- a/src/testing.rs +++ b/src/testing.rs @@ -11,11 +11,23 @@ pub use transport::{ wrap_link, }; -pub use crate::tree::mirror::streaming::remote::LinkCapture; +pub use crate::tree::mirror::streaming::remote::{HookCapture, HookStream, LinkCapture}; -/// Render captured V2 link traffic grouped by labeled logical streams. -pub fn render_v2_capture(a: &LinkCapture, b: &LinkCapture) -> String { - crate::tree::mirror::streaming::remote::render_v2_capture(a, b) +/// Render two hook captures grouped by labeled logical streams. +pub fn render_hook_capture(a: &HookCapture, b: &HookCapture) -> String { + crate::tree::mirror::streaming::remote::render_hook_capture(a, b) +} + +/// Assert the concatenation of observed items reproduces `wire` exactly: +/// the totality witness behind rendering hook items as a wire-byte pin. +pub fn assert_items_account_for(items: &[Vec], wire: &[u8]) { + crate::tree::mirror::streaming::remote::assert_items_account_for(items, wire); +} + +/// Parse one data stream's on-wire open label, returning +/// `((epoch, index), label byte length)`. +pub fn stream_label(bytes: &[u8]) -> ((u8, u8), usize) { + crate::tree::mirror::streaming::remote::stream_label(bytes) } /// A snapshot of the crate-wide census of live tree-node handles. @@ -103,20 +115,24 @@ pub fn frame_payload_chunk_len() -> usize { crate::tree::mirror::framing::PAYLOAD_CHUNK_LEN } -/// Bytes of the length header ahead of each leaf record in a supply run. +/// The wire prefix of one streaming-codec supply frame declaring a +/// `declared`-byte run. /// -/// Exposed so the allocator meter (`tests/decode_alloc.rs`) builds run -/// bodies from the wire's own width rather than a transcribed copy. -pub fn run_record_header_len() -> usize { - crate::tree::mirror::framing::LENGTH_HEADER_LEN +/// Prepend it to a run body to hand [`decode_supply_frame`] a decodable +/// byte stream; the prefix is built by the codec's own head writers, so +/// the meter cannot drift from the wire. +pub fn supply_frame_head(declared: usize) -> Vec { + crate::tree::mirror::streaming::remote::supply_frame_head(declared) } -/// The signal byte opening one streaming-codec supply frame. +/// A structurally valid lone-record run of exactly `len` bytes, with +/// arbitrary record content. /// -/// Prepend it to a length-headed supply body to hand -/// [`decode_supply_frame`] a decodable byte stream. -pub fn supply_signal_byte() -> u8 { - crate::tree::mirror::streaming::remote::supply_signal_byte() +/// Exposed so the allocator meter (`tests/decode_alloc.rs`) builds run +/// bodies from the wire's own record heads rather than a transcribed +/// copy. +pub fn lone_record_run(len: usize) -> Vec { + crate::tree::mirror::streaming::remote::lone_record_run(len) } /// Decode one streaming-codec supply frame, discarding the decoded run. diff --git a/src/testing/transport.rs b/src/testing/transport.rs index b24799a8e..6c7275f7b 100644 --- a/src/testing/transport.rs +++ b/src/testing/transport.rs @@ -230,7 +230,7 @@ impl State { } } - /// Record the read fault as injected and mint its error. + /// Record the read fault as injected and produce its error. fn inject_read(&mut self) -> io::Error { let fault = self.plan.fault.expect("an armed fault is configured"); let injected = InjectedIo { diff --git a/src/tests.rs b/src/tests.rs index 6b2419996..d4b29e626 100644 --- a/src/tests.rs +++ b/src/tests.rs @@ -5,6 +5,7 @@ //! [`Party`] and compare it to [`Party::seed`]. Both require in-crate access, //! so they live here rather than in `tests/`. +use crate::message::{PayloadCodec, PayloadDepthLimit}; use std::pin::Pin; use std::task::{Context, Poll}; @@ -16,6 +17,7 @@ use tokio::sync::{Mutex, watch}; use crate::bookmark::{Bookmarked, NoBookmark}; use crate::link::{Connector, Link, MemoryAcceptor, MemoryConnector, MemoryLink, memory}; +use crate::observe::Attachment; use crate::testing::{Quiescence, run_to_quiescence}; use crate::tree::{Root, Tree}; use crate::{Error, Inner, Peer, Retire}; @@ -23,15 +25,17 @@ use crate::{Error, Inner, Peer, Retire}; /// The preamble's wire length: magic(6) + proto_version(2) + network(16) + /// intent(1). The fault-injection budgets /// below land cuts on exact protocol boundaries relative to this. -const PREAMBLE_LEN: usize = 25; +const PREAMBLE_LEN: usize = crate::tree::mirror::handshake::V2_PREAMBLE_LEN; /// Insert each of `vals` into `k` as one committed batch. fn with_messages(k: Peer, vals: &[u64]) -> Peer { - let mut batch = k.batch(); - for &v in vals { - batch.send(v); - } - drop(batch); + k.batch(|batch| { + for &v in vals { + batch.send(v)?; + } + Ok::<(), crate::message::EncodeError>(()) + }) + .expect("flat test payloads are within any depth limit"); k } @@ -95,7 +99,7 @@ fn overlapping_retiree_party_is_rejected() { // region (not a disjoint fork), with an empty tree so its version equals the // survivor's and the survivor takes the absorb branch. let forged = Peer:: { - deserializer: crate::message::Message::deserializer::(), + codec: PayloadCodec::new::(PayloadDepthLimit::default()), network: survivor.network, protocol: survivor.protocol, window: survivor.window, @@ -105,6 +109,7 @@ fn overlapping_retiree_party_is_rejected() { tree: Tree::from_root(Root::default()), }), bookmark: Arc::new(Mutex::new(Bookmarked::new(NoBookmark))), + observe: Attachment::default(), }; // Each side's future owns its link: the absorber rejects the overlap @@ -140,7 +145,7 @@ fn overlapping_retiree_party_is_rejected() { #[test] fn retiring_all_forks_reconstitutes_the_seed_party() { let survivor = Peer::::seed(); - // Each child is a genuine party-disjoint fork, minted by serving a bootstrap. + // Each child is a genuine party-disjoint fork, created by serving a bootstrap. // All are empty, so they share the seed's version, are reflexively dominated, // and retire with no prior gossip. let (survivor, c1) = bootstrap_from(survivor); @@ -158,8 +163,8 @@ fn retiring_all_forks_reconstitutes_the_seed_party() { ); } -/// Bootstrap mints a fresh party by forking the provider's; retiring that peer -/// back must reclaim exactly that minted region. +/// Bootstrap creates a fresh party by forking the provider's; retiring that +/// peer back must reclaim exactly that forked region. /// /// Provider with real content, bootstrap (a wire fork), then retire the /// newcomer home: the provider's party normalizes back to [`Party::seed`], @@ -264,27 +269,32 @@ impl AsyncWrite for Fuse { /// frame plus the root-fan listing frame — so a [`Fuse`] budget can land on /// an exact protocol boundary. fn greeting_frame_len(retiree: &Peer) -> usize { - use crate::tree::mirror::streaming::{self, Local, materialized}; + use crate::tree::mirror::streaming::{self, Local, materialized, message::Greeting}; let root: streaming::Root = retiree.inner.borrow().tree.clone().root.into(); - let fan = pollster::block_on(materialized::greeting_fan(&Local, root.root)) + let fan = pollster::block_on(materialized::greeting_fan(&Local, root.root.clone())) .unwrap_or_else(|never| match never {}); - // The listing frame is raw radix-hash records: one byte plus a Merkle - // hash per child, the frame length carrying the count. - let listing_len = - materialized::fan_listing(&fan).len() * (1 + crate::tree::typed::hash::MERKLE_HASH_LEN); - crate::tree::mirror::framing::LENGTH_HEADER_LEN - + crate::tree::mirror::framing::GREETING_SIZE_WORDS_LEN - + retiree.snapshot().latest().as_bytes().len() - + crate::tree::mirror::framing::LENGTH_HEADER_LEN - + listing_len + // Reassemble the exact greeting the session sends — the same field + // sources the handshake draws from — and measure its one wire item. + let greeting = Greeting { + version: retiree.snapshot().latest().clone(), + set_len: root.len(), + max_version_bytes: root.max_version_bytes(), + payload_depth_limit: retiree.codec.limit().get(), + target_message_size: retiree.run_budget.bytes() as u64, + listing: materialized::fan_listing(&fan), + }; + crate::tree::mirror::streaming::remote::codec::greeting::encode_greeting(&greeting).len() } /// The wire length of `retiree`'s trailing party frame, so a [`Fuse`] budget /// can land on an exact protocol boundary. fn party_frame_len(retiree: &Peer) -> usize { - // The party frame's body is the canonical party encoding, bare. - crate::tree::mirror::framing::LENGTH_HEADER_LEN + party_of(retiree).as_bytes().len() + // The hand-off is the party-atom tag wrapping a byte string of the + // canonical party encoding. + use crate::tree::mirror::cbor::head_len; + let party = party_of(retiree).as_bytes().len(); + head_len(crate::tags::PARTY_TAG) + head_len(party as u64) + party } /// A connector whose opened streams draw on the link's shared fuse budget. @@ -641,9 +651,8 @@ fn root_hash_read_meter_is_live() { fn batch_commit_root_hash_reads() { let peer = with_messages(Peer::::seed(), &[1, 2]); let before = crate::tree::meter::root_hash_reads(); - let mut batch = peer.batch(); - batch.send(3); - drop(batch); + peer.batch(|batch| batch.send(3)) + .expect("a flat payload is within any depth limit"); assert_eq!( crate::tree::meter::root_hash_reads() - before, 0, diff --git a/src/tree.rs b/src/tree.rs index 64f044e80..8f6c91fe2 100644 --- a/src/tree.rs +++ b/src/tree.rs @@ -385,6 +385,9 @@ impl Tree { /// an earlier insert in the same batch overrides that insert (last /// action on a path wins). /// + /// An empty batch is a complete no-op: nothing ticks, the tree is + /// unchanged, and the returned flag is `false`. + /// /// A batch is applied to the tree in a single traversal, which is more /// efficient than applying its actions one at a time: in theory an /// O(log n) speedup over one-by-one insertion, in practice about 2-3x @@ -428,8 +431,7 @@ impl Tree { // version strictly greater than any prior insert at this party. The // strict tick on forgets is required by the mirror protocol's // deletion-honoring inference, which cannot distinguish "forgot it" - // from "never had it" when versions are equal. An empty batch is a - // complete no-op. + // from "never had it" when versions are equal. // The running version, advanced in place per action; each action // clones the post-tick value as the committed version that keys // its leaf. The reactions flow into `react` lazily; the whole diff --git a/src/tree/arb.rs b/src/tree/arb.rs index ac27d5a2d..40ce377ad 100644 --- a/src/tree/arb.rs +++ b/src/tree/arb.rs @@ -13,7 +13,7 @@ use crate::{Version, message::Message}; /// Distinct indices yield mutually *disjoint* parties, so versions ticked on /// different indices are causally concurrent — the test analogue of "different /// peers with independent histories". Because the chain is fully determined by -/// the index, independent proptest strategies can each mint the same disjoint +/// the index, independent proptest strategies can each derive the same disjoint /// parties without sharing any state, which is what lets two separately /// generated trees (e.g. `arb_tree_root(0, …)` and `arb_tree_root(1, …)`) end /// up with incomparable root versions. diff --git a/src/tree/mirror.rs b/src/tree/mirror.rs index ed9ca0734..f93f6dd6b 100644 --- a/src/tree/mirror.rs +++ b/src/tree/mirror.rs @@ -25,6 +25,7 @@ pub mod streaming; #[cfg(test)] mod tests; +pub(crate) mod cbor; pub(crate) mod framing; pub(crate) mod handshake; pub(crate) mod party; diff --git a/src/tree/mirror/alternating/backend/remote.rs b/src/tree/mirror/alternating/backend/remote.rs index 956c7bbdd..ee9f78c9a 100644 --- a/src/tree/mirror/alternating/backend/remote.rs +++ b/src/tree/mirror/alternating/backend/remote.rs @@ -50,7 +50,7 @@ use std::marker::PhantomData; use tokio::io::{AsyncRead, AsyncWrite}; -use crate::message::PayloadDeserializer; +use crate::message::PayloadCodec; use crate::tree::wire; use crate::Error; @@ -81,9 +81,9 @@ pub struct Connected; pub struct Exchange { reader: FrameRead, writer: FrameWrite, - /// The peer's payload deserializer: every `providing` channel this + /// The peer's payload codec: every `providing` channel this /// proxy decodes builds its leaf payloads through it. - deserializer: PayloadDeserializer, + codec: PayloadCodec, _phantom: PhantomData (V, H)>, } @@ -91,17 +91,13 @@ impl Exchange { /// Begin an [`Exchange`] on transport halves wrapped after the shared raw /// preamble has completed. /// - /// `deserializer` is the peer's payload deserializer, the typed + /// `codec` is the peer's payload codec, the typed /// ingress for every leaf this session decodes. - pub fn start( - reader: FrameRead, - writer: FrameWrite, - deserializer: PayloadDeserializer, - ) -> Self { + pub fn start(reader: FrameRead, writer: FrameWrite, codec: PayloadCodec) -> Self { Self { reader, writer, - deserializer, + codec, _phantom: PhantomData, } } @@ -110,15 +106,11 @@ impl Exchange { impl Exchange { /// Construct a [`Connected`]-state [`Exchange`] from already-framed /// reader/writer halves, threading them through from a predecessor stage. - fn connected( - reader: FrameRead, - writer: FrameWrite, - deserializer: PayloadDeserializer, - ) -> Self { + fn connected(reader: FrameRead, writer: FrameWrite, codec: PayloadCodec) -> Self { Self { reader, writer, - deserializer, + codec, _phantom: PhantomData, } } @@ -180,10 +172,10 @@ where /// [`recv_msg`] for the payload-bearing messages: the frame decodes /// through [`message::DecodeWith`], its leaf payloads through the peer's -/// deserializer. +/// codec. pub(super) async fn recv_msg_with( reader: &mut FrameRead, - deserializer: PayloadDeserializer, + codec: PayloadCodec, ) -> Result where R: AsyncRead + Unpin + Send, @@ -201,7 +193,7 @@ where }) .map_err(Error::Io)?; let mut slice = frame.as_slice(); - let msg = M::read_wire_with(&mut slice, deserializer).map_err(Error::Io)?; + let msg = M::read_wire_with(&mut slice, codec).map_err(Error::Io)?; if !slice.is_empty() { return Err(Error::Io(wire::invalid(format!( "{} trailing bytes after the decoded message", @@ -249,7 +241,7 @@ where Ok(protocol::Step::Continue { msg: peer, - next: Exchange::connected(self.reader, self.writer, self.deserializer), + next: Exchange::connected(self.reader, self.writer, self.codec), }) } } @@ -270,7 +262,7 @@ where // is `Infallible`, so `Done` is uninhabitable here. Ok(Step::Continue { msg, - next: Exchange::connected(self.reader, self.writer, self.deserializer), + next: Exchange::connected(self.reader, self.writer, self.codec), }) } } @@ -297,7 +289,7 @@ where let response: message::Opening = recv_msg(&mut self.reader).await?; Ok(Step::Continue { msg: response, - next: Exchange::connected(self.reader, self.writer, self.deserializer), + next: Exchange::connected(self.reader, self.writer, self.codec), }) } } @@ -320,7 +312,7 @@ where // counterparty to send back a non-trivial `providing` (the "we have, // they lack" Left case when we are the empty side). let response: message::Exchange = - recv_msg_with(&mut self.reader, self.deserializer).await?; + recv_msg_with(&mut self.reader, self.codec).await?; if response.requested.is_empty() && response.uncertain.is_empty() { Ok(Step::Done { @@ -330,7 +322,7 @@ where } else { Ok(Step::Continue { msg: response, - next: Exchange::connected(self.reader, self.writer, self.deserializer), + next: Exchange::connected(self.reader, self.writer, self.codec), }) } } @@ -368,8 +360,7 @@ where }); } - let response: message::Exchange = - recv_msg_with(&mut self.reader, self.deserializer).await?; + let response: message::Exchange = recv_msg_with(&mut self.reader, self.codec).await?; if response.requested.is_empty() && response.uncertain.is_empty() { Ok(Step::Done { @@ -379,7 +370,7 @@ where } else { Ok(Step::Continue { msg: response, - next: Exchange::connected(self.reader, self.writer, self.deserializer), + next: Exchange::connected(self.reader, self.writer, self.codec), }) } } @@ -409,7 +400,7 @@ where }); } - let response: message::Closing = recv_msg_with(&mut self.reader, self.deserializer).await?; + let response: message::Closing = recv_msg_with(&mut self.reader, self.codec).await?; if response.requested.is_empty() { Ok(Step::Done { @@ -419,7 +410,7 @@ where } else { Ok(Step::Continue { msg: response, - next: Exchange::connected(self.reader, self.writer, self.deserializer), + next: Exchange::connected(self.reader, self.writer, self.codec), }) } } @@ -447,8 +438,7 @@ where }); } - let response: message::Complete = - recv_msg_with(&mut self.reader, self.deserializer).await?; + let response: message::Complete = recv_msg_with(&mut self.reader, self.codec).await?; // `CompleteInitiator` is statically `Done`: the `Next` slot is // `Infallible`, so `Continue` is uninhabitable here. diff --git a/src/tree/mirror/alternating/backend/remote/tests.rs b/src/tree/mirror/alternating/backend/remote/tests.rs index b4320081c..a7cb184e9 100644 --- a/src/tree/mirror/alternating/backend/remote/tests.rs +++ b/src/tree/mirror/alternating/backend/remote/tests.rs @@ -11,7 +11,7 @@ //! `alternating/message/tests.rs`; the exact wire bytes in //! `alternating/wire_snapshot.rs`. -use crate::message::Message; +use crate::message::{PayloadCodec, PayloadDepthLimit}; use proptest::collection::vec; use proptest::prelude::*; @@ -40,11 +40,15 @@ fn recv(bytes: &[u8]) -> Result { } /// [`recv`] for the payload-bearing messages, through the production -/// deserializer-parameterized ingress with a unit-payload deserializer. +/// codec-parameterized ingress with a unit-payload codec. fn recv_with(bytes: &[u8]) -> Result { pollster::block_on(async { let mut reader = FrameRead::new(bytes); - recv_msg_with::(&mut reader, Message::deserializer::<()>()).await + recv_msg_with::( + &mut reader, + PayloadCodec::new::<()>(PayloadDepthLimit::default()), + ) + .await }) } diff --git a/src/tree/mirror/alternating/message.rs b/src/tree/mirror/alternating/message.rs index 34bb07737..346b75b86 100644 --- a/src/tree/mirror/alternating/message.rs +++ b/src/tree/mirror/alternating/message.rs @@ -72,17 +72,17 @@ //! the wire: the protocol's height schedule names the type each side expects //! next. -use crate::message::PayloadDeserializer; +use crate::message::PayloadCodec; use crate::tree::wire::{self, Decode, Encode}; /// Wire decode for the payload-bearing protocol messages: parses each -/// `providing` channel's leaf payloads through the peer's deserializer +/// `providing` channel's leaf payloads through the peer's codec /// (see [`read_providing`]); the payload-free messages stay on the plain /// [`Decode`]. pub trait DecodeWith: Sized { fn read_wire_with( reader: &mut R, - deserializer: PayloadDeserializer, + codec: PayloadCodec, ) -> std::io::Result; } @@ -103,11 +103,8 @@ pub type Providing = Vec<(Prefix, Node)>; /// Decode one `providing` channel: a `u32` count, then `(prefix, node)` /// pairs, the nodes' leaf payloads parsed through the peer's -/// deserializer (the protocol's typed ingress; see [`DecodeNode`]). -fn read_providing( - reader: &mut R, - deserializer: PayloadDeserializer, -) -> std::io::Result> +/// codec (the protocol's typed ingress; see [`DecodeNode`]). +fn read_providing(reader: &mut R, codec: PayloadCodec) -> std::io::Result> where H: DecodeNode, R: std::io::Read, @@ -118,7 +115,7 @@ where let mut items = Vec::new(); for _ in 0..count { let prefix = Prefix::::read_wire(reader)?; - let node = H::read_node(reader, deserializer)?; + let node = H::read_node(reader, codec)?; items.push((prefix, node)); } Ok(items) @@ -261,9 +258,9 @@ where { fn read_wire_with( reader: &mut R, - deserializer: PayloadDeserializer, + codec: PayloadCodec, ) -> std::io::Result { - let providing: Providing> = read_providing(reader, deserializer)?; + let providing: Providing> = read_providing(reader, codec)?; verify_pairs_canonical(&providing, "Exchange.providing")?; let requested: Vec>> = Decode::read_wire(reader)?; verify_keys_canonical(&requested, "Exchange.requested")?; @@ -342,9 +339,9 @@ impl Encode for Closing { impl DecodeWith for Closing { fn read_wire_with( reader: &mut R, - deserializer: PayloadDeserializer, + codec: PayloadCodec, ) -> std::io::Result { - let providing: Providing = read_providing(reader, deserializer)?; + let providing: Providing = read_providing(reader, codec)?; verify_pairs_canonical(&providing, "Closing.providing")?; let requested: Vec> = Decode::read_wire(reader)?; verify_keys_canonical(&requested, "Closing.requested")?; @@ -379,9 +376,9 @@ impl Encode for Complete { impl DecodeWith for Complete { fn read_wire_with( reader: &mut R, - deserializer: PayloadDeserializer, + codec: PayloadCodec, ) -> std::io::Result { - let providing: Providing = read_providing(reader, deserializer)?; + let providing: Providing = read_providing(reader, codec)?; verify_pairs_canonical(&providing, "Complete.providing")?; Ok(Self { providing }) } diff --git a/src/tree/mirror/alternating/message/tests.rs b/src/tree/mirror/alternating/message/tests.rs index 9e62c4571..4332632a3 100644 --- a/src/tree/mirror/alternating/message/tests.rs +++ b/src/tree/mirror/alternating/message/tests.rs @@ -9,6 +9,7 @@ //! build nodes via [`arb_root_node`] / [`arb_s_z_node`] / [`arb_leaf`]. The //! exact on-wire bytes are pinned by `mirror::alternating::wire_snapshot`. +use crate::message::{PayloadCodec, PayloadDepthLimit}; use std::collections::{BTreeMap, BTreeSet}; use proptest::collection::vec; @@ -185,10 +186,13 @@ proptest! { } /// Decode one payload-bearing message from an exact slice through the -/// unit-payload deserializer, rejecting trailing bytes. +/// unit-payload codec, rejecting trailing bytes. fn from_slice_with(bytes: &[u8]) -> std::io::Result { let mut input = bytes; - let m = M::read_wire_with(&mut input, Message::deserializer::<()>())?; + let m = M::read_wire_with( + &mut input, + PayloadCodec::new::<()>(PayloadDepthLimit::default()), + )?; if !input.is_empty() { return Err(std::io::Error::new( std::io::ErrorKind::InvalidData, @@ -202,7 +206,10 @@ fn from_slice_with(bytes: &[u8]) -> std::io::Result { /// trailing bytes: the test-side door to [`DecodeNode`], with unit payloads. fn node_from_slice(bytes: &[u8]) -> std::io::Result> { let mut input = bytes; - let node = H::read_node(&mut input, Message::deserializer::<()>())?; + let node = H::read_node( + &mut input, + PayloadCodec::new::<()>(PayloadDepthLimit::default()), + )?; if !input.is_empty() { return Err(std::io::Error::new( std::io::ErrorKind::InvalidData, diff --git a/src/tree/mirror/alternating/tests.rs b/src/tree/mirror/alternating/tests.rs index 22e5f6f30..961930dda 100644 --- a/src/tree/mirror/alternating/tests.rs +++ b/src/tree/mirror/alternating/tests.rs @@ -1,3 +1,4 @@ +use crate::message::{PayloadCodec, PayloadDepthLimit}; use std::cell::OnceCell; use std::future::Future; use std::pin::Pin; @@ -8,6 +9,7 @@ use proptest::prelude::*; use tokio::runtime::Runtime; use crate::Network; +use crate::observe::SessionHandle; use crate::tree::arb::{ arb_tree_root, leaf_parent_dispute_pair, leaf_parent_redaction_pair, nth_party, uncontained_supply_pair, @@ -104,7 +106,7 @@ fn mirror_via(a: crate::tree::Root, b: crate::tree::Root, scenario: Scenario) -> let remote_b = remote::Exchange::start( FrameRead::new(a_r), FrameWrite::new(a_w), - Message::deserializer::<()>(), + PayloadCodec::new::<()>(PayloadDepthLimit::default()), ); let client = mirror(local_a, remote_b); @@ -112,7 +114,7 @@ fn mirror_via(a: crate::tree::Root, b: crate::tree::Root, scenario: Scenario) -> let remote_a = remote::Exchange::start( FrameRead::new(b_r), FrameWrite::new(b_w), - Message::deserializer::<()>(), + PayloadCodec::new::<()>(PayloadDepthLimit::default()), ); let server = mirror(local_b, remote_a); @@ -319,7 +321,7 @@ fn uncontained_supply_is_rejected() { let remote_b = remote::Exchange::start( FrameRead::new(a_r), FrameWrite::new(a_w), - Message::deserializer::<()>(), + PayloadCodec::new::<()>(PayloadDepthLimit::default()), ); let receiver_side = mirror(local_receiver, remote_b); @@ -327,7 +329,7 @@ fn uncontained_supply_is_rejected() { let remote_a = remote::Exchange::start( FrameRead::new(b_r), FrameWrite::new(b_w), - Message::deserializer::<()>(), + PayloadCodec::new::<()>(PayloadDepthLimit::default()), ); let poisoned_side = mirror(local_poisoned, remote_a); @@ -442,12 +444,13 @@ fn handshake_flushes_over_buffering_transport() { let mut b_r = b_r; let mut a_w = HoldUntilFlush::new(a_w); let mut b_w = HoldUntilFlush::new(b_w); - let mut a_staged = handshake::Staged::new(); - let mut b_staged = handshake::Staged::new(); + let mut a_staged = handshake::Staged::new(crate::Protocol::V1); + let mut b_staged = handshake::Staged::new(crate::Protocol::V1); // The preamble carries only magic + version + network + intent, so // this exercises purely the flush/deadlock behavior of the framed // greeting exchange. + let observe = SessionHandle::default(); let (ra, rb) = tokio::join!( handshake::preamble( crate::Protocol::V1, @@ -455,7 +458,8 @@ fn handshake_flushes_over_buffering_transport() { Intent::Remain, &mut a_staged, &mut a_r, - &mut a_w + &mut a_w, + &observe ), handshake::preamble( crate::Protocol::V1, @@ -463,7 +467,8 @@ fn handshake_flushes_over_buffering_transport() { Intent::Remain, &mut b_staged, &mut b_r, - &mut b_w + &mut b_w, + &observe ), ); ra.is_ok() && rb.is_ok() diff --git a/src/tree/mirror/cbor.rs b/src/tree/mirror/cbor.rs new file mode 100644 index 000000000..9798c23de --- /dev/null +++ b/src/tree/mirror/cbor.rs @@ -0,0 +1,249 @@ +//! Canonical CBOR head primitives shared by the hand-written wire codecs. +//! +//! The V2 wire and the session surfaces around it (preamble, greeting, +//! party hand-off, stream labels, epilogue) spell every structure as +//! deterministic-encoding CBOR: shortest-form heads everywhere, definite +//! lengths only, one spelling per value. That contract is what keeps the +//! byte-pinning snapshot discipline meaningful — a value has exactly one +//! encoding, so a snapshot pins semantics, not an encoder's whim — and it +//! is enforced on ingress: [`read_head`] and [`read_head_async`] reject a +//! head that is indefinite, reserved, or wider than its value requires. +//! +//! Hand-written primitives, not a general CBOR reader, because the wire +//! needs the head as its unit of work: a codec here parses one head +//! incrementally against the transport, learns the item's declared +//! length, and skips or prices what follows in O(1) without +//! materializing it — and these hand-parsed positions are exactly where +//! the non-shortest, indefinite, and reserved spellings above are +//! rejected. A general CBOR reader offers no head-level incremental +//! API: it decodes whole items into values and buffers past item +//! boundaries, so neither the streaming boundary nor the spelling +//! enforcement can live there. +//! +//! This module owns only the *head* grammar (RFC 8949 §3: the initial +//! byte's major type and its argument). What follows a head — payload +//! bytes, nested items, tag content — belongs to the codec reading it; +//! each codec validates the majors and values it expects and prices its +//! own lengths. Writers here emit exactly what the readers accept, and +//! the round-trip property tests in this module hold the two together. + +use tokio::io::{AsyncRead, AsyncReadExt}; + +/// Major type of an unsigned integer item. +pub(crate) const MAJOR_UINT: u8 = 0; + +/// Major type of a definite-length byte string. +pub(crate) const MAJOR_BSTR: u8 = 2; + +/// Major type of a definite-length text string. +pub(crate) const MAJOR_TEXT: u8 = 3; + +/// Major type of a definite-length array. +pub(crate) const MAJOR_ARRAY: u8 = 4; + +/// Major type of a definite-length map. +pub(crate) const MAJOR_MAP: u8 = 5; + +/// Major type of a tag. +pub(crate) const MAJOR_TAG: u8 = 6; + +/// Tag number for an embedded CBOR sequence in a byte string (RFC 9277). +pub(crate) const TAG_CBOR_SEQUENCE: u64 = 63; + +/// Tag number for an embedded CBOR data item in a byte string (RFC 8949). +pub(crate) const TAG_EMBEDDED_ITEM: u64 = 24; + +/// Tag number for self-described CBOR (RFC 8949 §3.4.6): CBOR's own +/// magic, opening the V2 preamble and the stored bookmark. +/// +/// The production writers pin the tag's rendered bytes inside their prefix +/// literals (the V2 preamble's prefix, the bookmark's opening bytes), each +/// held to this constant by a committed pin test; the constant itself is +/// consumed only by the test-gated capture renderer, so it carries the +/// same gate. +#[cfg(any(test, feature = "test-internals"))] +pub(crate) const TAG_SELF_DESCRIBED: u64 = 55799; + +/// Bytes the shortest-form head for `value` occupies, any major type. +pub(crate) const fn head_len(value: u64) -> usize { + match value { + 0..=23 => 1, + 24..=0xff => 2, + 0x100..=0xffff => 3, + 0x1_0000..=0xffff_ffff => 5, + _ => 9, + } +} + +/// Append the shortest-form head `(major, value)` to `out`. +pub(crate) fn write_head(out: &mut Vec, major: u8, value: u64) { + debug_assert!(major < 8, "a CBOR major type is three bits"); + let major = major << 5; + match value { + 0..=23 => out.push(major | value as u8), + 24..=0xff => out.extend_from_slice(&[major | 24, value as u8]), + 0x100..=0xffff => { + out.push(major | 25); + out.extend_from_slice(&(value as u16).to_be_bytes()); + } + 0x1_0000..=0xffff_ffff => { + out.push(major | 26); + out.extend_from_slice(&(value as u32).to_be_bytes()); + } + _ => { + out.push(major | 27); + out.extend_from_slice(&value.to_be_bytes()); + } + } +} + +/// Append the head of a tag item to `out`. +pub(crate) fn write_tag(out: &mut Vec, tag: u64) { + write_head(out, MAJOR_TAG, tag); +} + +/// One decoded head: the item's major type and its argument (a value, +/// length, count, or tag number, by major type). +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct Head { + pub major: u8, + pub value: u64, +} + +/// A head violating the wire's deterministic-encoding contract, or no +/// head at all. +#[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)] +pub enum HeadError { + /// The input ended inside the head. + #[error("input ends inside a CBOR head")] + Truncated, + /// An indefinite-length head; the wire is definite-length only. + #[error("indefinite-length CBOR is not canonical")] + Indefinite, + /// A reserved additional-information value (28 through 30). + #[error("CBOR head uses a reserved additional-information value")] + Reserved, + /// A wider argument encoding than the value requires. + #[error("CBOR head is not in shortest form")] + NotShortest, +} + +/// Read one canonical head off the front of `input`, advancing past it. +pub(crate) fn read_head(input: &mut &[u8]) -> Result { + let (&initial, rest) = input.split_first().ok_or(HeadError::Truncated)?; + let major = initial >> 5; + let info = initial & 0x1f; + let (value, rest) = match info { + 0..=23 => (u64::from(info), rest), + 24 => { + let (&byte, rest) = rest.split_first().ok_or(HeadError::Truncated)?; + (u64::from(byte), rest) + } + 25 => { + let (bytes, rest) = split_argument::<2>(rest)?; + (u64::from(u16::from_be_bytes(bytes)), rest) + } + 26 => { + let (bytes, rest) = split_argument::<4>(rest)?; + (u64::from(u32::from_be_bytes(bytes)), rest) + } + 27 => { + let (bytes, rest) = split_argument::<8>(rest)?; + (u64::from_be_bytes(bytes), rest) + } + 28..=30 => return Err(HeadError::Reserved), + _ => return Err(HeadError::Indefinite), + }; + let width = 1 + (input.len() - rest.len() - 1); + if width != head_len(value) { + return Err(HeadError::NotShortest); + } + *input = rest; + Ok(Head { major, value }) +} + +/// Split a fixed-width head argument off `rest`. +fn split_argument(rest: &[u8]) -> Result<([u8; N], &[u8]), HeadError> { + if rest.len() < N { + return Err(HeadError::Truncated); + } + let (bytes, rest) = rest.split_at(N); + Ok((bytes.try_into().expect("split at the argument width"), rest)) +} + +/// How reading a head from a live transport failed. +#[derive(Debug, thiserror::Error)] +pub(crate) enum HeadReadError { + /// The transport failed (end-of-stream inside the head included). + #[error(transparent)] + Io(#[from] std::io::Error), + /// The head arrived whole but violates the deterministic contract. + #[error(transparent)] + Malformed(HeadError), +} + +/// Read one canonical head from `read`. +/// +/// A clean end-of-stream *before the first byte* returns `Ok(None)`; an +/// end-of-stream inside the head is an +/// [`UnexpectedEof`](std::io::ErrorKind::UnexpectedEof) I/O error. Not +/// cancel safe: a dropped future may have consumed part of the head. +pub(crate) async fn read_head_async( + read: &mut R, +) -> Result, HeadReadError> { + let mut initial = 0u8; + match read.read(std::slice::from_mut(&mut initial)).await? { + 0 => return Ok(None), + 1 => {} + _ => unreachable!("a one-byte read returns at most one byte"), + } + let extension = extension_len(initial)?; + let mut bytes = [0u8; 9]; + bytes[0] = initial; + read.read_exact(&mut bytes[1..1 + extension]).await?; + let mut input = &bytes[..1 + extension]; + read_head(&mut input) + .map(Some) + .map_err(HeadReadError::Malformed) +} + +/// Read one canonical head from a synchronous reader. +/// +/// The blocking twin of [`read_head_async`], with the same clean-close +/// and error contract; the sync codec oracle reads through this so the +/// two ingress paths share one head grammar. +#[cfg(test)] +pub(crate) fn read_head_io( + read: &mut R, +) -> Result, HeadReadError> { + let mut initial = 0u8; + match read.read(std::slice::from_mut(&mut initial))? { + 0 => return Ok(None), + 1 => {} + _ => unreachable!("a one-byte read returns at most one byte"), + } + let extension = extension_len(initial)?; + let mut bytes = [0u8; 9]; + bytes[0] = initial; + read.read_exact(&mut bytes[1..1 + extension])?; + let mut input = &bytes[..1 + extension]; + read_head(&mut input) + .map(Some) + .map_err(HeadReadError::Malformed) +} + +/// Bytes of head argument following an initial byte, before reading them. +fn extension_len(initial: u8) -> Result { + match initial & 0x1f { + 0..=23 => Ok(0), + 24 => Ok(1), + 25 => Ok(2), + 26 => Ok(4), + 27 => Ok(8), + 28..=30 => Err(HeadReadError::Malformed(HeadError::Reserved)), + _ => Err(HeadReadError::Malformed(HeadError::Indefinite)), + } +} + +#[cfg(test)] +mod tests; diff --git a/src/tree/mirror/cbor/tests.rs b/src/tree/mirror/cbor/tests.rs new file mode 100644 index 000000000..5b8bdba42 --- /dev/null +++ b/src/tree/mirror/cbor/tests.rs @@ -0,0 +1,105 @@ +use proptest::prelude::*; + +use super::*; + +/// Every head a writer emits reads back as the same `(major, value)` pair, +/// occupies exactly `head_len` bytes, and leaves trailing input untouched: +/// the writer and the canonical reader are inverses. +#[test] +fn heads_round_trip_at_their_stated_width() { + proptest!(|(major in 0u8..7, value: u64, trailing: Vec)| { + let mut bytes = Vec::new(); + write_head(&mut bytes, major, value); + prop_assert_eq!(bytes.len(), head_len(value)); + bytes.extend_from_slice(&trailing); + let mut input = bytes.as_slice(); + let head = read_head(&mut input).expect("a written head is canonical"); + prop_assert_eq!(head, Head { major, value }); + prop_assert_eq!(input, trailing.as_slice()); + }); +} + +/// A head whose argument is wider than its value requires is rejected as +/// non-shortest-form: the deterministic contract admits one spelling per +/// value. +#[test] +fn widened_heads_are_rejected() { + proptest!(|(major in 0u8..7, value: u64)| { + let widths: &[(u8, usize)] = &[(24, 1), (25, 2), (26, 4), (27, 8)]; + for &(info, width) in widths { + // Only widths strictly larger than the shortest form are + // non-canonical spellings of this value. + if width < head_len(value) || value >= 1u64 << (8 * width as u32).min(63) { + continue; + } + let mut bytes = vec![(major << 5) | info]; + bytes.extend_from_slice(&value.to_be_bytes()[8 - width..]); + let mut input = bytes.as_slice(); + prop_assert_eq!(read_head(&mut input), Err(HeadError::NotShortest)); + } + }); +} + +/// Indefinite-length and reserved additional-information heads are +/// rejected: the wire is definite-length, deterministic CBOR only. +#[test] +fn indefinite_and_reserved_heads_are_rejected() { + for major in 0u8..8 { + for (info, expected) in [ + (28, HeadError::Reserved), + (29, HeadError::Reserved), + (30, HeadError::Reserved), + (31, HeadError::Indefinite), + ] { + let bytes = [(major << 5) | info]; + let mut input = bytes.as_slice(); + assert_eq!(read_head(&mut input), Err(expected)); + } + } +} + +/// A head cut anywhere before its final byte is a truncation, and the +/// input is left unconsumed. +#[test] +fn truncated_heads_are_rejected() { + proptest!(|(major in 0u8..7, value: u64)| { + let mut bytes = Vec::new(); + write_head(&mut bytes, major, value); + for cut in 0..bytes.len() { + let mut input = &bytes[..cut]; + let before = input; + prop_assert_eq!(read_head(&mut input), Err(HeadError::Truncated)); + prop_assert_eq!(input, before); + } + }); +} + +/// The async head reader agrees with the slice reader on every written +/// head — the two ingress paths cannot drift — and reports a clean +/// end-of-stream before the first byte as `None`. +#[test] +fn async_heads_match_the_slice_reader() { + proptest!(|(major in 0u8..7, value: u64)| { + let mut bytes = Vec::new(); + write_head(&mut bytes, major, value); + let head = tokio::runtime::Builder::new_current_thread() + .build() + .expect("runtime builds") + .block_on(async { + let mut read = bytes.as_slice(); + read_head_async(&mut read).await + }) + .expect("a written head is canonical") + .expect("a nonempty stream yields a head"); + prop_assert_eq!(head, Head { major, value }); + }); + let none = tokio::runtime::Builder::new_current_thread() + .build() + .expect("runtime builds") + .block_on(async { + let mut read: &[u8] = &[]; + read_head_async(&mut read).await + }) + .expect("an empty stream is a clean close"); + assert!(none.is_none()); +} diff --git a/src/tree/mirror/framing.rs b/src/tree/mirror/framing.rs index ccbf75b12..52056542a 100644 --- a/src/tree/mirror/framing.rs +++ b/src/tree/mirror/framing.rs @@ -1,11 +1,14 @@ -//! Exact-read length-delimited framing shared by the mirror wire protocols. +//! Exact-read payload buffering, and the V1 wire's length-delimited +//! framing. //! -//! A framed body is a 4-byte big-endian length followed by exactly that many -//! payload bytes. The streaming protocol uses it for its greeting (the -//! causal-version and root-fan listing frames), variable-width supply runs -//! and their leaf records, and the trailing identity hand-off; -//! signal-delimited fixed bodies remain bare. The reader never consumes a byte -//! beyond the frame requested. +//! Two things live here. [`read_payload`] and [`resume_payload`] grow a +//! buffer only as bytes arrive — the memory policy every variable-length +//! body read in either protocol shares, and the one the allocator meters +//! price. Around them, [`FrameRead`] and [`FrameWrite`] carry the V1 +//! wire's frames: a 4-byte big-endian length followed by exactly that +//! many payload bytes. (The V2 wire's bodies are self-delimiting CBOR +//! items; only their payload reads come through here.) The reader never +//! consumes a byte beyond the frame requested. //! //! That guarantee makes a session boundary a stream position. A buffering //! reader can slurp leading bytes of traffic belonging after the current @@ -20,9 +23,12 @@ //! payload read outsizes the buffer and bypasses it. Caller-owned buffering //! is safe because it outlives a session and rides into the next one. -use tokio::io::{AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt}; +use tokio::io::{AsyncRead, AsyncReadExt}; +#[cfg(any(test, feature = "protocol-v1"))] +use tokio::io::{AsyncWrite, AsyncWriteExt}; /// Bytes occupied by the big-endian `u32` payload-length header. +#[cfg(any(test, feature = "protocol-v1", feature = "test-internals"))] pub(crate) const LENGTH_HEADER_LEN: usize = std::mem::size_of::(); /// The initial reservation granule for framed payload buffers. @@ -52,36 +58,6 @@ pub(crate) fn chunk_boundary_cuts(total: usize) -> Vec { cuts } -/// Bytes of one negotiated size word in the greeting's version frame: a -/// little-endian `u64`. -pub(crate) const GREETING_WORD_LEN: usize = std::mem::size_of::(); - -/// The greeting version frame's fixed prefix: three size words (the -/// sender's set size, version-size bound, and message-size target) ahead -/// of the version encoding. -/// -/// Sender, receiver, and every fixture measuring greeting frames must -/// agree on this width; it is defined once here so the layout can only -/// change in one place. -pub(crate) const GREETING_SIZE_WORDS_LEN: usize = 3 * GREETING_WORD_LEN; - -/// Split a greeting version frame's body into its three leading size -/// words: the sender's set size, version-size bound, and target message -/// size, in wire order (the version encoding follows the fixed prefix). -/// -/// `None` when the body is shorter than the prefix. Defined beside the -/// width constants so every reader of the layout — the handshake and the -/// capture renderer — decodes it through one function. -pub(crate) fn greeting_words(body: &[u8]) -> Option<(u64, u64, u64)> { - let word = |index: usize| { - let at = index * GREETING_WORD_LEN; - body.get(at..at + GREETING_WORD_LEN) - .and_then(|prefix| <[u8; GREETING_WORD_LEN]>::try_from(prefix).ok()) - .map(u64::from_le_bytes) - }; - Some((word(0)?, word(1)?, word(2)?)) -} - /// A payload length which cannot be represented by the framing header. #[derive(Debug, thiserror::Error)] #[error("payload length {len} exceeds the u32 framing limit")] @@ -93,7 +69,8 @@ pub struct LengthOverflow { pub source: std::num::TryFromIntError, } -/// Encode the checked big-endian length header shared by both wire codecs. +/// Encode the checked big-endian length header of the V1 wire codec. +#[cfg(any(test, feature = "protocol-v1"))] pub(crate) fn length_header(len: usize) -> Result<[u8; LENGTH_HEADER_LEN], LengthOverflow> { let len = u32::try_from(len).map_err(|source| LengthOverflow { len, source })?; Ok(len.to_be_bytes()) @@ -144,6 +121,7 @@ pub(crate) async fn resume_payload( Ok(payload) } +#[cfg(any(test, feature = "protocol-v1", feature = "test-internals"))] /// The read half of a session's transport, yielding one exact frame at a time. /// /// Stateless beyond the reader it wraps: it buffers nothing, so dropping it @@ -152,6 +130,7 @@ pub struct FrameRead { read: R, } +#[cfg(any(test, feature = "protocol-v1", feature = "test-internals"))] impl FrameRead { /// Wrap `read` for frame-at-a-time reading. pub fn new(read: R) -> Self { @@ -168,6 +147,7 @@ impl FrameRead { } } +#[cfg(any(test, feature = "protocol-v1", feature = "test-internals"))] impl FrameRead { /// Read one frame, growing the payload buffer as its bytes arrive. /// @@ -195,6 +175,7 @@ impl FrameRead { } } +#[cfg(any(test, feature = "protocol-v1"))] /// The write half of a session's transport, shipping one frame at a time. /// /// Every frame is flushed before [`frame`](Self::frame) returns, so dropping @@ -203,6 +184,7 @@ pub struct FrameWrite { write: W, } +#[cfg(any(test, feature = "protocol-v1"))] impl FrameWrite { /// Wrap `write` for frame-at-a-time writing. pub fn new(write: W) -> Self { @@ -219,6 +201,7 @@ impl FrameWrite { } } +#[cfg(any(test, feature = "protocol-v1"))] impl FrameWrite { /// Write `payload` as one frame — length header, then bytes — and flush. /// diff --git a/src/tree/mirror/handshake.rs b/src/tree/mirror/handshake.rs index 771677db8..65822601a 100644 --- a/src/tree/mirror/handshake.rs +++ b/src/tree/mirror/handshake.rs @@ -10,43 +10,96 @@ //! Keeping these phases separate permits a provider to learn that its peer is //! bootstrapping before it atomically snapshots the tree and forks its party. //! -//! ```text -//! [ magic = b"RUMORS": 6B | version: 2B (big-endian) -//! | network: 16B | intent: 1B ] -//! ``` +//! The preamble's spelling is the selected dialect's own: //! -//! Its 25-byte size is part of the wire dialect, so no redundant frame length -//! precedes it. Validation diagnoses magic, then protocol version, followed by -//! the semantic network/intent combination. Only after that validation may a -//! protocol trust peer-declared lengths. +//! - **V2**: one self-described CBOR item, so a V2 control stream is a +//! CBOR sequence from its very first byte — +//! `55799(["rumors", version: uint, network: bstr, intent: uint])`. +//! Every field's head is one byte at the values the dialect admits, so +//! the item is 30 bytes, fixed; that width is part of the dialect, so +//! no redundant frame length precedes it. +//! - **V1**: the legacy fixed frame, +//! `[ magic = b"RUMORS": 6B | version: 2B (big-endian) | network: 16B | +//! intent: 1B ]`, 25 bytes. +//! +//! Validation diagnoses magic, then protocol version, followed by the +//! semantic network/intent combination. Only after that validation may a +//! protocol trust peer-declared lengths. A V2 endpoint additionally +//! recognizes the legacy magic and diagnoses it as a version mismatch +//! rather than a foreign protocol, so a cross-dialect pairing reports +//! what it is. use tokio::io::{AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt}; -use crate::{Network, Protocol}; +use crate::{ + Network, Protocol, + observe::SessionHandle, + tree::mirror::cbor::{self, MAJOR_BSTR, MAJOR_UINT}, +}; + +/// The raw magic opening a legacy (V1) preamble frame. +/// +/// A V2 endpoint reads these bytes too — never to speak them, only to +/// diagnose a legacy peer as a version mismatch rather than a foreign +/// protocol — so the constant lives here unconditionally; the public +/// `PROTOCOL_MAGIC` name is `protocol-v1` vocabulary and re-spells it +/// behind that feature. +pub(crate) const LEGACY_MAGIC: [u8; 6] = *b"RUMORS"; -/// Bytes occupied by the fixed protocol marker. -const MAGIC_LEN: usize = crate::PROTOCOL_MAGIC.len(); +/// Bytes occupied by the legacy fixed protocol marker. +const MAGIC_LEN: usize = LEGACY_MAGIC.len(); -/// Bytes occupied by the big-endian wire-version field. +/// Bytes occupied by the legacy big-endian wire-version field. const VERSION_LEN: usize = std::mem::size_of::(); /// Canonical width of one network identifier. const NETWORK_LEN: usize = 16; -/// Bytes occupied by the intent discriminant. +/// Bytes occupied by the legacy intent discriminant. const INTENT_LEN: usize = std::mem::size_of::(); -/// Offset at which the wire version begins. +/// Offset at which the legacy wire version begins. const VERSION_AT: usize = MAGIC_LEN; -/// Offset at which the network identifier begins. +/// Offset at which the legacy network identifier begins. const NETWORK_AT: usize = VERSION_AT + VERSION_LEN; -/// Offset at which the intent discriminant sits. +/// Offset at which the legacy intent discriminant sits. const INTENT_AT: usize = NETWORK_AT + NETWORK_LEN; -/// Length of the complete fixed preamble. -const PREAMBLE_LEN: usize = INTENT_AT + INTENT_LEN; +/// Length of the complete legacy fixed preamble. +const LEGACY_PREAMBLE_LEN: usize = INTENT_AT + INTENT_LEN; + +/// The V2 preamble's fixed prefix: the self-described CBOR tag, the +/// four-item array head, and the text item `"rumors"`. +/// +/// A literal so validation is one comparison; `prefix_matches_the_writers` +/// pins it against the head writers' own rendering. +const V2_PREFIX: [u8; 11] = [ + 0xd9, 0xd9, 0xf7, 0x84, 0x66, b'r', b'u', b'm', b'o', b'r', b's', +]; + +/// Length of the complete V2 preamble item. +pub(crate) const V2_PREAMBLE_LEN: usize = V2_PREFIX.len() + 1 + (1 + NETWORK_LEN) + INTENT_LEN; + +/// The widest preamble either dialect reads. +const PREAMBLE_MAX: usize = { + // The buffer must hold whichever dialect is selected. + if V2_PREAMBLE_LEN > LEGACY_PREAMBLE_LEN { + V2_PREAMBLE_LEN + } else { + LEGACY_PREAMBLE_LEN + } +}; + +/// The exact preamble width of one dialect. +fn preamble_len(protocol: Protocol) -> usize { + match protocol { + #[cfg(any(test, feature = "protocol-v1"))] + Protocol::V1 => LEGACY_PREAMBLE_LEN, + Protocol::V2 => V2_PREAMBLE_LEN, + } +} /// A peer's declared purpose for one reconciliation session. #[derive(Debug, Clone, Copy, PartialEq, Eq)] @@ -63,7 +116,8 @@ impl Intent { self == Intent::Retire } - /// Render the intent to its one-byte wire discriminant. + /// Render the intent to its wire discriminant, shared by both + /// dialects (V1 spells it as a raw byte, V2 as a one-byte uint item). fn to_byte(self) -> u8 { match self { Intent::Remain => 0, @@ -91,20 +145,53 @@ pub(crate) struct Preamble { } impl Preamble { - /// Render one complete fixed-width preamble. - fn encode(self, protocol: Protocol) -> [u8; PREAMBLE_LEN] { - let mut bytes = [0; PREAMBLE_LEN]; - bytes[..MAGIC_LEN].copy_from_slice(&crate::PROTOCOL_MAGIC); - bytes[VERSION_AT..NETWORK_AT].copy_from_slice(&(protocol as u16).to_be_bytes()); - bytes[NETWORK_AT..INTENT_AT].copy_from_slice(&self.network.to_bytes()); - bytes[INTENT_AT] = self.intent.to_byte(); - bytes + /// Render one complete preamble in the selected dialect. + fn encode(self, protocol: Protocol) -> Vec { + match protocol { + #[cfg(any(test, feature = "protocol-v1"))] + Protocol::V1 => { + let mut bytes = [0; LEGACY_PREAMBLE_LEN]; + bytes[..MAGIC_LEN].copy_from_slice(&LEGACY_MAGIC); + bytes[VERSION_AT..NETWORK_AT].copy_from_slice(&(protocol as u16).to_be_bytes()); + bytes[NETWORK_AT..INTENT_AT].copy_from_slice(&self.network.to_bytes()); + bytes[INTENT_AT] = self.intent.to_byte(); + bytes.to_vec() + } + Protocol::V2 => { + let mut bytes = Vec::with_capacity(V2_PREAMBLE_LEN); + bytes.extend_from_slice(&V2_PREFIX); + cbor::write_head(&mut bytes, MAJOR_UINT, protocol as u64); + cbor::write_head(&mut bytes, MAJOR_BSTR, NETWORK_LEN as u64); + bytes.extend_from_slice(&self.network.to_bytes()); + cbor::write_head(&mut bytes, MAJOR_UINT, u64::from(self.intent.to_byte())); + debug_assert_eq!(bytes.len(), V2_PREAMBLE_LEN, "the dialect width is fixed"); + bytes + } + } } /// Parse and validate one complete peer-controlled preamble. - fn decode(bytes: &[u8; PREAMBLE_LEN], protocol: Protocol) -> Result { + fn decode(bytes: &[u8], protocol: Protocol) -> Result { + match protocol { + #[cfg(any(test, feature = "protocol-v1"))] + Protocol::V1 => Self::decode_legacy(bytes, protocol), + Protocol::V2 => Self::decode_v2(bytes, protocol), + } + } + + /// Parse the legacy fixed frame. + #[cfg(any(test, feature = "protocol-v1"))] + fn decode_legacy(bytes: &[u8], protocol: Protocol) -> Result { let remote_magic = bytes[..MAGIC_LEN].try_into().expect("magic width"); - if remote_magic != crate::PROTOCOL_MAGIC { + if remote_magic != LEGACY_MAGIC { + // A V2-opening peer is a version mismatch, not a foreign + // protocol — the mirror of the V2 decoder's legacy detection. + if bytes[..V2_PREFIX.len()] == V2_PREFIX && bytes[V2_PREFIX.len()] < 24 { + return Err(Error::VersionMismatch { + local_protocol: protocol, + remote_version: u64::from(bytes[V2_PREFIX.len()]), + }); + } return Err(Error::MagicMismatch { remote_magic }); } let remote_version = u16::from_be_bytes( @@ -115,7 +202,7 @@ impl Preamble { if remote_version != protocol as u16 { return Err(Error::VersionMismatch { local_protocol: protocol, - remote_version, + remote_version: u64::from(remote_version), }); } @@ -125,6 +212,73 @@ impl Preamble { .expect("network width"), ); let intent = Intent::from_byte(bytes[INTENT_AT])?; + Self::admit(network, intent) + } + + /// Parse the V2 self-described item. + fn decode_v2(bytes: &[u8], protocol: Protocol) -> Result { + if bytes[..V2_PREFIX.len()] != V2_PREFIX { + // A legacy-magic peer is a version mismatch, not a foreign + // protocol: report what it is. + if bytes[..MAGIC_LEN] == LEGACY_MAGIC { + let remote_version = u16::from_be_bytes( + bytes[VERSION_AT..NETWORK_AT] + .try_into() + .expect("version width"), + ); + return Err(Error::VersionMismatch { + local_protocol: protocol, + remote_version: u64::from(remote_version), + }); + } + return Err(Error::MagicMismatch { + remote_magic: bytes[..MAGIC_LEN].try_into().expect("magic width"), + }); + } + let mut input = &bytes[V2_PREFIX.len()..]; + let malformed = |defect| Error::Malformed { defect }; + let version = cbor::read_head(&mut input) + .ok() + .filter(|head| head.major == MAJOR_UINT) + .ok_or(malformed(PreambleDefect::Version))?; + if version.value != protocol as u64 { + return Err(Error::VersionMismatch { + local_protocol: protocol, + remote_version: version.value, + }); + } + cbor::read_head(&mut input) + .ok() + .filter(|head| head.major == MAJOR_BSTR && head.value == NETWORK_LEN as u64) + .ok_or(malformed(PreambleDefect::Network))?; + // Defensive: a validated version and network head leave 17 of the + // fixed item's 30 bytes here, so the 16 network bytes always fit; + // the bound keeps `split_at` in range under any layout drift. + if input.len() < NETWORK_LEN { + return Err(malformed(PreambleDefect::NetworkTruncated)); + } + let (network, rest) = input.split_at(NETWORK_LEN); + input = rest; + let network = Network::from_bytes(network.try_into().expect("network width")); + let intent = cbor::read_head(&mut input) + .ok() + .filter(|head| head.major == MAJOR_UINT) + .ok_or(malformed(PreambleDefect::Intent))?; + // Defensive: the one-byte intent item consumes the fixed item's + // last byte, so nothing can trail; the check guards any caller + // handing the decoder non-fixed input. + if !input.is_empty() { + return Err(malformed(PreambleDefect::TrailingBytes)); + } + let intent = Intent::from_byte(u8::try_from(intent.value).expect( + "the 30-byte preamble leaves exactly one byte for the intent item, \ + whose one-byte head's value is at most 23", + ))?; + Self::admit(network, intent) + } + + /// Enforce the semantic network/intent combination. + fn admit(network: Network, intent: Intent) -> Result { if network.is_bootstrap() && intent.retiring() { return Err(Error::BootstrapRetireConflict); } @@ -139,33 +293,91 @@ pub(crate) enum Error { #[error(transparent)] Io(#[from] std::io::Error), /// The peer is not speaking the rumors protocol. - #[error("peer is not a rumors stream (remote magic: {remote_magic:x?})")] + #[error("peer is not a rumors stream (leading bytes: {remote_magic:x?})")] MagicMismatch { remote_magic: [u8; 6] }, /// The peer speaks a different wire dialect. #[error("peer speaks rumors protocol version {remote_version}, we selected {local_protocol:?}")] VersionMismatch { local_protocol: Protocol, - remote_version: u16, + remote_version: u64, }, - /// The peer's intent byte has no defined meaning. - #[error("peer sent an invalid intent byte ({byte:#04x})")] + /// The preamble opened correctly but a field of it is not spelled + /// the way the dialect demands. + #[error("peer preamble is malformed: {defect}")] + Malformed { defect: PreambleDefect }, + /// The peer closed the stream inside its preamble. + #[error("peer closed after sending {received} of its {expected} preamble bytes")] + Truncated { received: usize, expected: usize }, + /// The peer's intent has no defined meaning. + #[error("peer sent an invalid intent ({byte:#04x})")] IntentInvalid { byte: u8 }, /// A peer cannot simultaneously receive and donate an identity. #[error("peer claimed to bootstrap and retire in the same session")] BootstrapRetireConflict, } +/// Which field of a correctly-opened preamble failed to parse. +/// +/// Carried by +/// [`Error::PreambleMalformed`](crate::Error::PreambleMalformed): the +/// peer opened as a rumors stream of the selected dialect, but one +/// field is not spelled the way the wire demands. The preamble is +/// deterministic-encoding CBOR — one spelling per field — so every +/// defect here is a counterparty bug, never an alternate encoding. +/// Reachable only for [`Protocol::V2`]: the legacy frame's fields are +/// fixed-width raw bytes with no spelling to get wrong. +#[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)] +#[non_exhaustive] +pub enum PreambleDefect { + /// The version item is not a shortest-form unsigned int. + #[error("the version item is not an unsigned int")] + Version, + + /// The network item is not a 16-byte byte string. + #[error("the network item is not a 16-byte byte string")] + Network, + + /// The network byte string's bytes end inside the preamble item. + /// + /// Defensively reachable only: in the fixed 30-byte V2 preamble, a + /// validated version and network head always leave 17 bytes — the + /// 16 network bytes and the one-byte intent — so this variant + /// guards the decoder's width arithmetic against layout drift, not + /// any input the current dialect admits. + #[error("the network bytes end inside the preamble item")] + NetworkTruncated, + + /// The intent item is not a shortest-form unsigned int. + #[error("the intent item is not an unsigned int")] + Intent, + + /// Bytes trail the preamble's single item. + /// + /// Defensively reachable only: in the fixed 30-byte V2 preamble + /// with a validated version and network head, the one-byte intent + /// item consumes the last byte, so this variant guards the + /// decoder's width arithmetic against layout drift, not any input + /// the current dialect admits. + #[error("bytes trail the preamble item")] + TrailingBytes, +} + /// A cancel-safe, partially received fixed preamble. pub(crate) struct Staged { - buf: [u8; PREAMBLE_LEN], + buf: [u8; PREAMBLE_MAX], + /// The selected dialect's exact width, filled before validation. + want: usize, + protocol: Protocol, filled: usize, } impl Staged { - /// Start with no received preamble bytes. - pub(crate) fn new() -> Self { + /// Start with no received preamble bytes, sized for one dialect. + pub(crate) fn new(protocol: Protocol) -> Self { Self { - buf: [0; PREAMBLE_LEN], + buf: [0; PREAMBLE_MAX], + want: preamble_len(protocol), + protocol, filled: 0, } } @@ -180,14 +392,36 @@ impl Staged { where R: AsyncRead + Unpin + ?Sized, { - while self.filled < self.buf.len() { - match reader.read(&mut self.buf[self.filled..]).await? { + while self.filled < self.want { + match reader.read(&mut self.buf[self.filled..self.want]).await? { 0 if self.filled == 0 => return Ok(Fill::Closed), 0 => { - return Err(Error::Io(std::io::Error::new( - std::io::ErrorKind::UnexpectedEof, - "peer closed mid-preamble", - ))); + // A V2 endpoint reading a legacy 25-byte preamble sees + // the close five bytes early; diagnose the dialect + // rather than reporting a bare cut. (Never under V1, + // whose own preamble legitimately opens with the + // magic, and never when the claimed version matches — + // that is not a dialect skew.) + if self.want == V2_PREAMBLE_LEN + && self.filled >= NETWORK_AT + && self.buf[..MAGIC_LEN] == LEGACY_MAGIC + { + let remote_version = u64::from(u16::from_be_bytes( + self.buf[VERSION_AT..NETWORK_AT] + .try_into() + .expect("version width"), + )); + if remote_version != self.protocol as u64 { + return Err(Error::VersionMismatch { + local_protocol: self.protocol, + remote_version, + }); + } + } + return Err(Error::Truncated { + received: self.filled, + expected: self.want, + }); } read => self.filled += read, } @@ -196,9 +430,15 @@ impl Staged { } /// Validate a completely received frame in diagnostic order. - fn validate(&self, protocol: Protocol) -> Result { - debug_assert_eq!(self.filled, PREAMBLE_LEN, "validate before full"); - Preamble::decode(&self.buf, protocol) + fn validate(&self) -> Result { + debug_assert_eq!(self.filled, self.want, "validate before full"); + Preamble::decode(&self.buf[..self.want], self.protocol) + } + + /// The completely received frame's bytes. + fn received(&self) -> &[u8] { + debug_assert_eq!(self.filled, self.want, "read back before full"); + &self.buf[..self.want] } } @@ -210,6 +450,7 @@ pub(crate) async fn preamble( staged: &mut Staged, reader: &mut R, writer: &mut W, + observe: &SessionHandle, ) -> Result where R: AsyncRead + Unpin + ?Sized, @@ -219,25 +460,34 @@ where let write = async { writer.write_all(&local).await.map_err(Error::Io)?; - writer.flush().await.map_err(Error::Io) + writer.flush().await.map_err(Error::Io)?; + observe.control_sent(&local); + Ok(()) }; let read = async { match staged.fill(reader).await? { Fill::Filled => Ok(()), - Fill::Closed => Err(Error::Io(std::io::Error::new( - std::io::ErrorKind::UnexpectedEof, - "peer closed before sending its preamble", - ))), + // The peer hung up without sending a byte: a zero-length + // truncation, distinct from a transport failure. + Fill::Closed => Err(Error::Truncated { + received: 0, + expected: staged.want, + }), } }; futures_util::future::try_join(write, read).await?; - staged.validate(protocol) + let preamble = staged.validate()?; + // Only a validated frame is delivered: the item contract holds for + // conforming exchanges, and a malformed preamble aborts the session + // instead of feeding observers a non-item. + observe.control_received(staged.received()); + Ok(preamble) } /// Progress of a cancel-safe preamble arrival. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub(crate) enum Fill { - /// All 25 bytes have arrived. + /// The dialect's full preamble has arrived. Filled, /// The peer closed before sending any preamble byte. Closed, diff --git a/src/tree/mirror/handshake/tests.rs b/src/tree/mirror/handshake/tests.rs index 370ceaac5..a03e4864b 100644 --- a/src/tree/mirror/handshake/tests.rs +++ b/src/tree/mirror/handshake/tests.rs @@ -1,142 +1,253 @@ use proptest::prelude::*; use tokio::io::{duplex, split}; -use super::{Error, Intent, PREAMBLE_LEN, Preamble, Staged, preamble}; +use super::{ + Error, Intent, Preamble, PreambleDefect, Staged, V2_PREAMBLE_LEN, V2_PREFIX, preamble, +}; +use crate::observe::SessionHandle; use crate::{Network, Protocol}; -/// Construct a fully received preamble with one caller-selected intent byte. +/// Construct a fully received V2 preamble with one caller-selected raw +/// byte in the intent item's place. fn staged(network: Network, intent: u8) -> Staged { - let mut staged = Staged::new(); - staged.buf[..6].copy_from_slice(&crate::PROTOCOL_MAGIC); - staged.buf[6..8].copy_from_slice(&(Protocol::V2 as u16).to_be_bytes()); - staged.buf[8..24].copy_from_slice(&network.to_bytes()); - staged.buf[24] = intent; - staged.filled = PREAMBLE_LEN; + let encoded = Preamble { + network, + intent: Intent::Remain, + } + .encode(Protocol::V2); + let mut staged = Staged::new(Protocol::V2); + staged.buf[..encoded.len()].copy_from_slice(&encoded); + staged.buf[V2_PREAMBLE_LEN - 1] = intent; + staged.filled = V2_PREAMBLE_LEN; staged } +/// The V2 preamble's pinned prefix literal is exactly what the head +/// writers render for the self-described tag, the four-item array, and +/// the text `"rumors"`: the validation constant cannot drift from the +/// encoder. +#[test] +fn prefix_matches_the_writers() { + use crate::tree::mirror::cbor::{self, MAJOR_ARRAY, MAJOR_TEXT}; + let mut prefix = Vec::new(); + cbor::write_tag(&mut prefix, cbor::TAG_SELF_DESCRIBED); + cbor::write_head(&mut prefix, MAJOR_ARRAY, 4); + cbor::write_head(&mut prefix, MAJOR_TEXT, "rumors".len() as u64); + prefix.extend_from_slice(b"rumors"); + assert_eq!(prefix, V2_PREFIX); +} + /// Both sides exchange the shared preamble over a one-byte transport without /// deadlock, preserving each peer's network and intent exactly. #[test] fn fragmented_exchange_is_symmetric() { - let left = Network::from_bytes([1; 16]); - let right = Network::from_bytes([2; 16]); - let (left_io, right_io) = duplex(1); - let (left_read, left_write) = split(left_io); - let (right_read, right_write) = split(right_io); - let mut left_read = left_read; - let mut left_write = left_write; - let mut right_read = right_read; - let mut right_write = right_write; - let mut left_staged = Staged::new(); - let mut right_staged = Staged::new(); - - let (seen_by_left, seen_by_right) = pollster::block_on(async { - tokio::join!( - preamble( - Protocol::V2, - left, - Intent::Remain, - &mut left_staged, - &mut left_read, - &mut left_write, - ), - preamble( - Protocol::V2, - right, - Intent::Retire, - &mut right_staged, - &mut right_read, - &mut right_write, - ), - ) - }); + for protocol in [Protocol::V1, Protocol::V2] { + let left = Network::from_bytes([1; 16]); + let right = Network::from_bytes([2; 16]); + let (left_io, right_io) = duplex(1); + let (left_read, left_write) = split(left_io); + let (right_read, right_write) = split(right_io); + let mut left_read = left_read; + let mut left_write = left_write; + let mut right_read = right_read; + let mut right_write = right_write; + let mut left_staged = Staged::new(protocol); + let mut right_staged = Staged::new(protocol); - assert_eq!( - seen_by_left.unwrap(), - Preamble { - network: right, - intent: Intent::Retire, - } - ); - assert_eq!( - seen_by_right.unwrap(), - Preamble { - network: left, - intent: Intent::Remain, - } - ); + let observe = SessionHandle::default(); + let (seen_by_left, seen_by_right) = pollster::block_on(async { + tokio::join!( + preamble( + protocol, + left, + Intent::Remain, + &mut left_staged, + &mut left_read, + &mut left_write, + &observe, + ), + preamble( + protocol, + right, + Intent::Retire, + &mut right_staged, + &mut right_read, + &mut right_write, + &observe, + ), + ) + }); + + assert_eq!( + seen_by_left.unwrap(), + Preamble { + network: right, + intent: Intent::Retire, + } + ); + assert_eq!( + seen_by_right.unwrap(), + Preamble { + network: left, + intent: Intent::Remain, + } + ); + } } -/// Intent decoding is exhaustive: exactly the two defined bytes are accepted -/// for an established network and every other byte retains its typed value. +/// Intent decoding is exhaustive over the raw byte in the intent item's +/// place. +/// +/// The two defined values are accepted, other small uint items are the +/// typed intent rejection, and bytes that are no one-byte uint item at +/// all are the malformed-preamble class. #[test] fn intent_byte_space_is_exhaustive() { let network = Network::from_bytes([1; 16]); for byte in u8::MIN..=u8::MAX { - match (byte, staged(network, byte).validate(Protocol::V2)) { + match (byte, staged(network, byte).validate()) { (0, Ok(preamble)) => assert_eq!(preamble.intent, Intent::Remain), (1, Ok(preamble)) => assert_eq!(preamble.intent, Intent::Retire), (0 | 1, other) => panic!("defined intent {byte} was rejected: {other:?}"), - (byte, Err(Error::IntentInvalid { byte: rejected })) => assert_eq!(rejected, byte), - (_, other) => panic!("invalid intent produced the wrong result: {other:?}"), + (2..=0x17, Err(Error::IntentInvalid { byte: rejected })) => { + assert_eq!(rejected, byte); + } + ( + 0x18.., + Err(Error::Malformed { + defect: PreambleDefect::Intent, + }), + ) => {} + (_, other) => panic!("invalid intent {byte} produced the wrong result: {other:?}"), } } } /// A peer that closes the connection at any point inside the preamble -/// surfaces a typed I/O error, never a hang and never a partial decode. +/// surfaces a typed truncation, never a hang and never a partial decode. /// -/// Every strict prefix of the 25-byte frame is a structurally distinct -/// truncation (the boundaries between magic, version, network, and intent -/// included), so the whole prefix space is swept: zero bytes is the -/// clean-goodbye close, every longer prefix a mid-preamble cut, and both -/// must resolve to [`Error::Io`] with `UnexpectedEof`. +/// Every strict prefix of the fixed item is a structurally distinct +/// truncation, so the whole prefix space is swept in both dialects, each +/// cut resolving to [`Error::Truncated`] carrying the exact byte counts +/// of the cut (a V2 endpoint cut where a whole legacy preamble ends with +/// a skewed version is instead the version mismatch it is — the separate +/// legacy-peer test). #[test] -fn every_truncation_boundary_is_a_typed_eof() { +fn every_truncation_boundary_is_typed() { + for protocol in [Protocol::V1, Protocol::V2] { + let network = Network::from_bytes([1; 16]); + let full = Preamble { + network, + intent: Intent::Remain, + } + .encode(protocol); + + for cut in 0..full.len() { + let mut staged = Staged::new(protocol); + let mut reader = &full[..cut]; + let mut writer = tokio::io::sink(); + let result = pollster::block_on(preamble( + protocol, + network, + Intent::Remain, + &mut staged, + &mut reader, + &mut writer, + &SessionHandle::default(), + )); + match result { + Err(Error::Truncated { received, expected }) => { + assert_eq!(received, cut, "the truncation reports the cut point"); + assert_eq!( + expected, + full.len(), + "the truncation reports the dialect's full width" + ); + } + other => { + panic!("cut after {cut} bytes must be a typed truncation, got {other:?}") + } + } + } + } +} + +/// A V2 endpoint whose peer speaks the legacy dialect diagnoses the +/// version mismatch, not a bare cut or a foreign protocol. +/// +/// The legacy 25-byte preamble ends five bytes short of the V2 item, and +/// its magic names the rumors protocol at version 1. +#[test] +fn legacy_peer_is_a_version_mismatch() { let network = Network::from_bytes([1; 16]); - let full = Preamble { + let legacy = Preamble { network, intent: Intent::Remain, } - .encode(Protocol::V2); + .encode(Protocol::V1); - for cut in 0..full.len() { - let mut staged = Staged::new(); - let mut reader = &full[..cut]; - let mut writer = tokio::io::sink(); - let result = pollster::block_on(preamble( - Protocol::V2, - network, - Intent::Remain, - &mut staged, - &mut reader, - &mut writer, - )); - match result { - Err(Error::Io(error)) => assert_eq!( - error.kind(), - std::io::ErrorKind::UnexpectedEof, - "cut after {cut} bytes must be an unexpected EOF", - ), - other => panic!("cut after {cut} bytes must be a typed I/O error, got {other:?}"), - } - } + // The peer sent its whole legacy preamble and closed. + let mut staged = Staged::new(Protocol::V2); + let mut reader = legacy.as_slice(); + let mut writer = tokio::io::sink(); + let result = pollster::block_on(preamble( + Protocol::V2, + network, + Intent::Remain, + &mut staged, + &mut reader, + &mut writer, + &SessionHandle::default(), + )); + assert!( + matches!( + result, + Err(Error::VersionMismatch { + local_protocol: Protocol::V2, + remote_version: 1, + }) + ), + "expected the dialect diagnosis, got {result:?}", + ); + + // The peer's next five bytes (its greeting) arrived too: the full + // 30-byte read then validates, and the magic check diagnoses the + // dialect ahead of any structural complaint. + let mut padded = legacy; + padded.extend_from_slice(&[0; 5]); + let mut staged = Staged::new(Protocol::V2); + let mut reader = padded.as_slice(); + let mut writer = tokio::io::sink(); + let result = pollster::block_on(preamble( + Protocol::V2, + network, + Intent::Remain, + &mut staged, + &mut reader, + &mut writer, + &SessionHandle::default(), + )); + assert!(matches!( + result, + Err(Error::VersionMismatch { + local_protocol: Protocol::V2, + remote_version: 1, + }) + )); } /// A wrong magic is diagnosed first, before any other field is judged. /// -/// The frame here is wrong in every field — magic, version, and intent — -/// and must still surface [`Error::MagicMismatch`] carrying the exact -/// remote bytes: the diagnostic order promised by the module docs puts -/// "not a rumors stream" ahead of "wrong dialect". +/// The item here is wrong in every field and must still surface +/// [`Error::MagicMismatch`] carrying the leading remote bytes: the +/// diagnostic order promised by the module docs puts "not a rumors +/// stream" ahead of "wrong dialect". #[test] fn magic_mismatch_is_diagnosed_first() { let mut wrong = staged(Network::from_bytes([1; 16]), 0xFF); wrong.buf[..6].copy_from_slice(b"SROMUR"); - wrong.buf[6..8].copy_from_slice(&0xFFFF_u16.to_be_bytes()); - let result = wrong.validate(Protocol::V2); + let result = wrong.validate(); assert!( matches!( &result, @@ -148,16 +259,16 @@ fn magic_mismatch_is_diagnosed_first() { /// A wrong wire version is diagnosed before the semantic fields. /// -/// With a correct magic but a foreign version, the frame's (invalid) -/// intent byte must never be reached: the typed rejection is +/// With a correct opening but a foreign version, the item's (invalid) +/// intent must never be reached: the typed rejection is /// [`Error::VersionMismatch`] carrying the remote's declared version, so a /// dialect skew is reported as such rather than as a garbled body. #[test] fn version_mismatch_is_diagnosed_before_intent() { let mut wrong = staged(Network::from_bytes([1; 16]), 0xFF); - wrong.buf[6..8].copy_from_slice(&7_u16.to_be_bytes()); + wrong.buf[V2_PREFIX.len()] = 0x07; - let result = wrong.validate(Protocol::V2); + let result = wrong.validate(); assert!( matches!( result, @@ -171,38 +282,37 @@ fn version_mismatch_is_diagnosed_before_intent() { } proptest! { - /// Any complete 25-byte preamble decodes exactly as the field-by-field - /// oracle predicts: a typed error naming the first invalid field in - /// diagnostic order, or the valid preamble — never a panic. + /// Any complete V2 preamble whose fields are canonically spelled + /// decodes exactly as the field-by-field oracle predicts. /// - /// The strategy weights the magic and version toward their valid values - /// so the deeper fields' arms are actually reached; the oracle - /// recomputes the documented diagnosis order (magic, then version, then - /// intent, then the network/intent combination) independently of the - /// decoder. + /// The prediction: a typed error naming the first invalid field in + /// diagnostic order, or the valid preamble — never a panic. #[test] fn arbitrary_preamble_decodes_by_the_oracle( - magic in prop_oneof![Just(crate::PROTOCOL_MAGIC), any::<[u8; 6]>()], - version in prop_oneof![Just(Protocol::V2 as u16), any::()], + magic_valid in prop_oneof![Just(true), any::()], + version in prop_oneof![Just(Protocol::V2 as u8), 0_u8..=0x17], network in any::<[u8; 16]>(), - intent in prop_oneof![0_u8..=3, any::()], + intent in prop_oneof![0_u8..=3, 0_u8..=0x17], ) { - let mut bytes = [0u8; PREAMBLE_LEN]; - bytes[..6].copy_from_slice(&magic); - bytes[6..8].copy_from_slice(&version.to_be_bytes()); - bytes[8..24].copy_from_slice(&network); - bytes[24] = intent; + let mut bytes = Vec::with_capacity(V2_PREAMBLE_LEN); + if magic_valid { + bytes.extend_from_slice(&V2_PREFIX); + } else { + bytes.extend_from_slice(b"SROMURxxxxx"); + } + bytes.push(version); + bytes.push(0x50); + bytes.extend_from_slice(&network); + bytes.push(intent); let result = Preamble::decode(&bytes, Protocol::V2); - let as_oracle = if magic != crate::PROTOCOL_MAGIC { - matches!( - &result, - Err(Error::MagicMismatch { remote_magic }) if *remote_magic == magic, - ) - } else if version != Protocol::V2 as u16 { + let as_oracle = if !magic_valid { + matches!(&result, Err(Error::MagicMismatch { remote_magic }) if remote_magic == b"SROMUR") + } else if version != Protocol::V2 as u8 { matches!( &result, - Err(Error::VersionMismatch { remote_version, .. }) if *remote_version == version, + Err(Error::VersionMismatch { remote_version, .. }) + if *remote_version == u64::from(version), ) } else if intent > 1 { matches!(&result, Err(Error::IntentInvalid { byte }) if *byte == intent) @@ -217,6 +327,14 @@ proptest! { }; prop_assert!(as_oracle, "decode disagreed with the oracle: {:?}", result); } + + /// Arbitrary bytes in the preamble's place decode to a typed error or + /// a valid preamble, never a panic: the parser is total over its + /// fixed-width input. + #[test] + fn arbitrary_bytes_never_panic(bytes in any::<[u8; V2_PREAMBLE_LEN]>()) { + let _ = Preamble::decode(&bytes, Protocol::V2); + } } /// The bootstrap placeholder composes only with remain intent; retirement @@ -224,16 +342,86 @@ proptest! { #[test] fn bootstrap_intent_matrix_is_exhaustive() { assert_eq!( - staged(Network::BOOTSTRAP, 0) - .validate(Protocol::V2) - .unwrap(), + staged(Network::BOOTSTRAP, 0).validate().unwrap(), Preamble { network: Network::BOOTSTRAP, intent: Intent::Remain, } ); assert!(matches!( - staged(Network::BOOTSTRAP, 1).validate(Protocol::V2), + staged(Network::BOOTSTRAP, 1).validate(), Err(Error::BootstrapRetireConflict) )); } + +// Defensive-variant exemption: `PreambleDefect::NetworkTruncated` and +// `PreambleDefect::TrailingBytes` deliberately have no construction tests. +// In the fixed 30-byte V2 preamble, a validated version and network head +// always leave exactly 17 bytes -- the 16 network bytes and the one-byte +// intent -- so neither arm is reachable from any input the dialect admits; +// both guard the decoder's width arithmetic. Every reachable defect +// (`Version`, `Network`, `Intent`) has a construction: `Intent` in +// `intent_byte_space_is_exhaustive`, the other two below. + +/// A version item that is not an unsigned int is the typed version +/// defect: a negative-int head in the version item's place fails the +/// major-type filter, and the defect names the version field. +#[test] +fn version_item_wrong_major_is_the_version_defect() { + let mut wrong = staged(Network::from_bytes([1; 16]), 0); + // 0x38: a two-byte negative-int head; its argument byte (the network + // head behind it) parses, so the head is well-formed but the wrong + // major type. + wrong.buf[V2_PREFIX.len()] = 0x38; + let result = wrong.validate(); + assert!( + matches!( + result, + Err(Error::Malformed { + defect: PreambleDefect::Version, + }), + ), + "expected the version defect, got {result:?}", + ); +} + +/// A widened spelling of the correct version value is the typed version +/// defect. +/// +/// The wire admits one spelling per value, so `0x18 0x02` (a two-byte +/// head for 2) is rejected as non-canonical before its value is compared +/// against the dialect. +#[test] +fn widened_version_spelling_is_the_version_defect() { + let mut wrong = staged(Network::from_bytes([1; 16]), 0); + wrong.buf[V2_PREFIX.len()..V2_PREFIX.len() + 2].copy_from_slice(&[0x18, 0x02]); + let result = wrong.validate(); + assert!( + matches!( + result, + Err(Error::Malformed { + defect: PreambleDefect::Version, + }), + ), + "expected the version defect, got {result:?}", + ); +} + +/// A network item that is not a 16-byte byte string is the typed network +/// defect: a byte-string head declaring 17 bytes fails the length filter, +/// and the defect names the network field. +#[test] +fn network_item_wrong_length_is_the_network_defect() { + let mut wrong = staged(Network::from_bytes([1; 16]), 0); + wrong.buf[V2_PREFIX.len() + 1] = 0x51; + let result = wrong.validate(); + assert!( + matches!( + result, + Err(Error::Malformed { + defect: PreambleDefect::Network, + }), + ), + "expected the network defect, got {result:?}", + ); +} diff --git a/src/tree/mirror/party.rs b/src/tree/mirror/party.rs index 9358b829c..8f9722473 100644 --- a/src/tree/mirror/party.rs +++ b/src/tree/mirror/party.rs @@ -1,37 +1,175 @@ //! Trailing identity hand-off after content reconciliation. use before::Party; -use tokio::io::{AsyncRead, AsyncWrite}; +use tokio::io::{AsyncRead, AsyncWrite, AsyncWriteExt}; use crate::{ - Error, - tree::mirror::framing::{FrameRead, FrameWrite}, + Error, Protocol, + observe::{CaptureRead, SessionHandle}, + tags::PARTY_TAG, + tree::mirror::cbor::{self, HeadError, MAJOR_BSTR}, }; +/// Which part of a delivered identity hand-off failed to parse. +/// +/// Carried by [`Error::HandOffMalformed`]: the peer +/// delivered its promised identity hand-off, but the item is not spelled +/// the way the wire demands, or its content is not one canonical party +/// encoding. The hand-off is deterministic-encoding CBOR wrapping a +/// canonical party encoding — one spelling per donation — so every defect +/// here is a counterparty bug, never an alternate encoding. Reachable +/// only for [`Protocol::V2`]. +#[derive(Debug, thiserror::Error)] +#[non_exhaustive] +pub enum HandOffDefect { + /// The item does not open with the party-atom tag. + #[error("the hand-off does not carry the party-atom tag")] + NotPartyTagged, + + /// The party-atom tag wraps something other than a byte string. + #[error("the party-atom tag does not wrap a byte string")] + NotAByteString, + + /// The byte string declares a length this host cannot address. + /// + /// A 64-bit host addresses any declarable length, so this arises + /// only on narrower targets (e.g. `wasm32`). + #[error("the hand-off declares an unaddressable length")] + UnaddressableLength, + + /// A head violates the wire's deterministic-encoding contract. + #[error("a hand-off head is not canonical: {0}")] + HeadMalformed(HeadError), + + /// The byte string's content is not one canonical party encoding. + /// + /// The body arrived whole — exactly the length its head declared — + /// so this is never a transport cut: the content itself is wrong. + /// An encoding the declared length cuts short is + /// [`Truncated`](before::error::Decode::Truncated) here, not + /// [`Error::HandOffTruncated`]. + #[error("the hand-off bytes are not one canonical party encoding: {0}")] + Undecodable(before::error::Decode), +} + /// Ship a donated party after reconciliation has transferred all content. /// /// Bootstrapping sends a freshly forked party from provider to newcomer; -/// retirement sends the retiree's whole party toward its absorber. The exact -/// frame boundary leaves a following session preamble untouched. -pub(crate) async fn send(party: Party, writer: &mut W) -> Result<(), Error> +/// retirement sends the retiree's whole party toward its absorber. The +/// hand-off's spelling is the selected dialect's: under V2, one +/// self-delimiting item — the party-atom tag wrapping a byte string of +/// the party's canonical encoding — and under the frozen V1 wire, one +/// length-delimited frame of the bare encoding. Either way its exact +/// boundary leaves a following session preamble untouched. +pub(crate) async fn send( + protocol: Protocol, + party: Party, + writer: &mut W, + observe: &SessionHandle, +) -> Result<(), Error> where W: AsyncWrite + Unpin + ?Sized, { - // The frame delimits, so the body is the party's canonical encoding, - // bare. - FrameWrite::new(writer).frame(party.as_bytes()).await?; + #[cfg(any(test, feature = "protocol-v1"))] + if protocol == Protocol::V1 { + // The frame delimits, so the body is the party's canonical + // encoding, bare. + crate::tree::mirror::framing::FrameWrite::new(writer) + .frame(party.as_bytes()) + .await?; + return Ok(()); + } + let _ = protocol; + let bytes = party.as_bytes(); + let mut item = Vec::with_capacity( + cbor::head_len(PARTY_TAG) + cbor::head_len(bytes.len() as u64) + bytes.len(), + ); + cbor::write_tag(&mut item, PARTY_TAG); + cbor::write_head(&mut item, MAJOR_BSTR, bytes.len() as u64); + item.extend_from_slice(bytes); + writer.write_all(&item).await.map_err(Error::Io)?; + writer.flush().await.map_err(Error::Io)?; + observe.control_sent(&item); Ok(()) } /// Receive the identity donation promised by the peer's preamble intent. -pub(crate) async fn receive(reader: &mut R) -> Result +pub(crate) async fn receive( + protocol: Protocol, + reader: &mut R, + observe: &SessionHandle, +) -> Result +where + R: AsyncRead + Unpin + ?Sized, +{ + #[cfg(any(test, feature = "protocol-v1"))] + if protocol == Protocol::V1 { + let bytes = crate::tree::mirror::framing::FrameRead::new(reader) + .frame() + .await?; + return decode_party_v1(&bytes); + } + let _ = protocol; + if observe.attached() { + let mut capture = CaptureRead::new(reader); + let party = receive_v2(&mut capture).await?; + observe.control_received(capture.bytes()); + Ok(party) + } else { + receive_v2(reader).await + } +} + +/// Read and decode one V2 hand-off item. +async fn receive_v2(reader: &mut R) -> Result where R: AsyncRead + Unpin + ?Sized, { - let bytes = FrameRead::new(reader).frame().await?; - Party::decode(&bytes[..]) + let malformed = |defect| Error::HandOffMalformed { defect }; + let head = read_head(reader).await?; + if head.major != cbor::MAJOR_TAG || head.value != PARTY_TAG { + return Err(malformed(HandOffDefect::NotPartyTagged)); + } + let head = read_head(reader).await?; + if head.major != MAJOR_BSTR { + return Err(malformed(HandOffDefect::NotAByteString)); + } + let Ok(len) = usize::try_from(head.value) else { + return Err(malformed(HandOffDefect::UnaddressableLength)); + }; + // `read_payload` spells a close mid-payload as `UnexpectedEof`; a + // transport failure keeps its own kind and passes through. + let bytes = crate::tree::mirror::framing::read_payload(&mut &mut *reader, len) + .await + .map_err(|e| match e.kind() { + std::io::ErrorKind::UnexpectedEof => Error::HandOffTruncated, + _ => Error::Io(e), + })?; + decode_party(&bytes) +} + +/// Decode one exact donation body into its canonical party. +/// +/// The body arrived whole, so every decode failure is the content's own: +/// a typed hand-off defect, never a transport error. The one exception +/// is the reader's own failure, which passes through — unreachable from +/// a slice, kept total. +fn decode_party(bytes: &[u8]) -> Result { + Party::decode(bytes).map_err(|defect| match defect { + before::error::Decode::Io(e) => Error::Io(e), + defect => Error::HandOffMalformed { + defect: HandOffDefect::Undecodable(defect), + }, + }) +} + +/// Decode one exact donation body into its canonical party, spelling +/// failures in the frozen V1 dialect's I/O vocabulary. +#[cfg(any(test, feature = "protocol-v1"))] +fn decode_party_v1(bytes: &[u8]) -> Result { + Party::decode(bytes) .map_err(|e| match e { - // A frame that ends inside the encoding is a truncation, not + // An item that ends inside the encoding is a truncation, not // corruption; the reader's own failures pass through. before::error::Decode::Truncated => { std::io::Error::new(std::io::ErrorKind::UnexpectedEof, e) @@ -42,5 +180,29 @@ where .map_err(Error::Io) } +/// Read one canonical head, treating any close as a truncation of the +/// hand-off the peer's preamble intent promised. +/// +/// `read_head_async` spells a close inside a head as `UnexpectedEof`, so +/// that kind joins the clean close before the first byte as +/// [`Error::HandOffTruncated`]; a transport failure keeps its own kind +/// and passes through as [`Error::Io`]. +async fn read_head(reader: &mut R) -> Result +where + R: AsyncRead + Unpin + ?Sized, +{ + match cbor::read_head_async(reader).await { + Ok(Some(head)) => Ok(head), + Ok(None) => Err(Error::HandOffTruncated), + Err(cbor::HeadReadError::Io(io)) if io.kind() == std::io::ErrorKind::UnexpectedEof => { + Err(Error::HandOffTruncated) + } + Err(cbor::HeadReadError::Io(io)) => Err(Error::Io(io)), + Err(cbor::HeadReadError::Malformed(head)) => Err(Error::HandOffMalformed { + defect: HandOffDefect::HeadMalformed(head), + }), + } +} + #[cfg(test)] mod tests; diff --git a/src/tree/mirror/party/tests.rs b/src/tree/mirror/party/tests.rs index 746749af1..8ff2e4512 100644 --- a/src/tree/mirror/party/tests.rs +++ b/src/tree/mirror/party/tests.rs @@ -1,45 +1,64 @@ //! Ingress validation of the trailing party-donation frame. //! //! The donated identity is the last peer-controlled payload of a bootstrap -//! or retire session: one length-delimited frame whose body must be exactly -//! one canonical party encoding. This suite feeds [`receive`] crafted frames -//! — truncations at each structural boundary, length lies in both -//! directions, trailing and arbitrary bodies — and pins that each surfaces -//! as the typed [`Error::Io`], never a panic, never a hang, and never a +//! or retire session: one party-atom-tagged byte string whose content must +//! be exactly one canonical party encoding. This suite feeds [`receive`] +//! crafted items — truncations at each structural boundary, length lies in +//! both directions, wrong tags, trailing and arbitrary bodies — and pins +//! that each surfaces as its typed diagnosis ([`Error::HandOffTruncated`] +//! for a stream that stops, [`Error::HandOffMalformed`] with the naming +//! defect for bytes that lie), never a panic, never a hang, and never a //! partial identity; and that a clean receive leaves the next session's //! bytes untouched in the transport. use before::Party; +use before::error::Decode; use proptest::collection::vec; use proptest::prelude::*; -use super::{receive, send}; +use super::{HandOffDefect, receive, send}; use crate::Error; +use crate::Protocol; +use crate::observe::SessionHandle; use crate::tree::arb::nth_party; -use crate::tree::mirror::framing::LENGTH_HEADER_LEN; +use crate::tree::mirror::cbor::{self, MAJOR_BSTR}; -/// Length-delimit one frame body exactly as [`send`] does. +/// Wrap one item content exactly as [`send`] does: the party-atom tag, +/// then a byte string of the content. fn frame(body: &[u8]) -> Vec { - let len = u32::try_from(body.len()).expect("test frame bodies fit in u32"); - let mut bytes = len.to_be_bytes().to_vec(); + let mut bytes = Vec::new(); + cbor::write_tag(&mut bytes, crate::tags::PARTY_TAG); + cbor::write_head(&mut bytes, MAJOR_BSTR, body.len() as u64); bytes.extend_from_slice(body); bytes } /// Receive a donation from crafted wire bytes through the production ingress. fn receive_party(bytes: &[u8]) -> Result { - pollster::block_on(async { receive(&mut &bytes[..]).await }) + pollster::block_on(async { + receive(Protocol::V2, &mut &bytes[..], &SessionHandle::default()).await + }) } -/// Unwrap the sole error variant this ingress can produce. -fn io_error(result: Result) -> std::io::Error { +/// Unwrap the typed defect of a malformed donation. +fn defect(result: Result) -> HandOffDefect { match result { - Err(Error::Io(error)) => error, + Err(Error::HandOffMalformed { defect }) => defect, Ok(_) => panic!("a malformed donation must not decode"), - Err(other) => panic!("the donation ingress fails as Error::Io, got {other:?}"), + Err(other) => { + panic!("a malformed donation fails as Error::HandOffMalformed, got {other:?}") + } } } +// Defensive-variant exemption: `HandOffDefect::UnaddressableLength` +// deliberately has no construction test. The byte-string head declares its +// length as a u64, and on a 64-bit host `usize::try_from` cannot fail, so +// the arm guards the width arithmetic and is unreachable from any input +// here. Only a 32-bit target (e.g. wasm32) can present a declarable length +// past `usize::MAX`, and this suite has no 32-bit test host. Every other +// defect variant has a construction below. + /// A donated party survives its wire trip intact. /// /// [`send`] and [`receive`] are each other's inverses: the received party @@ -50,84 +69,159 @@ fn io_error(result: Result) -> std::io::Error { fn a_donated_party_round_trips() { pollster::block_on(async { let mut wire = Vec::new(); - send(nth_party(3), &mut wire).await.expect("donation sends"); - let received = receive(&mut &wire[..]) + send( + Protocol::V2, + nth_party(3), + &mut wire, + &SessionHandle::default(), + ) + .await + .expect("donation sends"); + let received = receive(Protocol::V2, &mut &wire[..], &SessionHandle::default()) .await .expect("a canonical donation decodes"); assert_eq!(received, nth_party(3)); }); } -/// A peer that closes before or inside the frame header is a typed EOF. +/// A peer that closes before or inside the item's heads is a typed +/// truncation. /// -/// Every strict prefix of the four-byte length header — the close at the -/// boundary included — must resolve to [`Error::Io`] with `UnexpectedEof`, +/// Every strict prefix of the tag and byte-string heads — the close at +/// each boundary included — must resolve to [`Error::HandOffTruncated`], /// never a hang on bytes that cannot arrive. #[test] -fn truncated_frame_header_is_a_typed_eof() { - for cut in 0..LENGTH_HEADER_LEN { - let error = io_error(receive_party(&vec![0; cut])); - assert_eq!( - error.kind(), - std::io::ErrorKind::UnexpectedEof, - "cut after {cut} header bytes must be an unexpected EOF", +fn truncated_frame_header_is_a_typed_truncation() { + let heads = frame(&[0]); + let heads = &heads[..heads.len() - 1]; + for cut in 0..heads.len() { + assert!( + matches!(receive_party(&heads[..cut]), Err(Error::HandOffTruncated)), + "cut after {cut} head bytes must be the typed truncation", ); } } -/// A frame declaring more bytes than the peer sends is a typed EOF. +/// An item that is not the party-atom tag is a typed protocol violation. +/// +/// The tag is the hand-off's identity on the wire; a different tag (or a +/// bare byte string) must surface as +/// [`HandOffDefect::NotPartyTagged`], never decode. +#[test] +fn wrong_tag_is_a_typed_error() { + let mut wrong = Vec::new(); + cbor::write_tag(&mut wrong, crate::tags::VERSION_TAG); + cbor::write_head(&mut wrong, MAJOR_BSTR, 0); + assert!(matches!( + defect(receive_party(&wrong)), + HandOffDefect::NotPartyTagged + )); + + let mut bare = Vec::new(); + cbor::write_head(&mut bare, MAJOR_BSTR, 0); + assert!(matches!( + defect(receive_party(&bare)), + HandOffDefect::NotPartyTagged + )); +} + +/// A party-atom tag wrapping anything but a byte string is a typed +/// protocol violation. +/// +/// The tag's content is defined as a byte string of the party's canonical +/// encoding; any other major type must surface as +/// [`HandOffDefect::NotAByteString`], never decode. +#[test] +fn wrapped_non_byte_string_is_a_typed_error() { + let mut wrong = Vec::new(); + cbor::write_tag(&mut wrong, crate::tags::PARTY_TAG); + cbor::write_head(&mut wrong, cbor::MAJOR_UINT, 7); + assert!(matches!( + defect(receive_party(&wrong)), + HandOffDefect::NotAByteString + )); +} + +/// A non-canonical head inside the hand-off is a typed protocol violation. +/// +/// The wire is deterministic-encoding CBOR: a widened (non-shortest-form) +/// head spells a value the canonical wire never produces, and must +/// surface as [`HandOffDefect::HeadMalformed`] naming the head's fault. +#[test] +fn malformed_head_is_a_typed_error() { + // A two-byte spelling of tag 0: canonical is the one-byte head. + let widened = [0xd8, 0x00]; + assert!(matches!( + defect(receive_party(&widened)), + HandOffDefect::HeadMalformed(cbor::HeadError::NotShortest) + )); +} + +/// A frame declaring more bytes than the peer sends is a typed truncation. /// /// The over-declared length makes the exact body read run off the end of -/// the stream; the lie must surface as [`Error::Io`] with `UnexpectedEof`, -/// never as a partially filled body handed to the party decoder. +/// the stream; the lie must surface as [`Error::HandOffTruncated`], never +/// as a partially filled body handed to the party decoder. #[test] -fn over_declared_frame_is_a_typed_eof() { - let mut bytes = 16_u32.to_be_bytes().to_vec(); +fn over_declared_frame_is_a_typed_truncation() { + let mut bytes = Vec::new(); + cbor::write_tag(&mut bytes, crate::tags::PARTY_TAG); + cbor::write_head(&mut bytes, MAJOR_BSTR, 16); bytes.extend_from_slice(&[1, 2, 3, 4]); - let error = io_error(receive_party(&bytes)); - assert_eq!(error.kind(), std::io::ErrorKind::UnexpectedEof); + assert!(matches!( + receive_party(&bytes), + Err(Error::HandOffTruncated) + )); } -/// A zero-length frame body cannot carry an identity and is a typed error. +/// A zero-length frame body cannot carry an identity and is a typed defect. /// /// The anonymous (empty) id has no reader-path encoding — every canonical /// party carries at least one tag — so an empty body fails the decoder's -/// first bit read: [`Error::Io`] with `UnexpectedEof`, the under-declared -/// degenerate case. +/// first bit read. The body arrived whole (exactly the zero bytes its head +/// declared), so this is the content's own fault: +/// [`HandOffDefect::Undecodable`] with the decoder's truncation, never +/// [`Error::HandOffTruncated`], which is reserved for a stream that stops. #[test] fn empty_frame_body_is_a_typed_error() { - let error = io_error(receive_party(&frame(&[]))); - assert_eq!(error.kind(), std::io::ErrorKind::UnexpectedEof); + assert!(matches!( + defect(receive_party(&frame(&[]))), + HandOffDefect::Undecodable(Decode::Truncated) + )); } /// A party encoding cut short inside an honestly sized frame is rejected. /// /// The frame's header matches its body, but the body is a strict prefix of /// a canonical party encoding; the decoder runs out of bits and must -/// surface [`Error::Io`], never accept a smaller identity (which would -/// break party linearity). +/// surface [`HandOffDefect::Undecodable`], never accept a smaller identity +/// (which would break party linearity). #[test] fn under_declared_frame_is_a_typed_error() { let mut body = nth_party(3).as_bytes().to_vec(); body.truncate(body.len() - 1); - let error = io_error(receive_party(&frame(&body))); - assert_eq!(error.kind(), std::io::ErrorKind::UnexpectedEof); + assert!(matches!( + defect(receive_party(&frame(&body))), + HandOffDefect::Undecodable(Decode::Truncated) + )); } /// A frame with bytes after the party is rejected as non-canonical. /// /// The party encoding is prefix-free and [`receive`] decodes the frame body /// exactly: one identity, no remainder. Trailing garbage must surface as -/// typed `InvalidData` rather than being silently dropped. +/// [`HandOffDefect::Undecodable`] rather than being silently dropped. #[test] fn trailing_frame_bytes_are_rejected() { let mut body = nth_party(3).as_bytes().to_vec(); body.push(0xFF); - let error = io_error(receive_party(&frame(&body))); - assert_eq!(error.kind(), std::io::ErrorKind::InvalidData); + assert!(matches!( + defect(receive_party(&frame(&body))), + HandOffDefect::Undecodable(Decode::TrailingBits) + )); } /// Receiving one donation consumes exactly its frame, leaving later bytes @@ -140,11 +234,18 @@ fn trailing_frame_bytes_are_rejected() { fn bytes_after_the_frame_stay_untouched() { pollster::block_on(async { let mut wire = Vec::new(); - send(nth_party(3), &mut wire).await.expect("donation sends"); + send( + Protocol::V2, + nth_party(3), + &mut wire, + &SessionHandle::default(), + ) + .await + .expect("donation sends"); wire.extend_from_slice(b".RUMORS"); let mut cursor = &wire[..]; - receive(&mut cursor) + receive(Protocol::V2, &mut cursor, &SessionHandle::default()) .await .expect("a canonical donation decodes"); assert_eq!(cursor, b".RUMORS", "bytes after the donation were consumed"); @@ -152,16 +253,19 @@ fn bytes_after_the_frame_stay_untouched() { } proptest! { - /// Arbitrary frame bodies decode to a party or the typed [`Error::Io`] — - /// never a panic — and anything accepted is canonical. + /// Arbitrary frame bodies decode to a party or the typed + /// [`Error::HandOffMalformed`] — never a panic — and anything accepted + /// is canonical. /// /// The frame is honestly sized around an arbitrary body, so the fuzz /// lands on the party bit codec rather than on the allocator via a lied - /// length header (the header lies are pinned deterministically above). - /// The canonicality arm mirrors `before`'s decode fuzz target: an - /// accepted body re-encodes byte-for-byte, so no two frames name one - /// identity. Decoding a slice always terminates, so a completed run is - /// also the no-hang witness. + /// length header (the header lies are pinned deterministically above), + /// and the whole body always arrives: every rejection is the content's + /// own [`HandOffDefect::Undecodable`], never a truncation. The + /// canonicality arm mirrors `before`'s decode fuzz target: an accepted + /// body re-encodes byte-for-byte, so no two frames name one identity. + /// Decoding a slice always terminates, so a completed run is also the + /// no-hang witness. #[test] fn arbitrary_frame_bodies_never_panic(body in vec(any::(), 0..64)) { match receive_party(&frame(&body)) { @@ -169,8 +273,8 @@ proptest! { let reencoded = party.as_bytes().to_vec(); prop_assert_eq!(reencoded, body, "accepted donation was not canonical"); } - Err(Error::Io(_)) => {} - Err(other) => prop_assert!(false, "expected a typed I/O error, got {other:?}"), + Err(Error::HandOffMalformed { defect: HandOffDefect::Undecodable(_) }) => {} + Err(other) => prop_assert!(false, "expected a typed hand-off defect, got {other:?}"), } } } diff --git a/src/tree/mirror/streaming.rs b/src/tree/mirror/streaming.rs index 88baf7129..04a59d6ba 100644 --- a/src/tree/mirror/streaming.rs +++ b/src/tree/mirror/streaming.rs @@ -50,7 +50,7 @@ pub(crate) mod convert; mod driver; mod erased; pub mod materialized; -mod message; +pub(crate) mod message; mod protocol; pub mod remote; pub mod stats; diff --git a/src/tree/mirror/streaming/erased.rs b/src/tree/mirror/streaming/erased.rs index b59ff85d4..bec8d43b1 100644 --- a/src/tree/mirror/streaming/erased.rs +++ b/src/tree/mirror/streaming/erased.rs @@ -29,7 +29,7 @@ //! Outside the walk, pairing a height-5 payload with a height-6 consumer //! is a compile error, exactly as before: the schedule's typestates and //! message streams remain height-typed, and this module's two boundary -//! conversions are minted at one height parameter apiece. Inside the +//! conversions are instantiated at one height parameter apiece. Inside the //! walk, height agreement is a runtime-witnessed property: every prefix //! re-tag debug-asserts its byte length against the claimed height, the //! [`ops`] dispatch derives its height from that same length (so the @@ -158,7 +158,7 @@ fn receiver_stream(receiver: Receiver) -> ReceiverStreamOf { } } -/// Mint the outgoing-response edge: an erased sender for the response +/// Create the outgoing-response edge: an erased sender for the response /// pump, and the typed response stream the schedule consumes. /// /// The one edge whose two halves speak different vocabularies — erased diff --git a/src/tree/mirror/streaming/materialized.rs b/src/tree/mirror/streaming/materialized.rs index a3f75a633..98a4c9d02 100644 --- a/src/tree/mirror/streaming/materialized.rs +++ b/src/tree/mirror/streaming/materialized.rs @@ -20,7 +20,7 @@ //! - **returns** flow up: exactly one `Option` per query, in query //! order — the reconciled scope, `None` meaning it resolved to nothing //! (recursive deletion, the same reading as [`Backend::parent`]'s `None` -//! return). Returns are prefix-less: the consumer minted the query, so +//! return). Returns are prefix-less: the consumer issued the query, so //! the key is redundant and the pairing is purely positional. //! //! # Why this is deadlock-free @@ -102,6 +102,7 @@ use std::pin::pin; use std::sync::Arc; use std::sync::atomic::{AtomicU64, Ordering}; +use crate::message::PayloadDepthLimit; use crate::tree::{ mirror::contained, mirror::streaming::{ @@ -518,6 +519,10 @@ impl: Leaf>> protocol::Connect for Handshaking { // so they cannot drift from the tree they describe. set_len: self.root.len(), max_version_bytes: self.root.max_version_bytes(), + // The walk is not the wire: on a wire session the proxy + // stamps its codec's configured limit over this field at + // send, so an in-process participant carries the default. + payload_depth_limit: PayloadDepthLimit::default().get(), target_message_size: self.target_message_size, listing: fan_listing(&fan), }; @@ -570,6 +575,10 @@ impl: Leaf>> protocol::Accept for Handshaking { // so they cannot drift from the tree they describe. set_len: self.root.len(), max_version_bytes: self.root.max_version_bytes(), + // The walk is not the wire: on a wire session the proxy + // stamps its codec's configured limit over this field at + // send, so an in-process participant carries the default. + payload_depth_limit: PayloadDepthLimit::default().get(), target_message_size: self.target_message_size, listing: fan_listing(&fan), }; diff --git a/src/tree/mirror/streaming/materialized/progress.rs b/src/tree/mirror/streaming/materialized/progress.rs index 1495998a7..58ade15f9 100644 --- a/src/tree/mirror/streaming/materialized/progress.rs +++ b/src/tree/mirror/streaming/materialized/progress.rs @@ -196,7 +196,7 @@ impl Trace { /// weave's placement) — deliberately NOT wired into `assert_valid`: /// the encoder does not and should not satisfy it. /// - /// D5 as minted: once the resolution of a scope's last disputed child + /// D5 as stated: once the resolution of a scope's last disputed child /// has been emitted, any further wire or query of that scope before /// the parent summary is a violation; a scope with no disputed /// children must emit its parent before any wire or query. The diff --git a/src/tree/mirror/streaming/materialized/unknown.rs b/src/tree/mirror/streaming/materialized/unknown.rs index d90c419c0..727413f21 100644 --- a/src/tree/mirror/streaming/materialized/unknown.rs +++ b/src/tree/mirror/streaming/materialized/unknown.rs @@ -16,7 +16,7 @@ //! //! The walk runs on erased nodes, its level named by its prefix's byte //! length: one instantiation per backend, where a height-typed recursion -//! would mint one per level. Each recursive call boxes its future +//! would instantiate one per level. Each recursive call boxes its future //! ([`BoxFuture`]) exactly as the typed tower did — the type stays flat — //! and the depth is bounded by the prefix's remaining height, at most 32, //! so the recursion is stack-safe by construction rather than by input diff --git a/src/tree/mirror/streaming/message.rs b/src/tree/mirror/streaming/message.rs index 5f7528026..aca936b54 100644 --- a/src/tree/mirror/streaming/message.rs +++ b/src/tree/mirror/streaming/message.rs @@ -12,8 +12,8 @@ //! [`materialized`](crate::tree::mirror::streaming::materialized) for the ordering argument). //! //! The memory unit is one reply: a maximally disputed reply is 256 -//! reactions × a 256-entry listing ≈ fan² hashes ≈ 1.6 MB encoded -//! (≈ 3.3 MB while an encoded and a decoded copy coexist), transient, at +//! reactions × a 256-entry listing ≈ fan² hashes ≈ 1.8 MB encoded +//! (≈ 3.5 MB while an encoded and a decoded copy coexist), transient, at //! most one in flight per stage. use std::cmp::Ordering; @@ -91,7 +91,7 @@ pub struct Greeting { /// supplies is one its tree materializes, so it must encode within /// this bound, and a session that receives one over it fails with a /// typed violation - /// ([`DecodeError::OversizedVersion`](crate::tree::mirror::streaming::remote::DecodeError::OversizedVersion)). + /// ([`ReplyDecodeError::OversizedVersion`](crate::tree::mirror::streaming::remote::ReplyDecodeError::OversizedVersion)). pub max_version_bytes: u64, /// The sender's supply-run byte target /// ([`Peer::target_message_size`](crate::Peer::target_message_size)). @@ -101,6 +101,18 @@ pub struct Greeting { /// builds *and* the frames built for it, so the more /// memory-constrained end sets the pace. pub target_message_size: u64, + /// The sender's configured payload nesting-depth limit + /// ([`Peer::payload_depth_limit`](crate::Peer::payload_depth_limit)), + /// in scopes. + /// + /// A session proceeds only if the two exchanged values are equal: + /// the limit is a property of the shared set (every replica must be + /// able to hold and forward all content), so a mismatch in either + /// direction is a typed, unconditional abort at the handshake, before + /// the equal-versions resolution. On the wire the value is authored + /// by the proxy from the session's payload codec at send; an + /// in-process participant carries the default. + pub payload_depth_limit: u64, /// The sender's root children as `(radix, hash)` pairs in strictly /// ascending radix order; empty when the sender's tree is empty. pub listing: Vec<(u8, Hash)>, diff --git a/src/tree/mirror/streaming/remote.rs b/src/tree/mirror/streaming/remote.rs index aaf6470ce..a2219f8c2 100644 --- a/src/tree/mirror/streaming/remote.rs +++ b/src/tree/mirror/streaming/remote.rs @@ -67,17 +67,17 @@ //! stream is invisible to every other by the link contract. mod adapter; -mod codec; +pub(crate) mod codec; mod error; mod proxy; mod streams; #[cfg(any(test, feature = "test-internals"))] -pub use codec::LinkCapture; +pub use codec::{HookCapture, HookStream, LinkCapture}; #[cfg(any(test, feature = "test-internals"))] -pub(crate) use codec::render_v2_capture; +pub(crate) use codec::{assert_items_account_for, render_hook_capture, stream_label}; #[cfg(any(test, feature = "test-internals"))] -pub(crate) use codec::{decode_frame_discarded, supply_signal_byte}; +pub(crate) use codec::{decode_frame_discarded, lone_record_run, supply_frame_head}; pub use error::*; /// The codec's logical stream count, for cross-layer constant assertions. diff --git a/src/tree/mirror/streaming/remote/adapter/decode.rs b/src/tree/mirror/streaming/remote/adapter/decode.rs index a7b82e413..3c689a50f 100644 --- a/src/tree/mirror/streaming/remote/adapter/decode.rs +++ b/src/tree/mirror/streaming/remote/adapter/decode.rs @@ -1,4 +1,4 @@ -use crate::message::PayloadDeserializer; +use crate::message::PayloadCodec; use std::pin::pin; use std::task::Poll; @@ -71,7 +71,7 @@ pub fn early_supplies( ledger: SupplyLedger, parent: ErasedPrefix, frames: F, - deserializer: PayloadDeserializer, + codec: PayloadCodec, ) -> impl Stream>> + Send where B: Backend: Leaf>, @@ -96,7 +96,7 @@ where parent, frames, tx, - deserializer, + codec, )); let mut read_result: Option>> = None; loop { @@ -139,7 +139,7 @@ async fn read_early( parent: ErasedPrefix, mut frames: F, leaves: mpsc::Sender, B::Node), B::Error>>, - deserializer: PayloadDeserializer, + codec: PayloadCodec, ) -> Result<(), DecodeError> where B: Backend: Leaf>, @@ -154,7 +154,7 @@ where let flow = match frame { Frame::Reaction(WireReaction::Supply(records), flow) => { any = true; - for record in records.records(deserializer) { + for record in records.records(codec) { let (version, message) = record.map_err(DecodeError::Record)?; let (leaf_prefix, _) = supplies.observe::(parent, &version)?; // The set-length half of the greeting's priced @@ -206,7 +206,7 @@ pub async fn decode_reply( ledger: SupplyLedger, scope: Scope, frames: &mut F, - deserializer: PayloadDeserializer, + codec: PayloadCodec, ) -> Result>, DecodeError> where B: Backend: Leaf>, @@ -222,7 +222,7 @@ where let (_, prefix) = scope.next().ok_or(ScopeError::UnpositionedQuery)?; Ok(Scope::new(prefix, listing)) }, - deserializer, + codec, ) .await } @@ -234,7 +234,7 @@ pub async fn decode_leaf_reply( ledger: SupplyLedger, scope: Scope, frames: &mut F, - deserializer: PayloadDeserializer, + codec: PayloadCodec, ) -> Result>, DecodeError> where B: Backend: Leaf>, @@ -253,7 +253,7 @@ where let (_, prefix) = scope.next().ok_or(ScopeError::UnpositionedQuery)?; Ok(Scope::leaf(prefix)) }, - deserializer, + codec, ) .await } @@ -265,7 +265,7 @@ async fn decode( scope: Scope, frames: &mut F, question: Q, - deserializer: PayloadDeserializer, + codec: PayloadCodec, ) -> Result>, DecodeError> where B: Backend: Leaf>, @@ -286,15 +286,7 @@ where // reader's hand per reply stream, at `node_bytes(0, version_bound)` // plus the slot itself (the window's supply-decode envelope). let (tx, rx) = mpsc::channel::, B::Node), B::Error>>(FAN); - let read = read_reply::( - version_bytes, - &ledger, - scope, - frames, - question, - tx, - deserializer, - ); + let read = read_reply::(version_bytes, &ledger, scope, frames, question, tx, codec); let assemble = assemble_supplies::(backend, children_height, rx); let (read, assembled) = futures::future::join(read, assemble).await; let Some(ReadReply { @@ -318,7 +310,7 @@ async fn read_reply( frames: &mut F, mut question: Q, leaves: mpsc::Sender, B::Node), B::Error>>, - deserializer: PayloadDeserializer, + codec: PayloadCodec, ) -> Result>, DecodeError> where B: Backend: Leaf>, @@ -364,7 +356,7 @@ where // Records leave the run one at a time and flow straight into // assembly: the whole-run bound is its encoded bytes, never a // decoded vector of leaves. - for record in records.records(deserializer) { + for record in records.records(codec) { let (version, message) = record.map_err(DecodeError::Record)?; let (leaf_prefix, run) = read .supplies diff --git a/src/tree/mirror/streaming/remote/adapter/tests.rs b/src/tree/mirror/streaming/remote/adapter/tests.rs index 662fc575f..548d4895a 100644 --- a/src/tree/mirror/streaming/remote/adapter/tests.rs +++ b/src/tree/mirror/streaming/remote/adapter/tests.rs @@ -63,7 +63,7 @@ struct LeafCase { impl LeafCase { /// A deterministic test leaf: the version scalar folds `value` and - /// `ticks` together so distinct cases mint distinct versions — the + /// `ticks` together so distinct cases produce distinct versions — the /// axis paths derive from — while `value` also picks the payload. fn new(value: u64, ticks: u8) -> Self { Self { diff --git a/src/tree/mirror/streaming/remote/adapter/tests/backend_errors.rs b/src/tree/mirror/streaming/remote/adapter/tests/backend_errors.rs index 017244da9..cd4d7f2e7 100644 --- a/src/tree/mirror/streaming/remote/adapter/tests/backend_errors.rs +++ b/src/tree/mirror/streaming/remote/adapter/tests/backend_errors.rs @@ -1,6 +1,6 @@ //! Source-error propagation across every backend operation reachable by the adapter. -use crate::message::Message; +use crate::message::{PayloadCodec, PayloadDepthLimit}; use std::convert::Infallible; use futures::{StreamExt, stream}; @@ -136,7 +136,7 @@ where unbounded(), Scope::new(parent.erase(), &[]), &mut frames, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), )) .err() .expect("the injected decoding failure was not reached"); diff --git a/src/tree/mirror/streaming/remote/adapter/tests/fan_occupancy.rs b/src/tree/mirror/streaming/remote/adapter/tests/fan_occupancy.rs index 2fb909aaa..9e99615dd 100644 --- a/src/tree/mirror/streaming/remote/adapter/tests/fan_occupancy.rs +++ b/src/tree/mirror/streaming/remote/adapter/tests/fan_occupancy.rs @@ -15,6 +15,7 @@ //! negative control proving the probe reports the regime rather than a //! constant. +use crate::message::{PayloadCodec, PayloadDepthLimit}; use futures::{Stream, StreamExt, TryStreamExt, stream}; use before::Version; @@ -85,7 +86,7 @@ fn peak_occupancy(mut input: impl Stream + Unpin) -> usize { unbounded(), Scope::opening(&[]), &mut input, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), ) .await .expect("ascending in-scope leaves assemble"); @@ -138,7 +139,7 @@ fn eager_early_supplies_ride_the_same_ceiling() { unbounded(), Prefix::new().erase(), stream::iter(frames(&leaves)), - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), ) .try_collect() .await diff --git a/src/tree/mirror/streaming/remote/adapter/tests/malformed.rs b/src/tree/mirror/streaming/remote/adapter/tests/malformed.rs index 2337f049a..d63f31b89 100644 --- a/src/tree/mirror/streaming/remote/adapter/tests/malformed.rs +++ b/src/tree/mirror/streaming/remote/adapter/tests/malformed.rs @@ -1,5 +1,6 @@ //! Focused malformed-wire cases which are not naturally height-parametric. +use crate::message::{PayloadCodec, PayloadDepthLimit}; use std::{collections::BTreeMap, convert::Infallible}; use before::Version; @@ -46,7 +47,7 @@ fn bare_end_cannot_follow_reactions() { unbounded(), Scope::new(parent.erase(), &[(0, hash(0))]), &mut frames, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), ) .await .err() @@ -69,7 +70,7 @@ fn stream_exhaustion_before_a_boundary_is_truncation() { unbounded(), Scope::new(parent.erase(), &[(0, hash(0))]), &mut frames, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), ) .await .err() @@ -103,7 +104,7 @@ fn an_unpositioned_match_is_rejected_in_both_directions() { unbounded(), Scope::new(parent.erase(), &[(1, hash(1))]), &mut frames, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), ) .await .err() @@ -154,7 +155,7 @@ fn an_unpositioned_query_is_rejected_in_both_directions() { unbounded(), Scope::new(parent.erase(), &[]), &mut frames, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), ) .await .err() @@ -254,7 +255,7 @@ fn leaf_query_matrix_is_exhaustive() { unbounded(), Scope::new(parent.erase(), &scope_listing), &mut frames, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), ) .await }); @@ -295,7 +296,7 @@ fn stream_end_is_not_a_protocol_reply() { unbounded(), Scope::new(parent.erase(), &[]), &mut frames, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), )) .err() .expect("stream control must be consumed below the adapter"); @@ -357,7 +358,7 @@ fn a_multi_leaf_run_is_one_supplied_subtree() { unbounded(), scope.clone(), &mut input, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), ) .await .expect("ascending in-scope leaves assemble"); @@ -419,7 +420,7 @@ fn leaf_order_is_enforced_within_one_run() { unbounded(), Scope::opening(&[]), &mut input, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), ) .await .err() @@ -461,7 +462,7 @@ fn leaf_scope_is_enforced_within_one_run() { unbounded(), Scope::new(parent.erase(), &[]), &mut input, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), ) .await .err() @@ -474,15 +475,17 @@ fn leaf_scope_is_enforced_within_one_run() { assert_eq!(actual, <[u8; 32]>::from(outside.path())); } -/// The run body of a single zero-length record: one bare record header. -const ZERO_LENGTH_RECORD_RUN: [u8; 4] = [0, 0, 0, 0]; +/// The run body of a single empty-content record: the embedded-sequence +/// tag wrapping an empty byte string. +const ZERO_LENGTH_RECORD_RUN: [u8; 3] = [0xd8, 0x3f, 0x40]; -/// A zero-length record passes structural validation but fails canonically. +/// An empty-content record passes structural validation but fails +/// canonically. /// -/// A `00000000` record header inside a run chains exactly, so the wire -/// accepts the run's structure; the record's empty body cannot hold a -/// version, so the reply decoder reports `DecodeError::Record` carrying the -/// version decoder's `UnexpectedEof`. +/// A record whose byte string is empty chains exactly, so the wire +/// accepts the run's structure; the empty content cannot hold a tagged +/// version, so the reply decoder reports `DecodeError::Record` carrying +/// the version decoder's `UnexpectedEof`. #[test] fn a_zero_length_record_fails_as_a_version_decode_error() { let run = LeafRun::from_encoded(ZERO_LENGTH_RECORD_RUN.to_vec()) @@ -498,7 +501,7 @@ fn a_zero_length_record_fails_as_a_version_decode_error() { unbounded(), Scope::opening(&[]), &mut input, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), ) .await .err() @@ -536,7 +539,7 @@ fn a_version_over_the_declared_bound_is_rejected() { unbounded(), Scope::new(parent.erase(), &[]), &mut input, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), ) .await .expect("a version exactly at the declared bound is admitted"); @@ -550,7 +553,7 @@ fn a_version_over_the_declared_bound_is_rejected() { unbounded(), Scope::new(parent.erase(), &[]), &mut input, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), ) .await .err() @@ -630,7 +633,7 @@ fn a_reply_past_the_declared_set_len_fails_at_its_first_over_record() { SupplyLedger::new(declared), Scope::opening(&[]), &mut input, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), ) .await }); @@ -706,7 +709,7 @@ fn a_supply_run_cannot_resume_after_another_reaction() { unbounded(), Scope::opening(&[(1, hash(1))]), &mut input, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), ) .await .err() diff --git a/src/tree/mirror/streaming/remote/adapter/tests/opening.rs b/src/tree/mirror/streaming/remote/adapter/tests/opening.rs index f020a5615..0cd99b2cb 100644 --- a/src/tree/mirror/streaming/remote/adapter/tests/opening.rs +++ b/src/tree/mirror/streaming/remote/adapter/tests/opening.rs @@ -9,6 +9,7 @@ //! since one side builds it from its own message and the other from the //! listing that crossed the wire. +use crate::message::{PayloadCodec, PayloadDepthLimit}; use before::Version; use futures::{TryStreamExt, stream}; @@ -157,7 +158,7 @@ fn opening_supplies_decode_by_radix_group() { unbounded(), Prefix::new().erase(), stream::iter(frames), - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), ) .try_collect(), ) @@ -205,7 +206,7 @@ fn opening_supplies_past_the_declared_set_len_are_rejected() { SupplyLedger::new(1), Prefix::new().erase(), stream::iter(frames), - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), ) .try_collect::>() .await @@ -233,7 +234,7 @@ fn empty_opening_supply_reply_decodes_to_nothing() { unbounded(), Prefix::new().erase(), stream::iter(frames), - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), ) .try_collect(), ) @@ -254,7 +255,7 @@ fn second_opening_supply_reply_is_rejected() { unbounded(), Prefix::new().erase(), stream::iter(frames), - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), ) .try_collect::>() .await @@ -276,7 +277,7 @@ fn positional_reaction_in_opening_supplies_is_rejected() { unbounded(), Prefix::new().erase(), stream::iter(frames), - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), ) .try_collect::>() .await diff --git a/src/tree/mirror/streaming/remote/adapter/tests/parking.rs b/src/tree/mirror/streaming/remote/adapter/tests/parking.rs index ec94abcd4..3824cff65 100644 --- a/src/tree/mirror/streaming/remote/adapter/tests/parking.rs +++ b/src/tree/mirror/streaming/remote/adapter/tests/parking.rs @@ -19,6 +19,7 @@ //! from `FAN² × size_of::<(u8, Hash)>()`. These tests hold both shapes to //! that accounting. +use crate::message::{PayloadCodec, PayloadDepthLimit}; use std::mem; use futures::{TryStreamExt, stream}; @@ -55,7 +56,7 @@ const LEAVES: u64 = 512; /// heavier framing — fails the pin and forces the module doc's charged /// figure (and `streaming/message.rs`, which states it) to be /// re-derived rather than silently going stale. -const DISPUTED_REPLY_TRANSIENT_CEILING: usize = 3_380_000; +const DISPUTED_REPLY_TRANSIENT_CEILING: usize = 3_570_000; /// A parked decoded reply holds one pointer-sized node handle per supplied /// node — O(fan) handles independent of how many leaves streamed through @@ -126,7 +127,7 @@ fn parked_supply_reply_holds_handles_not_subtrees() { unbounded(), scope, &mut frames, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), )) .expect("a canonical supplied fan decodes"); @@ -214,7 +215,7 @@ fn maximally_disputed_reply_parks_bounded_skeleton() { unbounded(), Scope::opening(&listing), &mut frames, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), )) .expect("a canonical maximally disputed reply decodes"); diff --git a/src/tree/mirror/streaming/remote/adapter/tests/properties.rs b/src/tree/mirror/streaming/remote/adapter/tests/properties.rs index c542cabcb..27d98496a 100644 --- a/src/tree/mirror/streaming/remote/adapter/tests/properties.rs +++ b/src/tree/mirror/streaming/remote/adapter/tests/properties.rs @@ -1,6 +1,6 @@ //! Laws which hold uniformly across the adapter's type-level height ladder. -use crate::message::Message; +use crate::message::{PayloadCodec, PayloadDepthLimit}; use std::convert::Infallible; use futures::{StreamExt, TryStreamExt, stream}; @@ -115,7 +115,7 @@ impl AdapterHeight for Z { unbounded(), scope.clone(), &mut frames, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), )) .expect("an in-scope leaf decodes"); @@ -165,7 +165,7 @@ impl AdapterHeight for Z { unbounded(), scope, &mut frames, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), )) .expect("canonical matches decode"); prop_assert!(decoded.questions.is_empty(), "height 0"); @@ -243,7 +243,7 @@ impl AdapterHeight for Z { unbounded(), scope, &mut frames, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), )) .expect("canonical leaf reactions decode"); prop_assert_eq!(&decoded.questions, &expected_questions, "height 0"); @@ -295,7 +295,7 @@ impl AdapterHeight for Z { unbounded(), scope, &mut frames, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), )) .expect("canonical mixed leaf reactions decode"); prop_assert_eq!(&decoded.questions, &expected_questions, "height 0"); @@ -325,7 +325,7 @@ impl AdapterHeight for Z { unbounded(), Scope::new(parent.erase(), &[]), &mut frames, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), )) .err() .expect("duplicate leaves are not strictly ascending"); @@ -346,7 +346,7 @@ impl AdapterHeight for Z { unbounded(), Scope::new(foreign.erase(), &[]), &mut frames, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), )) .err() .expect("a leaf outside the retained scope must fail"); @@ -380,7 +380,7 @@ where unbounded(), scope.clone(), &mut frames, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), )) .expect("an in-scope leaf decodes"); @@ -430,7 +430,7 @@ where unbounded(), scope, &mut frames, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), )) .expect("canonical matches decode"); prop_assert!(decoded.questions.is_empty(), "height {}", Self::HEIGHT); @@ -514,7 +514,7 @@ where unbounded(), scope, &mut frames, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), )) .expect("canonical positional reactions decode"); prop_assert_eq!( @@ -582,7 +582,7 @@ where unbounded(), scope, &mut frames, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), )) .expect("canonical mixed reactions decode"); prop_assert_eq!( @@ -622,7 +622,7 @@ where unbounded(), Scope::new(parent.erase(), &[]), &mut frames, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), )) .err() .expect("duplicate leaves are not strictly ascending"); @@ -649,7 +649,7 @@ where unbounded(), Scope::new(foreign.erase(), &[]), &mut frames, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), )) .err() .expect("a leaf outside the retained scope must fail"); diff --git a/src/tree/mirror/streaming/remote/adapter/tests/runs.rs b/src/tree/mirror/streaming/remote/adapter/tests/runs.rs index 9b012fe0c..d3709fe4a 100644 --- a/src/tree/mirror/streaming/remote/adapter/tests/runs.rs +++ b/src/tree/mirror/streaming/remote/adapter/tests/runs.rs @@ -9,6 +9,7 @@ //! decode → re-encode path the session uses, with the decode side fed //! deliberately unbatched input to prove batching is the encoder's choice. +use crate::message::{PayloadCodec, PayloadDepthLimit}; use std::collections::BTreeMap; use futures::{TryStreamExt, stream}; @@ -114,7 +115,7 @@ fn recode(frames: Vec, budget: RunBudget) -> Vec { unbounded(), Scope::opening(&[]), &mut input, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), ) .await .expect("ascending in-scope leaves assemble"); @@ -141,7 +142,7 @@ fn runs_of(frames: &[Frame]) -> Vec<&LeafRun> { fn records_of(runs: &[&LeafRun]) -> Vec<(Version, Message)> { runs.iter() .flat_map(|run| { - run.records(Message::deserializer::()) + run.records(PayloadCodec::new::(PayloadDepthLimit::default())) .collect::, _>>() .expect("an encoder-produced run holds canonical records") }) @@ -191,7 +192,7 @@ proptest! { // would have pushed its frame past the budget. if position + 1 < runs.len() { let (version, message) = runs[position + 1] - .records(Message::deserializer::()) + .records(PayloadCodec::new::(PayloadDepthLimit::default())) .next() .expect("a nonempty run yields a first record") .expect("an encoder-produced run holds canonical records"); @@ -271,7 +272,7 @@ fn a_batched_run_round_trips_the_reply() { unbounded(), Scope::opening(&[]), &mut input, - Message::deserializer::(), + PayloadCodec::new::(PayloadDepthLimit::default()), ) .await .expect("the batched frame decodes") diff --git a/src/tree/mirror/streaming/remote/codec.rs b/src/tree/mirror/streaming/remote/codec.rs index 69a4dd5b3..ab27b49f0 100644 --- a/src/tree/mirror/streaming/remote/codec.rs +++ b/src/tree/mirror/streaming/remote/codec.rs @@ -1,12 +1,29 @@ //! The self-delimiting frame grammar shared by every logical wire stream. //! -//! A signal byte densely encodes `(frame state, stream)` rather than imposing a -//! bit-field boundary. There are ten frame states — four reaction forms, each +//! Every frame is one CBOR array item, so a directed stream's frames form +//! an RFC 8742 CBOR sequence a generic tool can walk: `[signal]` for a +//! body-free frame, `[signal, body]` otherwise. The wire is *emitted* as +//! deterministic-encoding CBOR — shortest-form heads everywhere, definite +//! lengths only, one spelling per value +//! ([`cbor`](crate::tree::mirror::cbor)) — which is what keeps the +//! byte-pinning snapshot discipline meaningful. Ingress validates +//! structure everywhere; every head the codec hand-parses additionally +//! rejects indefinite lengths and non-shortest spellings, while the two +//! positions a general CBOR reader decodes — a record's version atom +//! (its byte-string head) and the application payload — judge neither +//! spelling rule; the atom's *content* canonicality is enforced by its +//! own strict decoder. +//! +//! The signal is an unsigned int carrying the dense `(frame state, +//! stream)` code. There are ten frame states — four reaction forms, each //! continuing or ending its reply, plus a bare empty-reply end and a bare -//! stream-end control — and 17 streams. `state * 17 + stream` occupies values 0 -//! through 169; the other 86 byte values are reserved. Speaker and stream then -//! select a phase-specific subset: the initiator admits 162 placements and the -//! responder 163, rejecting the rest before their frame body is read. +//! stream-end control — and 17 streams. `state * 17 + stream` occupies +//! values 0 through 169; the rest of the byte-ranged code space is +//! reserved, and the signal's stream component deliberately restates the +//! transport label so a mislabeled stream is its own diagnosis. Speaker +//! and stream then select a phase-specific subset: the initiator admits +//! 162 placements and the responder 163, rejecting the rest before their +//! frame body is read. //! //! Reply and stream lifetimes are deliberately orthogonal. Every nonempty //! reply ends on its final reaction; an empty reply is one bare reply-end @@ -16,20 +33,26 @@ //! lets a lazy reply stream flush each item immediately without looking ahead //! to discover whether that item is also the stream's last. //! -//! An empty query is wholly represented by its signal. A nonempty query carries -//! `count - 1` in one byte, covering 1 through 256. A supply body is a -//! [`LeafRun`] behind an exact `u32` body length: one or more -//! backend-neutral `(Version, Message)` leaf records, each behind its own -//! exact `u32` record length. The codec validates the run's record framing -//! once its whole body arrives but leaves the records encoded; the adapter -//! decodes them one at a time, constructs its backend-specific leaves, and -//! validates their version-derived paths. How many records share one run -//! is the sender's choice within the session's [`RunBudget`], and the -//! decoder holds arriving frames to that same budget: any within-budget -//! batching decodes, a single record of any size decodes (the encoder's -//! minimum-one-record overhang), and a frame batching multiple records past -//! the budget is rejected typed ([`DecodeErrorKind::OverbatchedRun`]) -//! before its body is buffered. +//! An empty query is wholly represented by its signal. A nonempty query's +//! body is a `{radix: hash}` map of one to 256 children: CBOR +//! deterministic encoding mandates ascending keys and the wire's canonical +//! form mandates strictly ascending radixes, so the two disciplines are +//! one rule, enforced once at ingress. A supply body is a [`LeafRun`] +//! behind the embedded-CBOR-sequence tag (63) wrapping a byte string — +//! the byte-string head is the run's exact length, preserving O(1) skip +//! and up-front pricing — and the run's records are each the same shape +//! in miniature: tag 63 wrapping a byte string whose content is the +//! tagged version atom followed by the message's own CBOR payload. The +//! codec validates the run's record framing once its whole body arrives +//! but leaves the records encoded; the adapter decodes them one at a +//! time, constructs its backend-specific leaves, and validates their +//! version-derived paths. How many records share one run is the sender's +//! choice within the session's [`RunBudget`], and the decoder holds +//! arriving frames to that same budget: any within-budget batching +//! decodes, a single record of any size decodes (the encoder's +//! minimum-one-record overhang), and a frame batching multiple records +//! past the budget is rejected typed +//! ([`DecodeErrorKind::OverbatchedRun`]) before its body is buffered. //! //! Encoding trusts the protocol and adapter to produce phase-correct, //! canonically ordered frames; it performs no redundant semantic validation. @@ -45,14 +68,19 @@ mod decode; mod encode; mod error; mod frame; +pub(crate) mod greeting; mod signal; #[cfg(test)] pub use budget::SUPPLY_FRAME_OVERHEAD; pub use budget::{DEFAULT_TARGET_MESSAGE_SIZE, RunBudget}; +pub use crate::tree::mirror::cbor::HeadError; #[cfg(any(test, feature = "test-internals"))] -pub use capture::{LinkCapture, render_v2_capture}; +pub use capture::{ + HookCapture, HookStream, LinkCapture, assert_items_account_for, render_hook_capture, + stream_label, +}; pub use decode::FrameRead; #[cfg(test)] pub use decode::{decode, decode_exact}; @@ -65,23 +93,61 @@ pub use error::{ }; #[cfg(test)] pub use frame::WireFrame; -pub use frame::{Frame, LeafRun, LeafRunError, Reaction, validate_children}; +pub use frame::{Frame, LeafRun, LeafRunError, ListingIssue, Reaction}; +#[cfg(test)] +pub(crate) use frame::{parse_listing_map, write_listing}; +pub use greeting::GreetingError; pub use signal::{ DecodeSignalError, End, Flow, InvalidSignalPlacement, InvalidWireSignal, Speaker, Stream, StreamClass, }; -/// The signal byte opening one initiator-spoken, reply-ending supply frame. +/// The whole wire prefix of one initiator-spoken, reply-ending supply +/// frame declaring a `declared`-byte run: the frame's array head, its +/// signal head, and the run's embedded-sequence tag and byte-string head. /// -/// The allocator meter (`tests/decode_alloc.rs`) prepends it to a hand-built -/// supply body so the codec's supply read path is drivable from outside the -/// crate. +/// The allocator meter (`tests/decode_alloc.rs`) prepends it to a +/// hand-built run body so the codec's supply read path is drivable from +/// outside the crate. #[cfg(any(test, feature = "test-internals"))] -pub(crate) fn supply_signal_byte() -> u8 { - signal::WireSignal::encode( +pub(crate) fn supply_frame_head(declared: usize) -> Vec { + use crate::tree::mirror::cbor; + let code = signal::WireSignal::encode( Stream::new(0).expect("stream 0 is within the stream range"), signal::Signal::Supply(Flow::End), - ) + ); + let mut bytes = Vec::new(); + cbor::write_head(&mut bytes, cbor::MAJOR_ARRAY, 2); + cbor::write_head(&mut bytes, cbor::MAJOR_UINT, u64::from(code)); + cbor::write_tag(&mut bytes, cbor::TAG_CBOR_SEQUENCE); + cbor::write_head(&mut bytes, cbor::MAJOR_BSTR, declared as u64); + bytes +} + +/// A structurally valid lone-record run of exactly `len` bytes: one +/// record whose heads plus arbitrary content span the run. +/// +/// Record content decodes lazily, so the meter's bodies need only pass +/// run-record framing. Panics when no single record's head widths can +/// reach `len` exactly (the head-width gaps); the meters' lengths are +/// chosen away from those gaps. +#[cfg(any(test, feature = "test-internals"))] +pub(crate) fn lone_record_run(len: usize) -> Vec { + use crate::tree::mirror::cbor; + for width in [1usize, 2, 3, 5, 9] { + let Some(content) = len.checked_sub(cbor::head_len(cbor::TAG_CBOR_SEQUENCE) + width) else { + continue; + }; + if cbor::head_len(content as u64) != width { + continue; + } + let mut run = Vec::with_capacity(len); + cbor::write_tag(&mut run, cbor::TAG_CBOR_SEQUENCE); + cbor::write_head(&mut run, cbor::MAJOR_BSTR, content as u64); + run.extend((0..content).map(|i| i as u8)); + return run; + } + panic!("no lone record spans exactly {len} bytes"); } /// Decode one initiator-spoken frame from `read` under `budget`, dropping diff --git a/src/tree/mirror/streaming/remote/codec/budget.rs b/src/tree/mirror/streaming/remote/codec/budget.rs index 256b36d71..e37170ca7 100644 --- a/src/tree/mirror/streaming/remote/codec/budget.rs +++ b/src/tree/mirror/streaming/remote/codec/budget.rs @@ -6,7 +6,7 @@ //! chunked by **bytes**, not record count: the encoder accumulates records //! into the current run and flushes it when appending the next record would //! push the frame's full wire size — its [`SUPPLY_FRAME_OVERHEAD`]-byte -//! signal-and-length envelope plus the run body — past the budget. A run +//! head envelope plus the run body — past the budget. A run //! always carries at least one record, so a single record larger than the //! budget ships alone in its own frame, exceeding the budget by exactly //! that record's overhang. Runs never span protocol reactions: the batching @@ -29,54 +29,76 @@ //! session minimum, not only by counterparty courtesy. The public //! knob is [`Peer::target_message_size`](crate::Peer::target_message_size). //! -//! Framing headroom: runs ride the wire's `u32` length header -//! ([`framing`](crate::tree::mirror::framing)), so +//! Framing headroom: the wire caps a run body at `u32::MAX` bytes, so //! [`from_bytes`](RunBudget::from_bytes) saturates every budget at //! [`MAX_RUN_BUDGET_BYTES`] — a run flushed within budget always fits the -//! header. The one frame that can still outgrow it is a *single record* -//! larger than the header's ceiling (the minimum-one-record rule ships it -//! alone): that is a record-size limit of the wire, which no budget -//! setting can lift, and the encoder rejects it at the header before -//! writing anything. +//! cap. The one frame that can still outgrow it is a *single record* +//! larger than the cap (the minimum-one-record rule ships it alone): that +//! is a record-size limit of the wire, which no budget setting can lift, +//! and the encoder rejects it at record level before writing anything. -use crate::tree::mirror::framing::LENGTH_HEADER_LEN; +use crate::tree::mirror::cbor; use crate::tree::mirror::streaming::window::FAN; -use super::frame::{MAX_QUERY_CHILDREN, QUERY_CHILD_LEN, QUERY_COUNT_LEN}; +use super::frame::{MAX_QUERY_CHILDREN, listing_entry_len}; use super::signal::WireSignal; +/// The exact wire size of one full-fan query frame. +/// +/// Its array head, its signal head (every query code takes the two-byte +/// head), the listing map's head at the full fan, and one entry per radix +/// value — the map spelling's per-entry cost varies with the key's head +/// width, so the sum walks the radix space rather than multiplying. +const FULL_FAN_QUERY_FRAME_LEN: usize = { + let mut total = cbor::head_len(2) // the frame's two-item array head + + WireSignal::MAX_ENCODED_LEN + + cbor::head_len(MAX_QUERY_CHILDREN as u64); + let mut radix = 0usize; + while radix < MAX_QUERY_CHILDREN { + total += listing_entry_len(radix as u8); + radix += 1; + } + total +}; + /// Default supply-run byte budget: the size of the maximally disputed reply. /// /// Derived from the wire constants, not measured: the decode side's /// documented memory unit is one decoded *reply* (the streaming `message` /// module docs), and the largest non-supply reply is maximally disputed — -/// `FAN` reactions, each a full-fan query frame of one signal byte, -/// one count byte, and `MAX_QUERY_CHILDREN` children of -/// `QUERY_CHILD_LEN` bytes each. Batching at this default therefore -/// never raises the wire's established per-reply memory ceiling. -pub const DEFAULT_TARGET_MESSAGE_SIZE: usize = - FAN * (WireSignal::ENCODED_LEN + QUERY_COUNT_LEN + MAX_QUERY_CHILDREN * QUERY_CHILD_LEN); +/// `FAN` reactions, each a full-fan query frame. Batching at this default +/// therefore never raises the wire's established per-reply memory ceiling. +pub const DEFAULT_TARGET_MESSAGE_SIZE: usize = FAN * FULL_FAN_QUERY_FRAME_LEN; -/// Wire bytes a supply frame wraps around its run body: the signal byte and -/// the body's `u32` length header. +/// Wire bytes a supply frame wraps around its run body, charged at their +/// widest. +/// +/// The envelope: the frame's array head, the signal's widest head, and +/// the run's embedded-sequence tag with the widest byte-string head the +/// run cap admits. The heads narrow for small runs; charging the envelope +/// constant keeps the flush algebra exact-or-conservative, never +/// optimistic. /// /// The budget prices whole wire frames, so the encoder's flush accounting /// charges this envelope alongside the accumulated records — a frame's full /// wire size stays within the budget except when a single record alone /// exceeds it. -pub const SUPPLY_FRAME_OVERHEAD: usize = WireSignal::ENCODED_LEN + LENGTH_HEADER_LEN; +pub const SUPPLY_FRAME_OVERHEAD: usize = cbor::head_len(2) + + WireSignal::MAX_ENCODED_LEN + + cbor::head_len(cbor::TAG_CBOR_SEQUENCE) + + cbor::head_len(u32::MAX as u64); -/// The largest supply-run budget the wire's framing can honor: budgets -/// saturate here at construction. +/// The largest supply-run budget the wire can honor: budgets saturate +/// here at construction. /// /// A frame's full wire size is its [`SUPPLY_FRAME_OVERHEAD`] envelope -/// plus the run body, and the body's length must encode in the `u32` -/// header ([`framing`](crate::tree::mirror::framing)). Capping the -/// whole-frame budget at `u32::MAX` less the envelope keeps every -/// within-budget flush under the header's ceiling with the envelope -/// already paid; without the cap, an over-ceiling budget lets a run -/// grow past 4 GiB in RAM and then deterministically fail at the length -/// header, re-failing every retry while the divergence persists. +/// plus the run body, and the wire caps a run body at `u32::MAX` bytes +/// (the cap every pricing closed form is denominated in). Capping the +/// whole-frame budget at that ceiling less the envelope keeps every +/// within-budget flush under the cap with the envelope already paid; +/// without it, an over-ceiling budget lets a run grow past 4 GiB in RAM +/// and then deterministically fail at the run head, re-failing every +/// retry while the divergence persists. pub const MAX_RUN_BUDGET_BYTES: usize = u32::MAX as usize - SUPPLY_FRAME_OVERHEAD; /// The byte budget one supply frame may grow to before the encoder flushes it. @@ -87,7 +109,7 @@ pub const MAX_RUN_BUDGET_BYTES: usize = u32::MAX as usize - SUPPLY_FRAME_OVERHEA /// minimum-one-record rule keeps every leaf shippable, degrading a zero /// budget to the pre-batching one-leaf-per-frame wire traffic, and the /// constructor's [`MAX_RUN_BUDGET_BYTES`] ceiling keeps every -/// within-budget flush inside the wire's length header. +/// within-budget flush inside the wire's run byte cap. #[derive(Clone, Copy, Debug, PartialEq, Eq)] pub struct RunBudget { /// Wire-frame bytes admitted before the next record forces a flush. diff --git a/src/tree/mirror/streaming/remote/codec/budget/tests.rs b/src/tree/mirror/streaming/remote/codec/budget/tests.rs index 2ec4d1cf4..764911985 100644 --- a/src/tree/mirror/streaming/remote/codec/budget/tests.rs +++ b/src/tree/mirror/streaming/remote/codec/budget/tests.rs @@ -6,13 +6,37 @@ use super::*; /// assert loudly. #[test] fn default_budget_matches_its_derivation() { - assert_eq!(DEFAULT_TARGET_MESSAGE_SIZE, 1_638_912); + assert_eq!(DEFAULT_TARGET_MESSAGE_SIZE, 1_830_400); assert_eq!( RunBudget::default(), RunBudget::from_bytes(DEFAULT_TARGET_MESSAGE_SIZE) ); } +/// The full-fan frame constant prices exactly what the encoder writes. +/// +/// One query frame carrying every radix, constructed and encoded, is +/// byte-for-byte the closed form's value — so the default budget's +/// derivation cannot drift from the wire. +#[test] +fn full_fan_frame_len_matches_an_actual_encode() { + use crate::tree::typed::{Hash, hash::MERKLE_HASH_LEN}; + + use super::super::signal::{Flow, Speaker, Stream}; + use super::super::{Frame, Reaction, WireFrame, encode}; + + let children: Vec<(u8, Hash)> = (0..=u8::MAX) + .map(|radix| (radix, Hash([radix; MERKLE_HASH_LEN]))) + .collect(); + let frame: WireFrame = ( + Stream::new(8).expect("an interior stream exists"), + Frame::Reaction(Reaction::Query(children), Flow::Continue), + ); + let mut encoded = Vec::new(); + encode(Speaker::Initiator, &frame, &mut encoded).expect("a full-fan query encodes"); + assert_eq!(encoded.len(), FULL_FAN_QUERY_FRAME_LEN); +} + /// The budget's admission boundary charges the whole wire frame. /// /// A record is admitted exactly while the frame — the signal-and-length @@ -32,20 +56,20 @@ fn admission_charges_the_frame_envelope() { assert!(!body_only.admits(body, record)); } -/// The default budget stays within the `u32` framing header, so a +/// The default budget stays within the wire's run byte cap, so a /// default-sized run is always representable on the wire. #[test] fn default_budget_fits_the_framing_header() { assert!(u32::try_from(DEFAULT_TARGET_MESSAGE_SIZE).is_ok()); } -/// Budgets above the wire's framing ceiling saturate to it, so a run the -/// budget admits always flushes within the `u32` length header instead -/// of buffering past 4 GiB and deterministically failing the flush. +/// Budgets above the wire's run byte cap saturate to it, so a run the +/// budget admits always stays within the cap instead of buffering past +/// 4 GiB and deterministically failing at the run head. /// /// The boundary is checked at the admitted maximum: the largest /// body-plus-record the saturated budget accepts still leaves the -/// flushed frame's body encodable in the header, envelope included. The +/// flushed frame's body within the cap, envelope included. The /// negative control shows the ceiling binds: one byte past the admitted /// maximum is refused, so the saturated budget is a real bound, not a /// pass-through. @@ -59,8 +83,8 @@ fn over_ceiling_budgets_saturate_to_the_framing_ceiling() { let body = MAX_RUN_BUDGET_BYTES - SUPPLY_FRAME_OVERHEAD - 1; assert!(budget.admits(body, 1)); assert!( - crate::tree::mirror::framing::length_header(body + 1).is_ok(), - "an admitted flush must encode in the u32 length header", + super::super::frame::checked_run_len(body + 1).is_ok(), + "an admitted flush must stay within the run byte cap", ); // Negative control: the ceiling genuinely binds. assert!(!budget.admits(body + 1, 1)); diff --git a/src/tree/mirror/streaming/remote/codec/capture.rs b/src/tree/mirror/streaming/remote/codec/capture.rs index 503f288fc..41bf8ebf2 100644 --- a/src/tree/mirror/streaming/remote/codec/capture.rs +++ b/src/tree/mirror/streaming/remote/codec/capture.rs @@ -1,460 +1,724 @@ -//! Stable semantic rendering of captured V2 traffic. +//! CBOR reflection rendering of captured V2 traffic. //! -//! Every frame renders its exact bytes AND the parse tree those bytes -//! decode to — the greeting's size words, the root-fan listing's -//! children, a query's `(radix, hash)` children, and a supply run's -//! per-record versions with each message's byte count (the leaf payload -//! type is the caller's, so message bytes stay exact-but-opaque) — so a -//! snapshot re-accept diff names the semantic field that moved instead -//! of asking a reviewer to diff hex. Payload bytes that do not decode -//! render as an explicit failure line above their hex; they never pass -//! as silent hex. Structural violations of the capture itself (a -//! truncated frame, a mislabeled stream) stay panics: they mean the -//! capture harness, not the peer, is broken. +//! The snapshot suites pin every wire byte of a captured session, and +//! this module is the form that pin takes: each observed item — one +//! CBOR item per line of the hook's contract — renders as a fully +//! unfolded value tree in extended-diagnostic-style notation, with a +//! rumors naming layer as `/ comment /` annotations (signal names, +//! listing children, tagged-atom meanings). A reviewer reading a +//! re-accept diff sees the semantic field that moved; a generic CBOR +//! reader sees plain diagnostic notation. +//! +//! # Why a rendering with no hexdump is still a byte pin +//! +//! The wire is deterministic-encoding CBOR as a stated contract: one +//! spelling per value, shortest-form heads only. The renderer walks +//! each item with the codec's own canonical head grammar +//! ([`cbor::read_head`]) and shows the item's *complete* content — +//! every integer exactly, every byte string as full hex, every text +//! string escaped, every tag number, and structure in wire order. Under +//! the determinism contract a complete value tree has exactly one +//! encoding, so the rendering is injective on wire bytes: two different +//! byte streams cannot render identically. Wherever the walk cannot +//! vouch for that inversion — non-canonical heads, invalid UTF-8, +//! embedded content that does not fill its byte string, or nesting past +//! the renderer's depth bound (one budget spanning the whole walk: +//! structural descent and embedded-byte-string unfolds draw it down +//! together) — the subtree falls back to an explicit failure line above +//! its exact bytes as hex, which is injective trivially. Byte counts on +//! item and stream headers come from the transport capture, so totals +//! stay exact. +//! +//! # Where the bytes come from +//! +//! Capture enters through the public observation hook +//! ([`crate::observe`]): the harness records each directed stream's +//! items and hands them here as a [`HookCapture`]. The transport-level +//! byte capture ([`LinkCapture`]) remains the totality oracle: the +//! harness asserts, per directed stream, that the stream's on-wire open +//! label followed by the concatenated observed items reproduces the +//! transport bytes exactly ([`assert_items_account_for`], +//! [`stream_label`]) — that assertion is what licenses a rendering of +//! *items* as a pin of *wire bytes*. Structural violations of the +//! capture itself (an item that is not one canonical CBOR item where +//! the wire grammar requires one, a frame contradicting its stream, a +//! label that does not parse) are panics: they mean the capture +//! harness, not the peer, is broken. Application payload bytes are the +//! application's own CBOR and only ever fall back explicitly. use std::{collections::BTreeMap, fmt::Write as _}; use crate::Version; -use crate::tree::mirror::framing::{GREETING_SIZE_WORDS_LEN, LENGTH_HEADER_LEN, greeting_words}; -use crate::tree::mirror::streaming::message::initiates; -use crate::tree::typed::Hash; +use crate::observe::Role; +use crate::tree::mirror::cbor::{ + self, MAJOR_ARRAY, MAJOR_BSTR, MAJOR_MAP, MAJOR_TAG, MAJOR_TEXT, MAJOR_UINT, TAG_CBOR_SEQUENCE, + TAG_EMBEDDED_ITEM, +}; use super::{ - End, Speaker, Stream, - decode::parse_query, - frame::{LeafRun, QUERY_CHILD_LEN, QUERY_COUNT_BIAS, QUERY_COUNT_LEN, validate_children}, + Speaker, Stream, signal::{Signal, WireSignal}, }; #[cfg(test)] mod tests; -/// Bytes occupied by the fixed session preamble. -const PREAMBLE_LEN: usize = 25; - -// The label's width is defined canonically beside the sender that writes -// it; captures parse with the same constant. -use super::super::streams::LABEL_LEN; - -/// Everything one endpoint sent during a captured session. +/// Everything one endpoint sent during a captured session, at the +/// transport level. /// /// The link keeps logical streams physically separate, so a capture is -/// already demultiplexed: the control stream's exact bytes, plus each opened -/// data stream's exact bytes (label included), in open order. +/// already demultiplexed: the control stream's exact bytes, plus each +/// opened data stream's exact bytes (label included), in open order. +/// The rendering consumes the hook's [`HookCapture`]; this transport +/// form is the totality oracle beside it, and the wire-legibility +/// property's raw material. pub struct LinkCapture { - /// The control stream's outgoing bytes: preamble, the greeting's - /// causal-version and root-fan listing frames, and any trailing party - /// hand-off, in order. + /// The control stream's outgoing bytes: preamble, the greeting item, + /// and any trailing party hand-off and epilogue, in order. pub control: Vec, - /// Each opened data stream's outgoing bytes: its two-byte label, then + /// Each opened data stream's outgoing bytes: its label items, then /// its frames through the explicit end control. pub streams: Vec>, } -/// Render both endpoints' captures without retaining cross-stream order. +/// Everything one endpoint sent during a captured session, as the +/// observation hook delivered it: one byte buffer per CBOR item. +pub struct HookCapture { + /// The role this side was elected, if the session held an election. + pub role: Option, + /// The control stream's sent items, in order. + pub control: Vec>, + /// The sent data streams, in any order; rendering sorts by index. + pub streams: Vec, +} + +/// One sent data stream, as observed through the hook plus the wire +/// facts the hook deliberately does not carry (the label's epoch and +/// the exact transport byte count). +pub struct HookStream { + /// The stream's wire index, from the hook's stream identity. + pub index: u8, + /// The elected role that speaks this stream's frames. + pub speaker: Role, + /// The epoch carried by the stream's on-wire open label. + pub epoch: u8, + /// The stream's exact transport byte count, label included. + pub wire_len: usize, + /// The stream's frames, one CBOR item each, in stream order. + pub items: Vec>, +} + +/// Parse one data stream's on-wire open label: two canonical unsigned +/// int items, `(epoch, stream index)`. /// -/// Control bytes remain byte-exact. Data streams are keyed by their labeled -/// stream index — exact bytes and order within each stream, stream groups -/// sorted — discarding the incidental order in which independent streams -/// were opened. Parsing accounts for every captured byte once. -pub fn render_v2_capture(a: &LinkCapture, b: &LinkCapture) -> String { - let a_control = Control::parse(&a.control); - let b_control = Control::parse(&b.control); +/// Returns the label values and the label's byte length. Panics if the +/// label is not two canonical byte-ranged uints: the capture harness, +/// not the peer, is broken. +pub fn stream_label(bytes: &[u8]) -> ((u8, u8), usize) { + let mut rest = bytes; + let epoch = label_item(&mut rest, "epoch"); + let index = label_item(&mut rest, "stream index"); + ((epoch, index), bytes.len() - rest.len()) +} - let (a_streams, b_streams) = match (&a_control.version, &b_control.version) { - (None, None) => (None, None), - (Some(a_version), Some(b_version)) if a_version == b_version => { - assert!( - a.streams.is_empty() && b.streams.is_empty(), - "equal versions open no data streams", - ); - (None, None) - } - (Some(a_version), Some(b_version)) => { - // Mirror the session's role election: the smaller advertised - // set initiates, canonical version bytes break ties. - let a_len = a_control - .set_len - .expect("a version frame carries its set size"); - let b_len = b_control - .set_len - .expect("a version frame carries its set size"); - let a_speaker = if initiates(a_len, a_version, b_len, b_version) { - Speaker::Initiator - } else { - Speaker::Responder - }; - ( - Some(Streams::parse(a_speaker, &a.streams)), - Some(Streams::parse(a_speaker.other(), &b.streams)), - ) - } - _ => panic!("both directions must either carry or omit a version frame"), - }; +/// Read one label item: a canonical byte-ranged unsigned int. +fn label_item(rest: &mut &[u8], what: &str) -> u8 { + let head = cbor::read_head(rest) + .unwrap_or_else(|e| panic!("captured stream label {what} is canonical: {e}")); + assert_eq!( + head.major, MAJOR_UINT, + "captured label {what} is an unsigned int" + ); + u8::try_from(head.value).unwrap_or_else(|_| panic!("captured label {what} is byte-ranged")) +} + +/// Assert that the concatenation of `items` reproduces `wire` exactly. +/// +/// The totality witness that licenses rendering hook items as a pin of +/// wire bytes: every transport byte is some observed item's byte, once, +/// in order. Panics on any mismatch, naming the first divergence. +pub fn assert_items_account_for(items: &[Vec], wire: &[u8]) { + let mut rest = wire; + for (index, item) in items.iter().enumerate() { + assert!( + rest.len() >= item.len() && &rest[..item.len()] == item.as_slice(), + "observed item {index} does not match the wire at offset {}", + wire.len() - rest.len(), + ); + rest = &rest[item.len()..]; + } + assert!( + rest.is_empty(), + "{} wire byte(s) beyond the last observed item", + rest.len(), + ); +} +/// Render both endpoints' hook captures without retaining cross-stream +/// order. +/// +/// Data streams are keyed by their labeled stream index — exact items +/// and order within each stream, stream groups sorted — discarding the +/// incidental order in which independent streams were opened. +pub fn render_hook_capture(a: &HookCapture, b: &HookCapture) -> String { let mut rendered = String::new(); - render_direction("A -> B", &a_control, a_streams.as_ref(), &mut rendered); + render_direction("A -> B", a, &mut rendered); rendered.push('\n'); - render_direction("B -> A", &b_control, b_streams.as_ref(), &mut rendered); + render_direction("B -> A", b, &mut rendered); rendered } -/// The control stream's fixed prefix, optional greeting frames, and trailing -/// session bytes. -struct Control { - preamble: Vec, - version_frame: Option>, - version: Option, - /// The version frame's leading word: the sender's advertised set size, - /// the role election's primary key. - set_len: Option, - /// The version frame's remaining size words: the sender's - /// version-size bound and target message size. - max_version_bytes: Option, - target_message_size: Option, - /// The greeting's second frame: the sender's root-fan listing. - listing_frame: Option>, - trailing: Vec, -} +/// Render one direction: its control items, then its data streams in +/// stream-index order. +fn render_direction(label: &str, capture: &HookCapture, out: &mut String) { + writeln!(out, "direction {label}").unwrap(); + if let Some(role) = capture.role { + writeln!(out, "role: {role:?}").unwrap(); + } + for (index, item) in capture.control.iter().enumerate() { + let name = control_item_name(item); + writeln!( + out, + "control item {index} ({} bytes) / {name} /", + item.len() + ) + .unwrap(); + render_item(item, " ", out); + } -impl Control { - /// Split one captured control direction at its exact fixed boundaries. - fn parse(bytes: &[u8]) -> Self { - assert!(bytes.len() >= PREAMBLE_LEN, "capture omitted the preamble"); - let (preamble, rest) = bytes.split_at(PREAMBLE_LEN); - if rest.is_empty() { - return Self { - preamble: preamble.to_vec(), - version_frame: None, - version: None, - set_len: None, - max_version_bytes: None, - target_message_size: None, - listing_frame: None, - trailing: Vec::new(), - }; + let mut streams = BTreeMap::new(); + for stream in &capture.streams { + let previous = streams.insert(stream.index, stream); + assert!(previous.is_none(), "duplicate captured stream index"); + } + for stream in streams.values() { + let speaker = speaker(stream.speaker); + let wire_stream = Stream::new(stream.index).expect("hook stream index names a stream"); + writeln!( + out, + "{:?} stream {} (height {}), epoch {}, {} wire bytes", + speaker, + stream.index, + wire_stream.height(speaker), + stream.epoch, + stream.wire_len, + ) + .unwrap(); + for (index, item) in stream.items.iter().enumerate() { + writeln!(out, " frame {index} ({} bytes)", item.len()).unwrap(); + render_frame(speaker, wire_stream, item, out); } + } +} - // A session that ends before its causal greeting (a mutual retire - // declining at the preamble) still closes with the one-byte session - // epilogue marker: control bytes too short to be a version frame - // header are that trailing marker, not a truncated frame. - if rest.len() < LENGTH_HEADER_LEN { - return Self { - preamble: preamble.to_vec(), - version_frame: None, - version: None, - set_len: None, - max_version_bytes: None, - target_message_size: None, - listing_frame: None, - trailing: rest.to_vec(), - }; - } - let (version_frame, rest) = split_frame(rest, "version"); - // The version frame's body leads with its three size words, - // decoded through the same framing helper the handshake reads - // them with; the version encoding follows them. - let (set_len, max_version_bytes, target_message_size) = - greeting_words(&version_frame[LENGTH_HEADER_LEN..]) - .expect("captured version frame carries its three size words"); - let version = - Version::decode(&version_frame[LENGTH_HEADER_LEN + GREETING_SIZE_WORDS_LEN..]) - .expect("captured version frame is canonical"); - // The greeting always carries its listing frame directly behind the - // version frame (empty tree = empty listing, still framed). - let (listing_frame, rest) = split_frame(rest, "listing"); - Self { - preamble: preamble.to_vec(), - version_frame: Some(version_frame), - version: Some(version), - set_len: Some(set_len), - max_version_bytes: Some(max_version_bytes), - target_message_size: Some(target_message_size), - listing_frame: Some(listing_frame), - trailing: rest.to_vec(), - } +/// The elected role, in the codec's speaker vocabulary. +fn speaker(role: Role) -> Speaker { + match role { + Role::Initiator => Speaker::Initiator, + Role::Responder => Speaker::Responder, } } -/// Split one exact length-delimited frame (header included) off `bytes`. -fn split_frame<'a>(bytes: &'a [u8], what: &str) -> (Vec, &'a [u8]) { - assert!( - bytes.len() >= LENGTH_HEADER_LEN, - "truncated {what} frame header" +/// Name one control item by its shape. +/// +/// The control stream's items are position- and shape-determined: the +/// self-described tag opens the preamble, the embedded-item tag wraps +/// the greeting, a tagged party atom is the identity hand-off, and the +/// dot text item is the epilogue. +fn control_item_name(item: &[u8]) -> &'static str { + let mut probe = item; + let Ok(head) = cbor::read_head(&mut probe) else { + panic!("captured control item opens with a canonical head"); + }; + match (head.major, head.value) { + (MAJOR_TAG, cbor::TAG_SELF_DESCRIBED) => "preamble", + (MAJOR_TAG, TAG_EMBEDDED_ITEM) => "greeting", + (MAJOR_TAG, crate::tags::PARTY_TAG) => "party hand-off", + (MAJOR_TEXT, _) => "epilogue", + _ => panic!("captured control item has no known shape"), + } +} + +/// Render one data frame. +/// +/// The codec's frame grammar (array head and signal) is held to +/// panics — a violation means the capture is broken — while the body +/// renders through the generic walk, falling back explicitly where it +/// cannot vouch for inversion. +fn render_frame(speaker: Speaker, stream: Stream, item: &[u8], out: &mut String) { + let mut probe = item; + let head = cbor::read_head(&mut probe).expect("captured frame head is canonical"); + assert_eq!(head.major, MAJOR_ARRAY, "captured frame is an array"); + let signal = cbor::read_head(&mut probe).expect("captured signal is canonical"); + assert_eq!( + signal.major, MAJOR_UINT, + "captured signal is an unsigned int" ); - let len = - u32::from_be_bytes(bytes[..LENGTH_HEADER_LEN].try_into().expect("header width")) as usize; - let frame_end = LENGTH_HEADER_LEN + len; - assert!(bytes.len() >= frame_end, "truncated {what} frame"); - (bytes[..frame_end].to_vec(), &bytes[frame_end..]) + let code = u8::try_from(signal.value).expect("captured signal is in the dense code space"); + let (framed, semantic) = WireSignal::from_byte(speaker, code) + .expect("captured signal is valid") + .into_parts(); + assert_eq!(framed, stream, "captured frame contradicts its label"); + + writeln!(out, " [").unwrap(); + writeln!(out, " {code} / {semantic:?} /").unwrap(); + let naming = match semantic { + Signal::Query(_) => Naming::Listing, + Signal::Supply(_) => Naming::Run, + _ => Naming::Plain, + }; + let mut rest = probe; + while !rest.is_empty() { + let remaining = rest; + match parse_node(&mut rest, 0) { + Ok(node) => render_node(&node, naming, " ", 0, out), + Err(reason) => { + fallback(remaining, &reason, " ", out); + rest = &[]; + } + } + } + writeln!(out, " ]").unwrap(); +} + +/// Render one whole captured item (a control item) as a value tree. +fn render_item(item: &[u8], indent: &str, out: &mut String) { + let mut rest = item; + match parse_node(&mut rest, 0) { + Ok(node) if rest.is_empty() => render_node(&node, Naming::Plain, indent, 0, out), + Ok(_) => panic!("captured control item carries trailing bytes"), + Err(reason) => panic!("captured control item is not canonical CBOR: {reason}"), + } } -/// One direction's exact data streams, keyed by their labeled stream index. -struct Streams { - speaker: Speaker, - streams: BTreeMap, +/// The naming context a subtree renders under. +/// +/// `Listing` annotates a map as a `{radix => digest}` listing (hex +/// radix keys, `/ digest /` value comments, an order check in the +/// block comment); `Run` names a supply body's embedded sequence a +/// *supply run* and `Record` names the run's items *records*, so a +/// re-accept diff speaks the protocol's own vocabulary. +#[derive(Clone, Copy, PartialEq, Eq)] +enum Naming { + Plain, + Listing, + Run, + Record, } -/// One captured data stream: its label and its exact frames. -struct CapturedStream { - epoch: u8, - frames: Vec, +/// One parsed CBOR value, canonical-head-checked, structure preserved +/// in wire order. +#[derive(Debug)] +enum Node { + Uint(u64), + /// A major-1 negative integer holding `n`, meaning `-(n + 1)`. + Nint(u64), + Bytes(Vec), + Text(String), + Array(Vec), + Map(Vec<(Node, Node)>), + Tag(u64, Box), + /// A major-7 simple value. + Simple(u8), + /// A major-7 float: its width byte (25, 26, or 27) and raw bits. + Float(u8, u64), } -impl Streams { - /// Decode every captured stream through its explicit end control. - fn parse(speaker: Speaker, streams: &[Vec]) -> Self { - let mut parsed = BTreeMap::new(); - for bytes in streams { - assert!( - bytes.len() >= LABEL_LEN, - "captured stream omitted its label" - ); - let (label, mut rest) = bytes.split_at(LABEL_LEN); - let [epoch, index] = label.try_into().expect("label width"); - let labeled = Stream::new(index).expect("captured label names a logical stream"); +/// Nesting past this bound falls back to exact hex: the walk never +/// recurses on unbounded input-controlled depth. +/// +/// One budget spans the whole walk — an embedded byte string's content +/// re-parses at the depth already consumed above it, never at a fresh +/// zero — so structural descent and embedded unfolds are bounded +/// together. +const MAX_DEPTH: usize = 64; - let mut frames = Vec::new(); - let mut ended = false; - while !ended { - let (stream, signal, consumed) = raw_frame(speaker, rest); - assert_eq!(stream, labeled, "captured frame contradicts its label"); - ended = matches!(signal, Signal::End(End::Stream)); - frames.push(CapturedFrame { - semantic: format!("{signal:?}"), - payload: payload_lines(&signal, &rest[..consumed]), - bytes: rest[..consumed].to_vec(), - }); - rest = &rest[consumed..]; +/// Parse one canonical item off the front of `input`. +/// +/// Head canonicality comes from the codec's own grammar; major-7 items +/// are handled here because float widths are semantic, not +/// shortest-form arithmetic. Any violation is a typed reason for the +/// caller's explicit fallback. +fn parse_node(input: &mut &[u8], depth: usize) -> Result { + if depth >= MAX_DEPTH { + return Err(format!("nested deeper than {MAX_DEPTH}")); + } + let Some(&initial) = input.first() else { + return Err("input ends before an item".into()); + }; + if initial >> 5 == 7 { + return parse_major_seven(input); + } + let head = cbor::read_head(input).map_err(|e| e.to_string())?; + match head.major { + MAJOR_UINT => Ok(Node::Uint(head.value)), + 1 => Ok(Node::Nint(head.value)), + MAJOR_BSTR => { + let bytes = take(input, head.value)?; + Ok(Node::Bytes(bytes.to_vec())) + } + MAJOR_TEXT => { + let bytes = take(input, head.value)?; + let text = std::str::from_utf8(bytes).map_err(|_| "invalid UTF-8".to_string())?; + Ok(Node::Text(text.to_string())) + } + MAJOR_ARRAY => { + let mut items = Vec::new(); + for _ in 0..head.value { + items.push(parse_node(input, depth + 1)?); } - assert!(rest.is_empty(), "captured bytes after the stream end"); - let previous = parsed.insert(labeled, CapturedStream { epoch, frames }); - assert!(previous.is_none(), "duplicate captured stream label"); + Ok(Node::Array(items)) } - Self { - speaker, - streams: parsed, + MAJOR_MAP => { + let mut entries = Vec::new(); + for _ in 0..head.value { + let key = parse_node(input, depth + 1)?; + let value = parse_node(input, depth + 1)?; + entries.push((key, value)); + } + Ok(Node::Map(entries)) } + MAJOR_TAG => Ok(Node::Tag( + head.value, + Box::new(parse_node(input, depth + 1)?), + )), + _ => unreachable!("majors 0 through 6 handled; 7 split off above"), } } -/// Parse one honest frame's boundary without decoding its supplied payload. -fn raw_frame(speaker: Speaker, bytes: &[u8]) -> (Stream, Signal, usize) { - let (&byte, body) = bytes.split_first().expect("captured stream ended early"); - let (stream, signal) = WireSignal::from_byte(speaker, byte) - .expect("captured signal is valid") - .into_parts(); - let body_len = match signal { - Signal::Match(_) | Signal::QueryEmpty(_) | Signal::End(_) => 0, - Signal::Query(_) => { - let (&count, _) = body.split_first().expect("captured query has a count"); - QUERY_COUNT_LEN + (usize::from(count) + QUERY_COUNT_BIAS) * QUERY_CHILD_LEN +/// Parse one major-7 item: simple values inline, one-byte simples with +/// their canonical floor, floats by width with exact bits. +fn parse_major_seven(input: &mut &[u8]) -> Result { + let (&initial, rest) = input.split_first().expect("caller peeked the initial byte"); + let info = initial & 0x1f; + match info { + 0..=23 => { + *input = rest; + Ok(Node::Simple(info)) } - Signal::Supply(_) => { - assert!( - body.len() >= LENGTH_HEADER_LEN, - "captured supply has a length" - ); - let len = - u32::from_be_bytes(body[..LENGTH_HEADER_LEN].try_into().expect("header width")); - LENGTH_HEADER_LEN + len as usize + 24 => { + let (&value, rest) = rest + .split_first() + .ok_or("input ends inside a simple value")?; + if value < 32 { + return Err("one-byte simple value below 32 is not canonical".into()); + } + *input = rest; + Ok(Node::Simple(value)) } - }; - let consumed = WireSignal::ENCODED_LEN + body_len; - assert!(bytes.len() >= consumed, "captured frame is truncated"); - (stream, signal, consumed) -} - -/// One semantically decoded frame and the exact bytes which produced it. -struct CapturedFrame { - semantic: String, - /// The frame's decoded payload tree (or its explicit decode - /// failure), one rendered line per entry; empty for payload-free - /// frames. - payload: Vec, - bytes: Vec, -} - -/// Decode one captured frame's peer-supplied payload into rendered lines. -/// -/// Match, empty-query, and end frames carry no payload. A query decodes -/// to its `(radix, hash)` children and a supply to its leaf-record run; -/// payload bytes that do not decode render as an explicit failure line — -/// the hex below them then stands as the witness, never as the only -/// account. -fn payload_lines(signal: &Signal, frame: &[u8]) -> Vec { - match signal { - Signal::Match(_) | Signal::QueryEmpty(_) | Signal::End(_) => Vec::new(), - // Signal byte, count byte, then the exact children — the frame - // boundary already validated the arithmetic. - Signal::Query(_) => query_lines(&frame[WireSignal::ENCODED_LEN + QUERY_COUNT_LEN..]), - // Signal byte and length header, then the run body. - Signal::Supply(_) => { - supply_lines(frame[WireSignal::ENCODED_LEN + LENGTH_HEADER_LEN..].to_vec()) + 25..=27 => { + let width = 1usize << (info - 24); + if rest.len() < width { + return Err("input ends inside a float".into()); + } + let (bytes, rest) = rest.split_at(width); + let mut bits = 0u64; + for &byte in bytes { + bits = bits << 8 | u64::from(byte); + } + *input = rest; + Ok(Node::Float(info, bits)) } + 28..=30 => Err("reserved additional-information value".into()), + _ => Err("indefinite-length CBOR is not canonical".into()), } } -/// Render a nonempty query's children: each child's radix and hash, -/// decoded through the codec's own `parse_query` (canonical child order -/// included), so the renderer cannot drift from what the decoder -/// accepts. -fn query_lines(children: &[u8]) -> Vec { - let children = match parse_query(children) { - Ok(children) => children, - Err(err) => { - return vec![format!( - "query undecodable ({err}); the exact bytes stand below" - )]; - } - }; - let mut lines = vec![format!("query: {} child(ren)", children.len())]; - for (radix, hash) in &children { - lines.push(format!(" child 0x{radix:x}: {}", hex::encode(hash.0))); +/// Split `len` payload bytes off `input`. +fn take<'a>(input: &mut &'a [u8], len: u64) -> Result<&'a [u8], String> { + let len = usize::try_from(len).map_err(|_| "length exceeds memory".to_string())?; + if input.len() < len { + return Err("input ends inside a string".into()); } - lines + let (bytes, rest) = input.split_at(len); + *input = rest; + Ok(bytes) } -/// Render a supply frame's leaf-record run: each record's version and -/// its message's byte count. +/// Render one node at `indent`, one line per scalar or bracket. /// -/// The message payload type belongs to the caller, so message bytes are -/// counted, not decoded — they remain exact in the hex below. A run -/// whose record framing or version encoding does not decode renders the -/// failure explicitly. -fn supply_lines(run: Vec) -> Vec { - let run = match LeafRun::from_encoded(run) { - Ok(run) => run, - Err(err) => { - return vec![format!( - "supply run undecodable ({err}); the exact bytes stand below" - )]; +/// `depth` is the walk's one nesting budget, shared with +/// [`parse_node`]: it counts structural levels descended since the +/// walk's entry point, and an embedded byte string's content re-parses +/// at the depth already consumed above it, so structural descent and +/// embedded unfolds are bounded by [`MAX_DEPTH`] together. Invariant +/// every `render_*` call site preserves: the `depth` passed is no +/// greater than the depth its node was parsed at — so a node in hand +/// always fits the remaining budget, and only [`parse_node`] need +/// check the bound. +fn render_node(node: &Node, naming: Naming, indent: &str, depth: usize, out: &mut String) { + match node { + Node::Map(entries) if naming == Naming::Listing => { + render_listing(entries, indent, depth, out); } - }; - let mut lines = vec![format!("supply run: {} record(s)", run.record_count())]; - for (index, record) in run.record_slices().enumerate() { - let mut input = record; - match ciborium::de::from_reader::(&mut input) { - Ok(version) => lines.push(format!( - " record {index}: version {version}, message {} byte(s)", - input.len(), - )), - Err(err) => lines.push(format!( - " record {index} undecodable ({err}); the exact bytes stand below" - )), + Node::Map(entries) => { + writeln!(out, "{indent}{{").unwrap(); + for (key, value) in entries { + // The one context-sensitive key: a map value under the + // text key "listing" is a `{radix => digest}` listing. + let value_naming = match key { + Node::Text(text) if text == "listing" => Naming::Listing, + _ => Naming::Plain, + }; + let key = scalar(key).unwrap_or_else(|| "…".into()); + match scalar(value) { + Some(value) => writeln!(out, "{indent} {key} => {value}").unwrap(), + None => { + writeln!(out, "{indent} {key} =>").unwrap(); + let deeper = format!("{indent} "); + render_node(value, value_naming, &deeper, depth + 1, out); + } + } + } + writeln!(out, "{indent}}}").unwrap(); + } + Node::Array(items) => { + writeln!(out, "{indent}[").unwrap(); + let deeper = format!("{indent} "); + for item in items { + render_node(item, Naming::Plain, &deeper, depth + 1, out); + } + writeln!(out, "{indent}]").unwrap(); + } + Node::Tag(number, content) => render_tag(*number, content, naming, indent, depth + 1, out), + scalar_node => { + let text = scalar(scalar_node).expect("non-container nodes render inline"); + writeln!(out, "{indent}{text}").unwrap(); } } - lines } -/// Render one root-fan listing frame's children, or its explicit decode -/// failure: the listing is peer-controlled bytes, so the renderer must -/// never present undecodable bytes as a quietly hex-only frame. -/// -/// The canonical child order is held by the codec's own -/// `validate_children`, the same rule the handshake applies before -/// building scope from a received listing. -fn listing_lines(body: &[u8]) -> Vec { - const RECORD: usize = 1 + crate::tree::typed::hash::MERKLE_HASH_LEN; - if !body.len().is_multiple_of(RECORD) { - return vec![format!( - "listing undecodable ({} bytes is not a whole number of radix-hash records); \ - the exact bytes stand below", - body.len() - )]; - } - let children: Vec<(u8, Hash)> = body - .chunks_exact(RECORD) - .map(|record| { - let (&radix, hash) = record.split_first().expect("a record has a radix byte"); - let mut bytes = [0u8; crate::tree::typed::hash::MERKLE_HASH_LEN]; - bytes.copy_from_slice(hash); - (radix, Hash(bytes)) +/// Render a `{radix => digest}` listing map: hex radix keys, digest +/// annotations, and an explicit order verdict when the wire's +/// strictly-ascending canonical form is violated. +fn render_listing(entries: &[(Node, Node)], indent: &str, depth: usize, out: &mut String) { + let ascending = entries + .windows(2) + .all(|pair| match (&pair[0].0, &pair[1].0) { + (Node::Uint(a), Node::Uint(b)) => a < b, + _ => false, }) - .collect(); - if let Err(err) = validate_children(&children) { - return vec![format!( - "listing not canonical ({err}); the exact bytes stand below" - )]; - } - let mut lines = vec![format!("listing: {} child(ren)", children.len())]; - for (radix, hash) in &children { - lines.push(format!(" child 0x{radix:x}: {}", hex::encode(hash.0))); + || entries.len() < 2; + let order = if ascending { + "" + } else { + ", NON-CANONICAL ORDER" + }; + writeln!( + out, + "{indent}{{ / listing: {} child(ren){order} /", + entries.len() + ) + .unwrap(); + for (key, value) in entries { + let key = match key { + Node::Uint(radix) => format!("0x{radix:x}"), + other => scalar(other).unwrap_or_else(|| "…".into()), + }; + match value { + Node::Bytes(bytes) => { + writeln!( + out, + "{indent} {key} => h'{}' / digest /", + hex::encode(bytes) + ) + .unwrap(); + } + other => match scalar(other) { + Some(text) => writeln!(out, "{indent} {key} => {text}").unwrap(), + None => { + writeln!(out, "{indent} {key} =>").unwrap(); + let deeper = format!("{indent} "); + render_node(other, Naming::Plain, &deeper, depth + 1, out); + } + }, + } } - lines + writeln!(out, "{indent}}}").unwrap(); } -/// Render one physical direction in stable logical order. -fn render_direction(label: &str, control: &Control, streams: Option<&Streams>, out: &mut String) { - writeln!(out, "direction {label}").unwrap(); - render_block("preamble", &control.preamble, out); - if let Some(version) = &control.version { - writeln!(out, "version: {version}").unwrap(); - writeln!( - out, - "greeting words: set len {}, version-size bound {}, message-size target {}", - control - .set_len - .expect("a version frame carries its set size"), - control - .max_version_bytes - .expect("a version frame carries its version-size bound"), - control - .target_message_size - .expect("a version frame carries its message-size target"), - ) - .unwrap(); - render_block( - "version frame", - control.version_frame.as_deref().expect("version frame"), - out, - ); - let listing_frame = control.listing_frame.as_deref().expect("listing frame"); - for line in listing_lines(&listing_frame[LENGTH_HEADER_LEN..]) { - writeln!(out, "{line}").unwrap(); +/// Render one tagged node, unfolding embedded byte strings and +/// annotating the tags the protocol names. +/// +/// `depth` is the tag's *content* depth — the caller already counted +/// the tag's own structural level — and passes through unchanged. +fn render_tag( + number: u64, + content: &Node, + naming: Naming, + indent: &str, + depth: usize, + out: &mut String, +) { + match (number, content) { + (TAG_CBOR_SEQUENCE, Node::Bytes(bytes)) => { + let (name, inner) = match naming { + Naming::Run => ("supply run", Naming::Record), + Naming::Record => ("record", Naming::Plain), + _ => ("embedded sequence", Naming::Plain), + }; + render_embedded_as(number, name, inner, bytes, indent, depth, out); } - render_block("listing frame", listing_frame, out); - } - - if let Some(streams) = streams { - for (stream, captured) in &streams.streams { + (TAG_EMBEDDED_ITEM, Node::Bytes(bytes)) => { + render_embedded(number, "embedded item", bytes, indent, depth, out); + } + (crate::tags::VERSION_TAG, Node::Bytes(bytes)) => { + let meaning = match Version::decode(&bytes[..]) { + // The rendering is the version's whole ITC event tree in + // paper notation, never a scalar: a flat tree renders as + // its single uniform height (e.g. `3`), a forked one as + // the nested `(n, e1, e2)` form. + Ok(version) => format!("causal version, event tree: {version}"), + Err(e) => format!("causal version undecodable: {e}"), + }; writeln!( out, - "{:?} stream {} (height {}), epoch {}", - streams.speaker, - stream.index(), - stream.height(streams.speaker), - captured.epoch, + "{indent}{number}(h'{}') / {meaning} /", + hex::encode(bytes) ) .unwrap(); - for (index, frame) in captured.frames.iter().enumerate() { - writeln!(out, " frame {index}: {}", frame.semantic).unwrap(); - for line in &frame.payload { - writeln!(out, " {line}").unwrap(); - } - render_hex(&frame.bytes, " ", out); - } } - } - if !control.trailing.is_empty() { - render_block("trailing frame", &control.trailing, out); + (crate::tags::PARTY_TAG, Node::Bytes(bytes)) => { + writeln!(out, "{indent}{number}(h'{}') / party /", hex::encode(bytes)).unwrap(); + } + (crate::tags::CLOCK_TAG, Node::Bytes(bytes)) => { + writeln!(out, "{indent}{number}(h'{}') / clock /", hex::encode(bytes)).unwrap(); + } + (cbor::TAG_SELF_DESCRIBED, _) => { + writeln!(out, "{indent}{number}( / self-described CBOR /").unwrap(); + let deeper = format!("{indent} "); + render_node(content, naming, &deeper, depth, out); + writeln!(out, "{indent})").unwrap(); + } + (_, scalar_content) if scalar(scalar_content).is_some() => { + let text = scalar(scalar_content).expect("checked by the guard"); + writeln!(out, "{indent}{number}({text})").unwrap(); + } + _ => { + writeln!(out, "{indent}{number}(").unwrap(); + let deeper = format!("{indent} "); + render_node(content, naming, &deeper, depth, out); + writeln!(out, "{indent})").unwrap(); + } } } -/// Render one named exact byte block. -fn render_block(label: &str, bytes: &[u8], out: &mut String) { - writeln!(out, "{label}: {} bytes", bytes.len()).unwrap(); - render_hex(bytes, " ", out); +/// Unfold one embedded byte string (tag 24 or 63) as its parsed +/// item sequence, falling back to exact hex when the content is not +/// wholly canonical CBOR or when the walk's depth budget is spent. +fn render_embedded( + number: u64, + name: &str, + bytes: &[u8], + indent: &str, + depth: usize, + out: &mut String, +) { + render_embedded_as(number, name, Naming::Plain, bytes, indent, depth, out); } -/// Render stable eight-byte hexdump lines with a caller-selected indent. -fn render_hex(bytes: &[u8], indent: &str, out: &mut String) { - for (line, chunk) in bytes.chunks(8).enumerate() { - write!(out, "{indent}{:04x}:", line * 8).unwrap(); - for byte in chunk { - write!(out, " {byte:02x}").unwrap(); +/// [`render_embedded`], with the naming context the unfolded items +/// render under (a supply run's items are records). +/// +/// The content re-parses at `depth` — the budget already consumed +/// above this byte string — so a chain of embedded byte strings draws +/// down the same [`MAX_DEPTH`] bound as structural nesting, and spends +/// it here as the too-deep fallback. +fn render_embedded_as( + number: u64, + name: &str, + inner: Naming, + bytes: &[u8], + indent: &str, + depth: usize, + out: &mut String, +) { + let mut items = Vec::new(); + let mut rest = bytes; + let mut failure = None; + while !rest.is_empty() { + match parse_node(&mut rest, depth) { + Ok(node) => items.push(node), + Err(reason) => { + failure = Some(reason); + break; + } } - out.push('\n'); } + if let Some(reason) = failure { + writeln!(out, "{indent}{number}( / {name}, {} bytes /", bytes.len()).unwrap(); + fallback(bytes, &reason, &format!("{indent} "), out); + writeln!(out, "{indent})").unwrap(); + return; + } + if number == TAG_EMBEDDED_ITEM && items.len() != 1 { + writeln!(out, "{indent}{number}( / {name}, {} bytes /", bytes.len()).unwrap(); + fallback( + bytes, + &format!("embedded item holds {} items", items.len()), + &format!("{indent} "), + out, + ); + writeln!(out, "{indent})").unwrap(); + return; + } + let count = match (number, inner) { + (TAG_CBOR_SEQUENCE, Naming::Record) => format!(", {} record(s)", items.len()), + (TAG_CBOR_SEQUENCE, _) => format!(", {} item(s)", items.len()), + _ => String::new(), + }; + writeln!( + out, + "{indent}{number}(<< / {name}{count}, {} bytes /", + bytes.len() + ) + .unwrap(); + let deeper = format!("{indent} "); + for item in &items { + render_node(item, inner, &deeper, depth, out); + } + writeln!(out, "{indent}>>)").unwrap(); +} + +/// Render one scalar node inline, or `None` for containers. +fn scalar(node: &Node) -> Option { + Some(match node { + Node::Uint(value) => format!("{value}"), + Node::Nint(value) => format!("-{}", u128::from(*value) + 1), + Node::Bytes(bytes) => format!("h'{}'", hex::encode(bytes)), + Node::Text(text) => format!("{text:?}"), + Node::Simple(20) => "false".into(), + Node::Simple(21) => "true".into(), + Node::Simple(22) => "null".into(), + Node::Simple(23) => "undefined".into(), + Node::Simple(value) => format!("simple({value})"), + Node::Float(25, bits) => format!("float16'{bits:04x}'"), + Node::Float(26, bits) => format!("float32'{bits:08x}'"), + Node::Float(_, bits) => format!("float64'{bits:016x}'"), + Node::Array(_) | Node::Map(_) => return None, + // Tags the protocol names always render through the block path, + // so their annotations cannot be skipped by an inline rendering. + Node::Tag( + TAG_CBOR_SEQUENCE + | TAG_EMBEDDED_ITEM + | cbor::TAG_SELF_DESCRIBED + | crate::tags::PARTY_TAG + | crate::tags::VERSION_TAG + | crate::tags::CLOCK_TAG, + _, + ) => return None, + Node::Tag(number, content) => format!("{number}({})", scalar(content)?), + }) +} + +/// Render an explicit walk failure above the exact bytes it convicts: +/// the fallback that keeps the rendering injective where the generic +/// walk cannot vouch for inversion. +fn fallback(bytes: &[u8], reason: &str, indent: &str, out: &mut String) { + writeln!( + out, + "{indent}!! not rendered as CBOR ({reason}); the exact bytes stand here:" + ) + .unwrap(); + writeln!(out, "{indent}h'{}'", hex::encode(bytes)).unwrap(); } diff --git a/src/tree/mirror/streaming/remote/codec/capture/tests.rs b/src/tree/mirror/streaming/remote/codec/capture/tests.rs index 35b2fb32b..c7c4c1518 100644 --- a/src/tree/mirror/streaming/remote/codec/capture/tests.rs +++ b/src/tree/mirror/streaming/remote/codec/capture/tests.rs @@ -1,172 +1,354 @@ -//! The capture renderer's payload-decoding pins. +//! The reflection renderer's pins. //! -//! Two commitments: the decoded parse tree names the semantic field a -//! snapshot re-accept moved (the committed fixture pair below differs in -//! exactly one field and exactly one rendered line), and payload bytes -//! that do not decode render as an explicit failure line, never as -//! silent hex. +//! Three commitments: the rendered value tree localizes the semantic +//! field a snapshot re-accept moved to exactly one rendered line +//! carrying the exact value (its surrounding vocabulary is the wire +//! snapshots' to pin), bytes the walk cannot vouch for render as +//! an explicit failure above their exact hex (never as a silently pretty +//! tree, never as silent omission), and the totality witness +//! ([`assert_items_account_for`]) refuses any gap between observed items +//! and wire bytes. use super::*; -use crate::message::Message; +use crate::message::{Message, PayloadDepthLimit}; +use crate::tree::typed::Hash; use crate::tree::typed::hash::MERKLE_HASH_LEN; -use super::super::frame::LeafRun; +use super::super::encode::encode; +use super::super::frame::{Frame, LeafRun, Reaction, write_listing}; +use super::super::signal::Flow; -/// Encode a listing as its wire form: raw radix-hash records. -fn encode_listing(children: &[(u8, Hash)]) -> Vec { - let mut body = Vec::new(); - for (radix, hash) in children { - body.push(*radix); - body.extend_from_slice(hash.as_bytes()); - } - body +/// Render one embedded run (the supply body's tag-63 content) to lines. +fn run_lines(run: &LeafRun) -> Vec { + let mut out = String::new(); + render_embedded( + TAG_CBOR_SEQUENCE, + "embedded sequence", + run.as_bytes(), + "", + 0, + &mut out, + ); + out.lines().map(str::to_string).collect() } -/// The capture renderer decodes each supply record structurally. +/// Two supply runs differing only in one record's version render line +/// sets that differ in exactly one line, and that line carries the +/// version's exact rendering: the field-level account an insta +/// re-accept diff shows. /// -/// Two runs differing only in the record's version render record lines -/// that differ exactly at the line naming that record, with the record -/// count and the (identical) payload accounting unchanged: the -/// field-level account an insta re-accept shows beside the hex. +/// Containment, not equality: the record's version-addressed hash +/// moves on the same line. The annotation's surrounding vocabulary is +/// deliberately not asserted here; the wire snapshots pin it, and a +/// reviewer judges its changes at re-accept. #[test] -fn supply_decode_names_the_field_that_moved() { - let party = before::Party::seed(); +fn supply_reflection_localizes_the_field_that_moved() { + let mut party = before::Party::seed(); + let other = party.fork(); let mut low = Version::new(); low.tick(&party); + low.tick(&other); + low.tick(&other); let mut high = low.clone(); - high.tick(&party); + high.tick(&other); let render = |version: &Version| { let mut run = LeafRun::new(); run.push(version, &Message::new(7_u64)) .expect("one small record fits any run"); - supply_lines(run.as_bytes().to_vec()) + run_lines(&run) }; let a = render(&low); let b = render(&high); - assert_eq!(a.len(), 2, "one header line and one record line"); - assert_eq!(a[0], b[0], "the record count did not move"); - assert_ne!(a[1], b[1], "the record line names the moved field"); - assert!(a[1].contains(&format!("version {low}"))); - assert!(b[1].contains(&format!("version {high}"))); - // The message is identical on both sides, so both record lines - // account it identically: one CBOR byte for the small u64. - assert!(a[1].ends_with("message 1 byte(s)")); - assert!(b[1].ends_with("message 1 byte(s)")); -} - -/// Unparseable supply payloads render an explicit decode failure, never -/// silent hex. -/// -/// A run with broken record framing convicts the whole run, and a -/// structurally framed record whose version bytes do not decode -/// convicts that record by index. -#[test] -fn undecodable_supply_renders_failure_not_silent_hex() { - // A record header promising more bytes than the run carries. - let torn = vec![0, 0, 0, 9, 1, 2, 3]; - let lines = supply_lines(torn); - assert_eq!(lines.len(), 1); + assert_eq!(a.len(), b.len(), "one field moved, no line appeared"); + let diffs: Vec<_> = a.iter().zip(&b).filter(|(a, b)| a != b).collect(); + assert_eq!(diffs.len(), 1, "exactly one rendered line moved: {diffs:?}"); + let (a_line, b_line) = diffs[0]; + // The unbalanced ticks across the fork keep the event tree + // non-flat, so its rendering carries punctuation neither hex nor a + // tag digit can spell: containment cannot match vacuously inside + // the line's other tokens. + let low_text = low.to_string(); + let high_text = high.to_string(); assert!( - lines[0].contains("supply run undecodable"), - "torn framing must convict the run: {lines:?}" + low_text.contains('('), + "the fixture is non-flat: {low_text}" + ); + assert!( + a_line.contains(&low_text), + "the moved line carries the exact version rendering: {a_line}" ); - - // Valid record framing around bytes that are no version encoding. - let mut framed = vec![0, 0, 0, 3]; - framed.extend_from_slice(&[0xff, 0xff, 0xff]); - let lines = supply_lines(framed); - assert_eq!(lines[0], "supply run: 1 record(s)"); assert!( - lines[1].contains("record 0 undecodable"), - "a garbage version must convict its record: {lines:?}" + b_line.contains(&high_text), + "the moved line carries the exact version rendering: {b_line}" ); + // The identical payload renders identically as its own line: the + // small u64 is a bare CBOR int, visible directly. + assert!(a.iter().any(|line| line.trim() == "7"), "{a:?}"); } -/// The greeting's root-fan listing decodes to one line per child naming -/// its radix and full hash; bytes that are not a canonical listing -/// render the explicit failure instead. +/// Embedded content the walk cannot vouch for renders as an explicit +/// failure line above the exact bytes, never as silence or a partial +/// tree presented as whole. #[test] -fn listing_decodes_children_and_convicts_garbage() { - let children = vec![ - (0x3_u8, Hash([0xab; MERKLE_HASH_LEN])), - (0xc_u8, Hash([0x01; MERKLE_HASH_LEN])), - ]; - let body = encode_listing(&children); - let lines = listing_lines(&body); - assert_eq!(lines[0], "listing: 2 child(ren)"); - assert_eq!( - lines[1], - format!(" child 0x3: {}", "ab".repeat(MERKLE_HASH_LEN)) +fn undecodable_embedded_content_falls_back_explicitly_to_hex() { + // 0xf8 0x05: a one-byte simple value below 32 is not canonical. + let garbage = [0xf8, 0x05]; + let mut out = String::new(); + render_embedded( + TAG_CBOR_SEQUENCE, + "embedded sequence", + &garbage, + "", + 0, + &mut out, ); - assert_eq!( - lines[2], - format!(" child 0xc: {}", "01".repeat(MERKLE_HASH_LEN)) + assert!( + out.contains("!! not rendered as CBOR"), + "the failure is explicit: {out}" ); - - let truncated = &body[..body.len() - 1]; - let lines = listing_lines(truncated); - assert_eq!(lines.len(), 1); assert!( - lines[0].contains("listing undecodable"), - "a truncated listing must convict itself: {lines:?}" + out.contains(&format!("h'{}'", hex::encode(garbage))), + "the exact bytes stand: {out}" ); } -/// A nonempty query's children decode to one line per child naming its -/// radix and full hash, in wire order. +/// A tag-24 embedded item holding anything but exactly one item falls +/// back explicitly: an embedded *item* is one item by definition. #[test] -fn query_children_decode_to_radix_and_hash() { - let mut children = Vec::new(); - children.push(0x0_u8); - children.extend_from_slice(&[0x22; MERKLE_HASH_LEN]); - children.push(0xf_u8); - children.extend_from_slice(&[0x9d; MERKLE_HASH_LEN]); - let lines = query_lines(&children); - assert_eq!(lines[0], "query: 2 child(ren)"); - assert_eq!( - lines[1], - format!(" child 0x0: {}", "22".repeat(MERKLE_HASH_LEN)) - ); - assert_eq!( - lines[2], - format!(" child 0xf: {}", "9d".repeat(MERKLE_HASH_LEN)) - ); +fn embedded_item_with_two_items_falls_back() { + let mut bytes = Vec::new(); + cbor::write_head(&mut bytes, MAJOR_UINT, 1); + cbor::write_head(&mut bytes, MAJOR_UINT, 2); + let mut out = String::new(); + render_embedded(TAG_EMBEDDED_ITEM, "embedded item", &bytes, "", 0, &mut out); + assert!(out.contains("holds 2 items"), "{out}"); + assert!(out.contains("!! not rendered as CBOR"), "{out}"); } -/// The renderer decodes queries through the codec's own path, canonical -/// child order included: descending radixes convict the frame with an -/// explicit failure line, never silent hex. +/// A listing map renders each child as a hex radix and an annotated +/// digest, and the block comment carries the child count. #[test] -fn non_canonical_query_renders_failure_not_silent_hex() { - let mut children = Vec::new(); - children.push(0xf_u8); - children.extend_from_slice(&[0x9d; MERKLE_HASH_LEN]); - children.push(0x0_u8); - children.extend_from_slice(&[0x22; MERKLE_HASH_LEN]); - let lines = query_lines(&children); - assert_eq!(lines.len(), 1); +fn listing_renders_children_with_digest_annotations() { + let mut bytes = Vec::new(); + write_listing( + &mut bytes, + &[ + (0x3_u8, Hash([0xab; MERKLE_HASH_LEN])), + (0xc_u8, Hash([0x01; MERKLE_HASH_LEN])), + ], + ); + let mut input = bytes.as_slice(); + let node = parse_node(&mut input, 0).expect("the codec writes canonical listings"); + let mut out = String::new(); + render_node(&node, Naming::Listing, "", 0, &mut out); + assert!(out.contains("/ listing: 2 child(ren) /"), "{out}"); assert!( - lines[0].contains("query undecodable"), - "descending children must convict the query: {lines:?}" + out.contains(&format!( + "0x3 => h'{}' / digest /", + "ab".repeat(MERKLE_HASH_LEN) + )), + "{out}" ); + assert!( + out.contains(&format!( + "0xc => h'{}' / digest /", + "01".repeat(MERKLE_HASH_LEN) + )), + "{out}" + ); + assert!(!out.contains("NON-CANONICAL"), "{out}"); } -/// The listing is held to the same canonical child order the handshake -/// enforces before building scope from it: out-of-order children render -/// an explicit conviction, never a quietly decoded tree. +/// A listing whose radixes are not strictly ascending renders with an +/// explicit order verdict: the violation is visible in the transcript, +/// and the entries still render completely, in wire order. #[test] -fn non_canonical_listing_renders_failure_not_silent_hex() { - let children = vec![ - (0xc_u8, Hash([0x01; MERKLE_HASH_LEN])), - (0x3_u8, Hash([0xab; MERKLE_HASH_LEN])), - ]; - let body = encode_listing(&children); - let lines = listing_lines(&body); - assert_eq!(lines.len(), 1); +fn descending_listing_renders_an_order_verdict() { + let mut bytes = Vec::new(); + cbor::write_head(&mut bytes, MAJOR_MAP, 2); + for radix in [0xf_u8, 0x0] { + cbor::write_head(&mut bytes, MAJOR_UINT, u64::from(radix)); + cbor::write_head(&mut bytes, MAJOR_BSTR, MERKLE_HASH_LEN as u64); + bytes.extend_from_slice(&[radix; MERKLE_HASH_LEN]); + } + let mut input = bytes.as_slice(); + let node = parse_node(&mut input, 0).expect("heads are canonical; order is not"); + let mut out = String::new(); + render_node(&node, Naming::Listing, "", 0, &mut out); + assert!(out.contains("NON-CANONICAL ORDER"), "{out}"); + assert!(out.contains("0xf =>"), "first entry renders: {out}"); + assert!(out.contains("0x0 =>"), "second entry renders: {out}"); +} + +/// A version-tagged byte string whose bytes are no version encoding +/// annotates the failure rather than inventing a meaning. +#[test] +fn garbage_version_atom_annotates_undecodable() { + let mut bytes = Vec::new(); + cbor::write_tag(&mut bytes, crate::tags::VERSION_TAG); + cbor::write_head(&mut bytes, MAJOR_BSTR, 3); + bytes.extend_from_slice(&[0xff, 0xff, 0xff]); + let mut input = bytes.as_slice(); + let node = parse_node(&mut input, 0).expect("the wrapper is canonical"); + let mut out = String::new(); + render_node(&node, Naming::Plain, "", 0, &mut out); + assert!(out.contains("causal version undecodable"), "{out}"); + assert!(out.contains("h'ffffff'"), "the atom bytes stand: {out}"); +} + +/// The totality witness accepts exactly the wire it was given, split at +/// any item boundaries. +#[test] +fn items_accounting_accepts_the_exact_wire() { + let wire = [1_u8, 2, 3, 4, 5]; + assert_items_account_for(&[vec![1, 2], vec![3], vec![4, 5]], &wire); + assert_items_account_for(&[vec![1, 2, 3, 4, 5]], &wire); + assert_items_account_for(&[], &[]); +} + +/// A wire byte no observed item accounts for is refused. +#[test] +#[should_panic(expected = "beyond the last observed item")] +fn items_accounting_refuses_unobserved_bytes() { + assert_items_account_for(&[vec![1, 2]], &[1, 2, 3]); +} + +/// An observed item the wire does not carry is refused. +#[test] +#[should_panic(expected = "does not match the wire")] +fn items_accounting_refuses_diverging_items() { + assert_items_account_for(&[vec![1, 9]], &[1, 2]); +} + +/// The stream label parses to its epoch and index, and reports its +/// exact byte length. +#[test] +fn stream_label_parses_epoch_and_index() { + let mut bytes = Vec::new(); + cbor::write_head(&mut bytes, MAJOR_UINT, 1); + cbor::write_head(&mut bytes, MAJOR_UINT, 200); + bytes.push(0xee); + let ((epoch, index), len) = stream_label(&bytes); + assert_eq!((epoch, index), (1, 200)); + assert_eq!(len, 3, "one short head and one byte-argument head"); +} + +/// Control items are named by their shape; an unknown shape is a broken +/// capture, not a renderable one. +#[test] +fn control_items_are_named_by_shape() { + let mut preamble = Vec::new(); + cbor::write_tag(&mut preamble, cbor::TAG_SELF_DESCRIBED); + assert_eq!(control_item_name(&preamble), "preamble"); + + let mut greeting = Vec::new(); + cbor::write_tag(&mut greeting, TAG_EMBEDDED_ITEM); + assert_eq!(control_item_name(&greeting), "greeting"); + + let mut party = Vec::new(); + cbor::write_tag(&mut party, crate::tags::PARTY_TAG); + assert_eq!(control_item_name(&party), "party hand-off"); + + let mut epilogue = Vec::new(); + cbor::write_head(&mut epilogue, cbor::MAJOR_TEXT, 1); + epilogue.push(b'.'); + assert_eq!(control_item_name(&epilogue), "epilogue"); +} + +/// Nesting past the walk's depth bound falls back explicitly instead of +/// recursing without bound on input-controlled depth. +#[test] +fn nesting_past_the_depth_bound_falls_back() { + let mut bytes = Vec::new(); + for _ in 0..=MAX_DEPTH { + cbor::write_head(&mut bytes, MAJOR_ARRAY, 1); + } + cbor::write_head(&mut bytes, MAJOR_UINT, 0); + let mut input = bytes.as_slice(); + let error = parse_node(&mut input, 0).expect_err("too deep to vouch for"); + assert!(error.contains("deeper than"), "{error}"); +} + +/// Build a chain of `levels` nested embedded byte strings — each level +/// one tag-24 item wrapping the next level's encoding as a byte +/// string — bottoming out at a single `0x00` uint. +/// +/// Written outside-in: encoded lengths follow the recurrence +/// `len[0] = 1` (the innermost uint) and +/// `len[i + 1] = tag head (2 bytes) + byte-string head + len[i]`, so +/// every head is computed before any byte is emitted and the build is +/// linear in the output size. +fn embedded_chain(levels: usize) -> Vec { + let mut lens = vec![1_usize]; + for _ in 0..levels { + let inner = *lens.last().expect("the list starts nonempty"); + lens.push(2 + cbor::head_len(inner as u64) + inner); + } + let mut bytes = Vec::with_capacity(lens[levels]); + for &len in lens[..levels].iter().rev() { + cbor::write_tag(&mut bytes, TAG_EMBEDDED_ITEM); + cbor::write_head(&mut bytes, MAJOR_BSTR, len as u64); + } + cbor::write_head(&mut bytes, MAJOR_UINT, 0); + bytes +} + +/// The rendered output carries the depth fallback: the explicit +/// too-deep failure line, with the convicted bytes standing as hex on +/// the line below it. +fn assert_depth_fallback(out: &str) { assert!( - lines[0].contains("listing not canonical"), - "descending children must convict the listing: {lines:?}" + out.contains(&format!("nested deeper than {MAX_DEPTH}")), + "the depth fallback is explicit: {out}" ); + let fallback_hex = out + .lines() + .skip_while(|line| !line.contains("nested deeper than")) + .nth(1) + .unwrap_or_default(); + assert!( + fallback_hex.trim_start().starts_with("h'"), + "the exact bytes stand under the failure line: {fallback_hex:?}" + ); +} + +/// A chain of embedded byte strings nested far past the depth bound +/// renders to the explicit depth fallback instead of overflowing the +/// stack. +/// +/// The walk's one depth budget spans embedded-byte-string re-parses, +/// so no input-controlled nesting recurses without bound. +#[test] +fn deep_embedded_chain_falls_back_instead_of_recursing() { + let bytes = embedded_chain(10 * MAX_DEPTH); + let mut out = String::new(); + render_item(&bytes, "", &mut out); + assert_depth_fallback(&out); +} + +/// The same nesting arriving as a supply record's payload — the +/// harness-reachable path, a captured frame rendered whole — hits the +/// same depth fallback. +/// +/// An application payload of legal CBOR can nest arbitrarily, and the +/// frame walk must return, never overflow. +#[test] +fn deep_payload_through_the_frame_path_falls_back() { + let payload = Message::from_slice::( + &embedded_chain(10 * MAX_DEPTH), + PayloadDepthLimit::default(), + ) + .expect("the chain is exactly one CBOR item"); + let mut run = LeafRun::new(); + run.push(&Version::new(), &payload) + .expect("one record fits a fresh run"); + let stream = Stream::new(0).expect("stream 0 names a stream"); + let frame = (stream, Frame::Reaction(Reaction::Supply(run), Flow::End)); + let mut bytes = Vec::new(); + encode(Speaker::Initiator, &frame, &mut bytes).expect("a supply frame encodes"); + let mut out = String::new(); + render_frame(Speaker::Initiator, stream, &bytes, &mut out); + assert_depth_fallback(&out); } diff --git a/src/tree/mirror/streaming/remote/codec/decode.rs b/src/tree/mirror/streaming/remote/codec/decode.rs index 9b1b94fc8..037ba98f2 100644 --- a/src/tree/mirror/streaming/remote/codec/decode.rs +++ b/src/tree/mirror/streaming/remote/codec/decode.rs @@ -1,13 +1,9 @@ //! Self-delimiting frame decoding. -#[cfg(test)] -use std::slice; - #[cfg(test)] use std::io::{ErrorKind, Read}; -use crate::tree::mirror::framing::LENGTH_HEADER_LEN; -use crate::tree::typed::{Hash, hash::MERKLE_HASH_LEN}; +use crate::tree::mirror::cbor::{self, HeadReadError, MAJOR_ARRAY, MAJOR_UINT}; mod async_io; @@ -16,15 +12,14 @@ pub use async_io::FrameRead; #[cfg(test)] use super::budget::RunBudget; #[cfg(test)] +use super::frame::{Frame, LeafRun, Reaction, WireFrame}; use super::{ - error::FramePart, - frame::{Frame, LeafRun, QUERY_COUNT_BIAS, Reaction, WireFrame}, -}; -use super::{ - error::{DecodeError, DecodeErrorKind}, - frame::{QUERY_CHILD_LEN, validate_children}, + error::{DecodeError, DecodeErrorKind, FramePart}, + frame::ListingIssue, signal::{Signal, Speaker, Stream, WireSignal}, }; +#[cfg(test)] +use crate::tree::typed::{Hash, hash::MERKLE_HASH_LEN}; /// Decode one frame from `read`, leaving subsequent bytes untouched. #[cfg(test)] @@ -76,18 +71,44 @@ impl<'a, R: Read> FrameDecoder<'a, R> { } fn decode(mut self) -> Result { + let arity = self + .arity() + .map_err(|kind| DecodeError::direction(self.speaker, kind))?; let (stream, signal) = self.signal()?; - let frame = self - .body(signal) + let frame = check_arity(signal, arity) + .and_then(|()| self.body(signal)) .map_err(|kind| DecodeError::stream(self.speaker, stream, kind))?; Ok((stream, frame)) } + /// Read the frame's array head; this oracle treats a clean close as a + /// truncation, since its callers always expect a frame. + fn arity(&mut self) -> Result { + let head = cbor::read_head_io(self.read) + .map_err(|e| head_error(FramePart::FrameHead, e))? + .ok_or_else(|| { + head_error( + FramePart::FrameHead, + HeadReadError::Io(ErrorKind::UnexpectedEof.into()), + ) + })?; + frame_arity(head) + } + fn signal(&mut self) -> Result<(Stream, Signal), DecodeError> { - let byte = self - .byte(FramePart::Signal) + let head = cbor::read_head_io(self.read) + .map_err(|e| head_error(FramePart::Signal, e)) + .and_then(|head| { + head.ok_or_else(|| { + head_error( + FramePart::Signal, + HeadReadError::Io(ErrorKind::UnexpectedEof.into()), + ) + }) + }) .map_err(|kind| DecodeError::direction(self.speaker, kind))?; - decode_signal(self.speaker, byte) + let code = signal_code(head).map_err(|kind| DecodeError::direction(self.speaker, kind))?; + decode_signal(self.speaker, code) } fn body(&mut self, signal: Signal) -> Result { @@ -102,41 +123,50 @@ impl<'a, R: Read> FrameDecoder<'a, R> { } fn query(&mut self) -> Result, DecodeErrorKind> { - let count = usize::from(self.byte(FramePart::QueryCount)?) + QUERY_COUNT_BIAS; - // One bulk read for the whole listing rather than one call per child. - let mut listing = vec![0; count * QUERY_CHILD_LEN]; - self.read_exact(&mut listing, FramePart::QueryChildren)?; - - parse_query(&listing) + let head = self.head(FramePart::QueryChildren)?; + let mut listing = query_listing(head)?; + let count = head.value; + for _ in 0..count { + let key = self.head(FramePart::QueryChildren)?; + let radix = listing.key(key).map_err(listing_issue)?; + let value = self.head(FramePart::QueryChildren)?; + super::frame::ListingBuilder::value_head(value).map_err(listing_issue)?; + let mut digest = [0; MERKLE_HASH_LEN]; + self.read_exact(&mut digest, FramePart::QueryChildren)?; + listing.entry(radix, digest); + } + Ok(listing.finish()) } fn supply(&mut self) -> Result { - let mut header = [0; LENGTH_HEADER_LEN]; - self.read_exact(&mut header, FramePart::SupplyLength)?; - let len = u32::from_be_bytes(header) as usize; + let tag = self.head(FramePart::SupplyLength)?; + let body = self.head(FramePart::SupplyLength)?; + let len = run_head(tag, body)?; // The run-budget ingress check, mirroring the async reader's exactly // (see `AsyncFrameDecoder::supply` for the memory argument this // oracle does not need): an over-budget frame is legal only as one // lone record spanning the whole body, decided from the first - // record's length header alone. + // record's heads alone. if !self.budget.covers(len) { let budget = self.budget; let overbatched = move || DecodeErrorKind::OverbatchedRun { declared: super::budget::SUPPLY_FRAME_OVERHEAD.saturating_add(len), budget: budget.bytes(), }; - if len < LENGTH_HEADER_LEN { + // A body too short to hold a record's heads cannot be a lone + // record: rejected on the declared length alone. + if len < super::frame::RECORD_TAG_LEN + 1 { return Err(overbatched()); } - let mut first = [0; LENGTH_HEADER_LEN]; - self.read_exact(&mut first, FramePart::SupplyRun)?; - let record = u32::from_be_bytes(first) as usize; - if !lone_record_spans(len, record) { + let Some((prefix, record)) = self.record_prefix()? else { + return Err(overbatched()); + }; + if !super::frame::lone_record_spans(len, record) { return Err(overbatched()); } let mut run = vec![0; len]; - run[..LENGTH_HEADER_LEN].copy_from_slice(&first); - self.read_exact(&mut run[LENGTH_HEADER_LEN..], FramePart::SupplyRun)?; + run[..prefix.len()].copy_from_slice(&prefix); + self.read_exact(&mut run[prefix.len()..], FramePart::SupplyRun)?; return Ok(LeafRun::from_encoded(run)?); } // This oracle deliberately reads the whole declared body at once so @@ -149,10 +179,29 @@ impl<'a, R: Read> FrameDecoder<'a, R> { Ok(LeafRun::from_encoded(run)?) } - fn byte(&mut self, part: FramePart) -> Result { - let mut byte = 0; - self.read_exact(slice::from_mut(&mut byte), part)?; - Ok(byte) + /// Read the first record's heads inside an over-budget run, returning + /// the exact bytes consumed and the record content length; `None` when + /// they are not a record's heads (over budget, the distinction from + /// malformed is moot). + fn record_prefix(&mut self) -> Result, u64)>, DecodeErrorKind> { + let mut prefix = Vec::new(); + let tag = self.head(FramePart::SupplyRun)?; + cbor::write_head(&mut prefix, tag.major, tag.value); + if tag.major != cbor::MAJOR_TAG || tag.value != cbor::TAG_CBOR_SEQUENCE { + return Ok(None); + } + let body = self.head(FramePart::SupplyRun)?; + cbor::write_head(&mut prefix, body.major, body.value); + if body.major != cbor::MAJOR_BSTR { + return Ok(None); + } + Ok(Some((prefix, body.value))) + } + + fn head(&mut self, part: FramePart) -> Result { + cbor::read_head_io(self.read) + .map_err(|e| head_error(part, e))? + .ok_or_else(|| head_error(part, HeadReadError::Io(ErrorKind::UnexpectedEof.into()))) } fn read_exact(&mut self, bytes: &mut [u8], part: FramePart) -> Result<(), DecodeErrorKind> { @@ -168,39 +217,145 @@ impl<'a, R: Read> FrameDecoder<'a, R> { } } -/// Whether a run body of `len` bytes is exactly one record: the first -/// record's header plus the record it declares span the body. -/// -/// The lone-record test of the run-budget ingress check, shared by the -/// async reader and the sync oracle so the two decoders draw the -/// over-budget legality boundary identically. A body this predicate -/// rejects may also be structurally malformed; over budget, that -/// distinction is moot — either way the frame is not the one legal -/// overhang — so the check does not refine it further. -fn lone_record_spans(len: usize, first_record_len: usize) -> bool { - LENGTH_HEADER_LEN.saturating_add(first_record_len) == len +/// Validate a frame's array head: a definite array of one or two items. +pub(super) fn frame_arity(head: cbor::Head) -> Result { + if head.major != MAJOR_ARRAY { + return Err(DecodeErrorKind::FrameShape { + detail: "frame item is not an array", + }); + } + if !(1..=2).contains(&head.value) { + return Err(DecodeErrorKind::FrameShape { + detail: "frame array is not one or two items", + }); + } + Ok(head.value) } -fn decode_signal(speaker: Speaker, byte: u8) -> Result<(Stream, Signal), DecodeError> { - let wire = WireSignal::from_byte(speaker, byte) - .map_err(|invalid| DecodeError::stream(speaker, invalid.stream(), invalid.into()))?; - Ok(wire.into_parts()) +/// Validate a signal head: an unsigned int within the dense code space's +/// byte range. Codes above the dense space but within the byte range keep +/// their reserved-value taxonomy downstream. +pub(super) fn signal_code(head: cbor::Head) -> Result { + if head.major != MAJOR_UINT { + return Err(DecodeErrorKind::Malformed { + part: FramePart::Signal, + detail: "signal is not an unsigned int", + }); + } + u8::try_from(head.value).map_err(|_| DecodeErrorKind::Malformed { + part: FramePart::Signal, + detail: "signal is outside the dense code space", + }) +} + +/// Enforce the frame array's length against its signal's body arity. +pub(super) fn check_arity(signal: Signal, arity: u64) -> Result<(), DecodeErrorKind> { + let expected = match signal { + Signal::Match(_) | Signal::QueryEmpty(_) | Signal::End(_) => 1, + Signal::Query(_) | Signal::Supply(_) => 2, + }; + if arity != expected { + return Err(DecodeErrorKind::FrameArity { + expected, + found: arity, + }); + } + Ok(()) +} + +/// Open a query body: its head must be a nonempty listing map (an empty +/// query travels as its own signal), within the radix space. +pub(super) fn query_listing( + head: cbor::Head, +) -> Result { + if head.major != cbor::MAJOR_MAP { + return Err(DecodeErrorKind::Malformed { + part: FramePart::QueryChildren, + detail: "query body is not a listing map", + }); + } + if head.value == 0 { + return Err(DecodeErrorKind::Malformed { + part: FramePart::QueryChildren, + detail: "a nonempty query's listing is empty", + }); + } + super::frame::ListingBuilder::new(head.value).map_err(listing_issue) +} + +/// Open a supply body: the run's embedded-sequence tag and byte-string +/// head, held to the wire's run byte cap. +pub(super) fn run_head(tag: cbor::Head, body: cbor::Head) -> Result { + if tag.major != cbor::MAJOR_TAG || tag.value != cbor::TAG_CBOR_SEQUENCE { + return Err(DecodeErrorKind::Malformed { + part: FramePart::SupplyLength, + detail: "supply body does not open with the embedded-sequence tag", + }); + } + if body.major != cbor::MAJOR_BSTR { + return Err(DecodeErrorKind::Malformed { + part: FramePart::SupplyLength, + detail: "supply tag does not wrap a byte string", + }); + } + u32::try_from(body.value) + .map(|len| len as usize) + .map_err(|_| DecodeErrorKind::Malformed { + part: FramePart::SupplyLength, + detail: "supply run exceeds the run byte cap", + }) } -/// `pub(super)` for the capture renderer, which decodes captured query -/// children through the same canonical path (order validation included). -pub(super) fn parse_query(listing: &[u8]) -> Result, DecodeErrorKind> { - let mut children = Vec::with_capacity(listing.len() / QUERY_CHILD_LEN); - for record in listing.chunks_exact(QUERY_CHILD_LEN) { - let (&radix, encoded_hash) = record - .split_first() - .expect("a query child record contains its radix"); - let mut hash = [0; MERKLE_HASH_LEN]; - hash.copy_from_slice(encoded_hash); - children.push((radix, Hash(hash))); - } - validate_children(&children)?; - Ok(children) +/// Type a listing-map violation into the frame error taxonomy. +pub(super) fn listing_issue(issue: ListingIssue) -> DecodeErrorKind { + match issue { + ListingIssue::Order(order) => DecodeErrorKind::QueryOutOfOrder(order), + ListingIssue::Head(_) => DecodeErrorKind::Malformed { + part: FramePart::QueryChildren, + detail: "listing head is not canonical", + }, + ListingIssue::Shape(detail) => DecodeErrorKind::Malformed { + part: FramePart::QueryChildren, + detail, + }, + ListingIssue::Truncated => DecodeErrorKind::Malformed { + part: FramePart::QueryChildren, + detail: "listing hash bytes are truncated", + }, + } +} + +/// Type a head-read failure by the frame part it interrupted. +pub(super) fn head_error(part: FramePart, error: HeadReadError) -> DecodeErrorKind { + match error { + HeadReadError::Io(source) => match source.kind() { + std::io::ErrorKind::UnexpectedEof => DecodeErrorKind::Truncated { + missing: part, + source, + }, + _ => DecodeErrorKind::Read { part, source }, + }, + HeadReadError::Malformed(head) => DecodeErrorKind::Malformed { + part, + detail: head_detail(head), + }, + } +} + +/// Name a deterministic-contract violation for the error taxonomy. +fn head_detail(error: cbor::HeadError) -> &'static str { + match error { + cbor::HeadError::Truncated => "truncated head", + cbor::HeadError::Indefinite => "indefinite-length head", + cbor::HeadError::Reserved => "reserved head", + cbor::HeadError::NotShortest => "head not in shortest form", + } +} + +pub(super) fn decode_signal(speaker: Speaker, code: u8) -> Result<(Stream, Signal), DecodeError> { + let wire = WireSignal::from_byte(speaker, code) + .map_err(|invalid| DecodeError::stream(speaker, invalid.stream(), invalid.into()))?; + Ok(wire.into_parts()) } #[cfg(test)] diff --git a/src/tree/mirror/streaming/remote/codec/decode/async_io.rs b/src/tree/mirror/streaming/remote/codec/decode/async_io.rs index a490a0025..019998ef9 100644 --- a/src/tree/mirror/streaming/remote/codec/decode/async_io.rs +++ b/src/tree/mirror/streaming/remote/codec/decode/async_io.rs @@ -1,35 +1,43 @@ //! Exact asynchronous input for the self-delimiting frame grammar. -use std::slice; - use std::io::ErrorKind; use tokio::io::{AsyncRead, AsyncReadExt}; +use crate::observe::{CaptureRead, StreamObserver}; + use super::super::{ budget::RunBudget, error::{DecodeError, DecodeErrorKind, FramePart}, - frame::{Frame, LeafRun, QUERY_CHILD_LEN, QUERY_COUNT_BIAS, Reaction, WireFrame}, - signal::{Signal, Speaker, Stream}, + frame::{Frame, LeafRun, ListingBuilder, Reaction, WireFrame}, + signal::{Signal, Speaker}, +}; +use super::{ + check_arity, decode_signal, frame_arity, head_error, listing_issue, query_listing, run_head, + signal_code, }; -use super::{decode_signal, lone_record_spans, parse_query}; use crate::tree::{ - mirror::framing::{LENGTH_HEADER_LEN, read_payload, resume_payload}, - typed::Hash, + mirror::cbor, + mirror::framing::{read_payload, resume_payload}, + typed::{Hash, hash::MERKLE_HASH_LEN}, }; /// Async frame reader over one speaker's transport direction. /// -/// EOF before a signal is a clean direction close and returns `None`. Once a -/// signal arrives, a missing component is a contextual truncation. Variable -/// bodies are read at their declared size and validated exactly once, with -/// supply bodies additionally held to the session's run budget before they -/// are buffered ([`DecodeErrorKind::OverbatchedRun`]). +/// EOF before a frame's array head is a clean direction close and returns +/// `None`. Once that head arrives, a missing component is a contextual +/// truncation. Variable bodies are read at their declared size and +/// validated exactly once, with supply bodies additionally held to the +/// session's run budget before they are buffered +/// ([`DecodeErrorKind::OverbatchedRun`]). pub struct FrameRead { speaker: Speaker, /// The session's negotiated run budget, enforced on every supply frame /// this direction delivers. budget: RunBudget, read: R, + /// The directed stream's observer, if any: handed each accepted + /// frame's exact wire bytes, and costing one branch when absent. + observe: Option>, } impl FrameRead { @@ -40,9 +48,16 @@ impl FrameRead { speaker, budget, read, + observe: None, } } + /// Deliver every accepted frame to `observe`, when one is attached. + pub fn observed(mut self, observe: Option>) -> Self { + self.observe = observe; + self + } + /// Recover the transport half. The reader buffers nothing (every /// read is exact), so between frames the half rests exactly at a /// frame boundary. @@ -59,38 +74,57 @@ impl FrameRead { /// Not cancel safe. A dropped `frame` future may already have consumed /// part of a frame — the exact reads do not give bytes back — leaving /// the direction mid-frame, where the next call would parse body bytes - /// as a signal. Either retain the in-flight future across polls until - /// it resolves, or read nothing further from this direction after a - /// cancellation. + /// as a frame head. Either retain the in-flight future across polls + /// until it resolves, or read nothing further from this direction after + /// a cancellation. pub async fn frame(&mut self) -> Result, DecodeError> { - let Some((stream, signal)) = read_signal(self.speaker, &mut self.read).await? else { - return Ok(None); - }; - let frame = AsyncFrameDecoder::new(&mut self.read, self.budget) - .body(signal) - .await - .map_err(|kind| DecodeError::stream(self.speaker, stream, kind))?; - Ok(Some((stream, frame))) + match &mut self.observe { + None => read_frame(&mut self.read, self.speaker, self.budget).await, + Some(observe) => { + // Retain the consumed bytes so the observer sees the + // frame's true wire spelling, never a re-encoding. Only + // an accepted whole frame is delivered: a clean close + // consumed nothing, and an error leaves a fragment. + let mut capture = CaptureRead::new(&mut self.read); + let result = read_frame(&mut capture, self.speaker, self.budget).await; + if let Ok(Some(_)) = &result { + observe.message(capture.bytes()); + } + result + } + } } } -async fn read_signal( +/// Read and decode one frame from `read`; the contract is +/// [`FrameRead::frame`]'s. +async fn read_frame( + read: &mut R, speaker: Speaker, - read: &mut (impl AsyncRead + Unpin), -) -> Result, DecodeError> { - let mut byte = 0; - match read.read(slice::from_mut(&mut byte)).await { - Ok(0) => Ok(None), - Ok(1) => decode_signal(speaker, byte).map(Some), - Ok(_) => unreachable!("a one-byte async read returns at most one byte"), - Err(source) => Err(DecodeError::direction( - speaker, - DecodeErrorKind::Read { - part: FramePart::Signal, - source, - }, - )), + budget: RunBudget, +) -> Result, DecodeError> { + let Some(head) = cbor::read_head_async(&mut *read) + .await + .map_err(|e| head_error(FramePart::FrameHead, e)) + .map_err(|kind| DecodeError::direction(speaker, kind))? + else { + return Ok(None); + }; + let mut decoder = AsyncFrameDecoder::new(read, budget); + let arity = frame_arity(head).map_err(|kind| DecodeError::direction(speaker, kind))?; + let signal_head = decoder + .head(FramePart::Signal) + .await + .map_err(|kind| DecodeError::direction(speaker, kind))?; + let code = signal_code(signal_head).map_err(|kind| DecodeError::direction(speaker, kind))?; + let (stream, signal) = decode_signal(speaker, code)?; + let frame = async { + check_arity(signal, arity)?; + decoder.body(signal).await } + .await + .map_err(|kind| DecodeError::stream(speaker, stream, kind))?; + Ok(Some((stream, frame))) } /// Reads a body after its signal has established the frame grammar. @@ -117,18 +151,25 @@ impl<'a, R: AsyncRead + Unpin> AsyncFrameDecoder<'a, R> { } async fn query(&mut self) -> Result, DecodeErrorKind> { - let count = usize::from(self.byte(FramePart::QueryCount).await?) + QUERY_COUNT_BIAS; - let mut listing = vec![0; count * QUERY_CHILD_LEN]; - self.read_exact(&mut listing, FramePart::QueryChildren) - .await?; - parse_query(&listing) + let head = self.head(FramePart::QueryChildren).await?; + let mut listing = query_listing(head)?; + for _ in 0..head.value { + let key = self.head(FramePart::QueryChildren).await?; + let radix = listing.key(key).map_err(listing_issue)?; + let value = self.head(FramePart::QueryChildren).await?; + ListingBuilder::value_head(value).map_err(listing_issue)?; + let mut digest = [0; MERKLE_HASH_LEN]; + self.read_exact(&mut digest, FramePart::QueryChildren) + .await?; + listing.entry(radix, digest); + } + Ok(listing.finish()) } async fn supply(&mut self) -> Result { - let mut header = [0; LENGTH_HEADER_LEN]; - self.read_exact(&mut header, FramePart::SupplyLength) - .await?; - let len = u32::from_be_bytes(header) as usize; + let tag = self.head(FramePart::SupplyLength).await?; + let body = self.head(FramePart::SupplyLength).await?; + let len = run_head(tag, body)?; let run = if self.budget.covers(len) { read_payload(self.read, len) .await @@ -138,39 +179,66 @@ impl<'a, R: AsyncRead + Unpin> AsyncFrameDecoder<'a, R> { // within, so the one shape an honest encoder can still have // produced is a single record spanning the whole body (the // minimum-one-record overhang). That is decidable from the - // first record's length header alone, so nothing beyond it is + // first record's heads alone, so nothing beyond them is // read until the frame is known legal: a violating frame is // rejected before its body is buffered, keeping the decode // inside the memory envelope the budget priced. A body too - // short to hold a record header cannot be a lone record and is - // rejected on the declared length alone. + // short to hold a record's heads cannot be a lone record and + // is rejected on the declared length alone. let budget = self.budget; let overbatched = move || DecodeErrorKind::OverbatchedRun { declared: super::super::budget::SUPPLY_FRAME_OVERHEAD.saturating_add(len), budget: budget.bytes(), }; - if len < LENGTH_HEADER_LEN { + if len < super::super::frame::RECORD_TAG_LEN + 1 { return Err(overbatched()); } - let mut first = [0; LENGTH_HEADER_LEN]; - self.read_exact(&mut first, FramePart::SupplyRun).await?; - let record = u32::from_be_bytes(first) as usize; - if !lone_record_spans(len, record) { + let Some((prefix, record)) = self.record_prefix().await? else { + return Err(overbatched()); + }; + if !super::super::frame::lone_record_spans(len, record) { return Err(overbatched()); } - // Legal lone record: resume the body read behind the header + // Legal lone record: resume the body read behind the heads // already consumed, in the same single buffer. - resume_payload(self.read, first.to_vec(), len) + resume_payload(self.read, prefix, len) .await .map_err(|source| classify(FramePart::SupplyRun, source))? }; Ok(LeafRun::from_encoded(run)?) } - async fn byte(&mut self, part: FramePart) -> Result { - let mut byte = 0; - self.read_exact(slice::from_mut(&mut byte), part).await?; - Ok(byte) + /// Read the first record's heads inside an over-budget run, returning + /// the exact bytes consumed and the record content length. + /// + /// `None` when they are not a record's heads: over budget, the + /// distinction from malformed is moot — either way the frame is not + /// the legal overhang. + async fn record_prefix(&mut self) -> Result, u64)>, DecodeErrorKind> { + let mut prefix = Vec::new(); + let tag = self.head(FramePart::SupplyRun).await?; + cbor::write_head(&mut prefix, tag.major, tag.value); + if tag.major != cbor::MAJOR_TAG || tag.value != cbor::TAG_CBOR_SEQUENCE { + return Ok(None); + } + let body = self.head(FramePart::SupplyRun).await?; + cbor::write_head(&mut prefix, body.major, body.value); + if body.major != cbor::MAJOR_BSTR { + return Ok(None); + } + Ok(Some((prefix, body.value))) + } + + async fn head(&mut self, part: FramePart) -> Result { + cbor::read_head_async(self.read) + .await + .map_err(|e| head_error(part, e))? + .ok_or_else(|| { + head_error( + part, + cbor::HeadReadError::Io(ErrorKind::UnexpectedEof.into()), + ) + }) } async fn read_exact( diff --git a/src/tree/mirror/streaming/remote/codec/decode/tests.rs b/src/tree/mirror/streaming/remote/codec/decode/tests.rs index e5df2a9a9..ca40ccff0 100644 --- a/src/tree/mirror/streaming/remote/codec/decode/tests.rs +++ b/src/tree/mirror/streaming/remote/codec/decode/tests.rs @@ -1,22 +1,21 @@ +use crate::message::{PayloadCodec, PayloadDepthLimit}; use proptest::prelude::*; use super::*; use crate::Version; use crate::message::Message; use crate::tree::arb::arb_version; +use crate::tree::mirror::cbor::{MAJOR_BSTR, MAJOR_MAP, MAJOR_TAG, TAG_CBOR_SEQUENCE}; +use crate::tree::typed::{Hash, hash::MERKLE_HASH_LEN}; use super::super::{ error::{DecodeLeafError, Origin, QueryOrderError}, - frame::{QUERY_COUNT_BIAS, QUERY_COUNT_LEN}, + frame::{LeafRunError, MAX_QUERY_CHILDREN, RECORD_TAG_LEN}, signal::{DecodeSignalError, End, Flow, Speaker, Stream, StreamError}, }; const SPEAKERS: [Speaker; 2] = [Speaker::Initiator, Speaker::Responder]; -/// A CBOR byte-string header promising two version bytes, cut short after -/// one: the version field ends inside its own framing. -const TRUNCATED_VERSION: &[u8] = &[0x42, 0x01]; - fn stream(index: u8) -> Stream { Stream::new(index).unwrap() } @@ -27,24 +26,77 @@ fn signal(stream: Stream, signal: Signal) -> u8 { .to_byte() } +/// The frame head of a `arity`-item frame carrying `code`: the array head +/// then the signal's unsigned-int head. +fn frame_head(arity: u64, code: u8) -> Vec { + let mut head = Vec::new(); + cbor::write_head(&mut head, cbor::MAJOR_ARRAY, arity); + cbor::write_head(&mut head, MAJOR_UINT, u64::from(code)); + head +} + +/// A whole body-free frame. +fn bare_frame(stream: Stream, s: Signal) -> Vec { + frame_head(1, signal(stream, s)) +} + +/// A whole supply frame declaring `body.len()` run bytes and carrying +/// `body`. fn supply(stream: Stream, flow: Flow, body: &[u8]) -> Vec { - let mut encoded = vec![signal(stream, Signal::Supply(flow))]; - encoded.extend_from_slice(&(body.len() as u32).to_be_bytes()); + supply_declaring(stream, flow, body.len(), body) +} + +/// A supply frame declaring `declared` run bytes while carrying `body`. +fn supply_declaring(stream: Stream, flow: Flow, declared: usize, body: &[u8]) -> Vec { + let mut encoded = frame_head(2, signal(stream, Signal::Supply(flow))); + cbor::write_head(&mut encoded, MAJOR_TAG, TAG_CBOR_SEQUENCE); + cbor::write_head(&mut encoded, MAJOR_BSTR, declared as u64); encoded.extend_from_slice(body); encoded } -/// One length-prefixed leaf record as it appears inside a run body: the -/// version as one CBOR value, then the payload's CBOR bytes bare. +/// A whole query frame carrying `children` as its listing map, written +/// raw (no canonical-order validation) so tests can synthesize +/// violations. +fn query(stream: Stream, flow: Flow, children: &[(u8, Hash)]) -> Vec { + let mut encoded = frame_head(2, signal(stream, Signal::Query(flow))); + super::super::frame::write_listing(&mut encoded, children); + encoded +} + +/// One leaf record as it appears inside a run body: the embedded-sequence +/// tag and byte-string head, then the tagged version atom, then the +/// payload's CBOR bytes bare. fn record(version: &Version, message: &Message) -> Vec { - let mut body = Vec::new(); - ciborium::ser::into_writer(version, &mut body).unwrap(); - body.extend_from_slice(message.as_slice()); - let mut record = (body.len() as u32).to_be_bytes().to_vec(); - record.extend_from_slice(&body); + let mut content = Vec::new(); + cbor::write_head(&mut content, MAJOR_TAG, crate::tags::VERSION_TAG); + ciborium::ser::into_writer(version, &mut content).unwrap(); + content.extend_from_slice(message.as_slice()); + let mut record = Vec::new(); + cbor::write_head(&mut record, MAJOR_TAG, TAG_CBOR_SEQUENCE); + cbor::write_head(&mut record, MAJOR_BSTR, content.len() as u64); + record.extend_from_slice(&content); record } +/// A record item wrapping raw content bytes, for malformed-content cases. +fn raw_record(content: &[u8]) -> Vec { + let mut record = Vec::new(); + cbor::write_head(&mut record, MAJOR_TAG, TAG_CBOR_SEQUENCE); + cbor::write_head(&mut record, MAJOR_BSTR, content.len() as u64); + record.extend_from_slice(content); + record +} + +/// A record's content for `version` and `message`, without its item heads. +fn record_content(version: &Version, message: &Message) -> Vec { + let mut content = Vec::new(); + cbor::write_head(&mut content, MAJOR_TAG, crate::tags::VERSION_TAG); + ciborium::ser::into_writer(version, &mut content).unwrap(); + content.extend_from_slice(message.as_slice()); + content +} + fn arb_speaker() -> impl Strategy { prop_oneof![Just(Speaker::Initiator), Just(Speaker::Responder)] } @@ -53,22 +105,31 @@ fn arb_flow() -> impl Strategy { prop_oneof![Just(Flow::Continue), Just(Flow::End)] } -/// Reserved signal states retain the stream encoded alongside them. +/// The stream constructor rejects an index past the stream range with a +/// typed error naming the index. #[test] -fn invalid_signals_are_rejected() { +fn out_of_range_stream_index_is_rejected() { assert_eq!( Stream::new(Stream::COUNT), Err(StreamError::Invalid { index: Stream::COUNT }) ); +} + +/// Reserved signal codes within the byte range retain the stream encoded +/// alongside them; codes past the byte range and non-int signal items are +/// malformed signals. +#[test] +fn invalid_signals_are_rejected() { for byte in WireSignal::BYTE_COUNT..=u8::MAX { for speaker in SPEAKERS { let invalid = WireSignal::from_byte(speaker, byte).unwrap_err(); let DecodeSignalError::Reserved(reserved) = invalid else { panic!("unexpected signal error") }; - let error = decode_exact(speaker, RunBudget::default(), &[byte]).unwrap_err(); + let encoded = frame_head(1, byte); + let error = decode_exact(speaker, RunBudget::default(), &encoded).unwrap_err(); assert_eq!(error.origin, Origin::stream(speaker, reserved.stream())); let DecodeErrorKind::InvalidSignal(DecodeSignalError::Reserved(source)) = error.kind else { @@ -80,6 +141,89 @@ fn invalid_signals_are_rejected() { assert!(std::error::Error::source(&source).is_some()); } } + // Past the byte range, and a non-int item where the signal belongs. + for speaker in SPEAKERS { + let mut encoded = Vec::new(); + cbor::write_head(&mut encoded, cbor::MAJOR_ARRAY, 1); + cbor::write_head(&mut encoded, MAJOR_UINT, 256); + let error = decode_exact(speaker, RunBudget::default(), &encoded).unwrap_err(); + assert_eq!(error.origin, Origin::direction(speaker)); + assert!(matches!( + error.kind, + DecodeErrorKind::Malformed { + part: FramePart::Signal, + .. + } + )); + + let mut encoded = Vec::new(); + cbor::write_head(&mut encoded, cbor::MAJOR_ARRAY, 1); + cbor::write_head(&mut encoded, MAJOR_BSTR, 0); + let error = decode_exact(speaker, RunBudget::default(), &encoded).unwrap_err(); + assert!(matches!( + error.kind, + DecodeErrorKind::Malformed { + part: FramePart::Signal, + .. + } + )); + } +} + +/// A frame item that is not a one- or two-element array, or whose array +/// length contradicts its signal's body arity, is rejected typed. +#[test] +fn frame_shape_is_enforced() { + let stream = stream(4); + for speaker in SPEAKERS { + // Not an array at all. + let error = decode_exact(speaker, RunBudget::default(), &[0x00]).unwrap_err(); + assert!(matches!(error.kind, DecodeErrorKind::FrameShape { .. })); + // A three-item array. + let error = decode_exact(speaker, RunBudget::default(), &[0x83]).unwrap_err(); + assert!(matches!(error.kind, DecodeErrorKind::FrameShape { .. })); + // A body-free signal inside a two-item array. + let encoded = frame_head(2, signal(stream, Signal::Match(Flow::Continue))); + let error = decode_exact(speaker, RunBudget::default(), &encoded).unwrap_err(); + assert_eq!(error.origin, Origin::stream(speaker, stream)); + assert!(matches!( + error.kind, + DecodeErrorKind::FrameArity { + expected: 1, + found: 2 + } + )); + // A body-bearing signal inside a one-item array. + let encoded = frame_head(1, signal(stream, Signal::Query(Flow::Continue))); + let error = decode_exact(speaker, RunBudget::default(), &encoded).unwrap_err(); + assert!(matches!( + error.kind, + DecodeErrorKind::FrameArity { + expected: 2, + found: 1 + } + )); + } +} + +/// A widened (non-shortest-form) signal head is rejected: the wire admits +/// one spelling per value. +#[test] +fn widened_signal_heads_are_rejected() { + let stream = stream(3); + let code = signal(stream, Signal::Match(Flow::Continue)); + for speaker in SPEAKERS { + // The code spelled with a needlessly wide argument. + let encoded = [0x81, 0x19, 0x00, code]; + let error = decode_exact(speaker, RunBudget::default(), &encoded).unwrap_err(); + assert!(matches!( + error.kind, + DecodeErrorKind::Malformed { + part: FramePart::Signal, + .. + } + )); + } } /// Truncation identifies both the absent component and its known origin. @@ -87,27 +231,33 @@ fn invalid_signals_are_rejected() { fn truncated_bodies_are_rejected() { let stream = stream(4); for speaker in SPEAKERS { + let query_head = frame_head(2, signal(stream, Signal::Query(Flow::Continue))); + let mut half_listing = query_head.clone(); + cbor::write_head(&mut half_listing, MAJOR_MAP, 1); + let supply_head = frame_head(2, signal(stream, Signal::Supply(Flow::Continue))); let cases = [ - (Vec::new(), FramePart::Signal, Origin::direction(speaker)), + (Vec::new(), FramePart::FrameHead, Origin::direction(speaker)), + (vec![0x81], FramePart::Signal, Origin::direction(speaker)), ( - vec![signal(stream, Signal::Query(Flow::Continue))], - FramePart::QueryCount, + query_head, + FramePart::QueryChildren, Origin::stream(speaker, stream), ), ( - vec![signal(stream, Signal::Query(Flow::Continue)), u8::MIN], + half_listing, FramePart::QueryChildren, Origin::stream(speaker, stream), ), ( - vec![signal(stream, Signal::Supply(Flow::Continue))], + supply_head.clone(), FramePart::SupplyLength, Origin::stream(speaker, stream), ), ( { - let mut frame = vec![signal(stream, Signal::Supply(Flow::Continue))]; - frame.extend_from_slice(&1_u32.to_be_bytes()); + let mut frame = supply_head; + cbor::write_head(&mut frame, MAJOR_TAG, TAG_CBOR_SEQUENCE); + cbor::write_head(&mut frame, MAJOR_BSTR, 4); frame }, FramePart::SupplyRun, @@ -116,13 +266,13 @@ fn truncated_bodies_are_rejected() { ]; for (encoded, missing, origin) in cases { let error = decode_exact(speaker, RunBudget::default(), &encoded).unwrap_err(); - assert_eq!(error.origin, origin); + assert_eq!(error.origin, origin, "case {missing:?}"); let DecodeErrorKind::Truncated { missing: actual, source, } = error.kind else { - panic!("unexpected error kind"); + panic!("unexpected error kind for {missing:?}: {:?}", error.kind); }; assert_eq!(actual, missing); assert_eq!(source.kind(), std::io::ErrorKind::UnexpectedEof); @@ -132,7 +282,12 @@ fn truncated_bodies_are_rejected() { proptest! { /// An arbitrary run of supplied records decodes into a frame carrying the - /// exact run body, without decoding any record eagerly. + /// exact run body, byte for byte. + /// + /// Deferral of record decoding is what + /// `a_zero_length_record_is_structurally_valid` pins: its record only a + /// non-eager decoder can accept; this body's records are all well-formed, + /// so byte equality alone cannot tell eager from lazy. #[test] fn supplied_run_is_decoded_structurally( index in 1_u8..Stream::MAX, @@ -155,13 +310,11 @@ proptest! { } } -/// Structurally invalid runs are rejected at the wire with their exact cause: -/// an empty run, a record header past the run's end, or a record body past -/// the run's end. +/// Structurally invalid runs are rejected at the wire with their exact +/// cause: an empty run, bytes that are no record item, or a record's +/// content past the run's end. #[test] fn malformed_run_structure_is_typed() { - use super::super::frame::LeafRunError; - let stream = stream(8); for speaker in SPEAKERS { let empty = decode_exact( @@ -176,20 +329,21 @@ fn malformed_run_structure_is_typed() { DecodeErrorKind::InvalidRun(LeafRunError::Empty) )); - let short_header = decode_exact( + // Bytes where a record item belongs that are not one. + let not_a_record = decode_exact( speaker, RunBudget::default(), - &supply(stream, Flow::Continue, &[0, 0]), + &supply(stream, Flow::Continue, &[0x00, 0x00]), ) .unwrap_err(); - assert_eq!(short_header.origin, Origin::stream(speaker, stream)); + assert_eq!(not_a_record.origin, Origin::stream(speaker, stream)); assert!(matches!( - short_header.kind, - DecodeErrorKind::InvalidRun(LeafRunError::TruncatedHeader { remaining: 2 }) + not_a_record.kind, + DecodeErrorKind::InvalidRun(LeafRunError::NotARecord { remaining: 2, .. }) )); - let mut overrun = 2_u32.to_be_bytes().to_vec(); - overrun.push(0); + // A record declaring more content than the run holds. + let overrun = raw_record(&[0, 0])[..RECORD_TAG_LEN + 2].to_vec(); let short_record = decode_exact( speaker, RunBudget::default(), @@ -207,16 +361,16 @@ fn malformed_run_structure_is_typed() { } } -/// A zero-length record header inside a run body is structurally valid. +/// An empty-content record inside a run body is structurally valid. /// -/// From raw wire bytes, a run body of one bare `00000000` header chains -/// exactly, so the codec accepts the frame and defers the record's failure -/// to its record iterator: the empty body cannot hold a version, and the -/// iterator reports the version decoder's `UnexpectedEof`. +/// From raw wire bytes, a run body of one record whose byte string is +/// empty chains exactly, so the codec accepts the frame and defers the +/// record's failure to its record iterator: the empty content cannot hold +/// a tagged version, and the iterator reports the version decode failure. #[test] fn a_zero_length_record_is_structurally_valid() { let stream = stream(8); - let encoded = supply(stream, Flow::End, &[0, 0, 0, 0]); + let encoded = supply(stream, Flow::End, &raw_record(&[])); for speaker in SPEAKERS { let (decoded_stream, frame) = decode_exact(speaker, RunBudget::default(), &encoded).unwrap(); @@ -226,7 +380,7 @@ fn a_zero_length_record_is_structurally_valid() { }; assert_eq!(run.record_count(), 1); let error = run - .records(Message::deserializer::()) + .records(PayloadCodec::new::(PayloadDepthLimit::default())) .next() .unwrap() .unwrap_err(); @@ -241,11 +395,13 @@ fn a_zero_length_record_is_structurally_valid() { /// which types each failure and retains the source error. #[test] fn supplied_record_errors_are_typed() { - let mut truncated_version = (TRUNCATED_VERSION.len() as u32).to_be_bytes().to_vec(); - truncated_version.extend_from_slice(TRUNCATED_VERSION); - let run = LeafRun::from_encoded(truncated_version).unwrap(); + // A version byte string promising two bytes, cut short after one. + let mut content = Vec::new(); + cbor::write_head(&mut content, MAJOR_TAG, crate::tags::VERSION_TAG); + content.extend_from_slice(&[0x42, 0x01]); + let run = LeafRun::from_encoded(raw_record(&content)).unwrap(); let error = run - .records(Message::deserializer::()) + .records(PayloadCodec::new::(PayloadDepthLimit::default())) .next() .unwrap() .unwrap_err(); @@ -254,13 +410,26 @@ fn supplied_record_errors_are_typed() { }; assert_eq!(source.kind(), std::io::ErrorKind::UnexpectedEof); - let mut version = Vec::new(); - ciborium::ser::into_writer(&Version::new(), &mut version).unwrap(); - let mut missing_message = (version.len() as u32).to_be_bytes().to_vec(); - missing_message.extend_from_slice(&version); - let run = LeafRun::from_encoded(missing_message).unwrap(); + // An untagged version where the tagged atom belongs. + let mut content = Vec::new(); + ciborium::ser::into_writer(&Version::new(), &mut content).unwrap(); + let run = LeafRun::from_encoded(raw_record(&content)).unwrap(); + let error = run + .records(PayloadCodec::new::(PayloadDepthLimit::default())) + .next() + .unwrap() + .unwrap_err(); + let DecodeLeafError::Version(source) = error else { + panic!("unexpected record error"); + }; + assert_eq!(source.kind(), std::io::ErrorKind::InvalidData); + + // A tagged version with no message behind it. + let content = record_content(&Version::new(), &Message::new(0u64)); + let missing_message = &content[..content.len() - Message::new(0u64).as_slice().len()]; + let run = LeafRun::from_encoded(raw_record(missing_message)).unwrap(); let error = run - .records(Message::deserializer::()) + .records(PayloadCodec::new::(PayloadDepthLimit::default())) .next() .unwrap() .unwrap_err(); @@ -272,13 +441,11 @@ fn supplied_record_errors_are_typed() { // Bytes past the canonical pair make the payload malformed: the // payload runs to the record's end, so the deserializer's // exactly-one-value check is what rejects the excess. - ciborium::ser::into_writer(&0_u64, &mut version).unwrap(); - version.push(u8::MIN); - let mut trailing = (version.len() as u32).to_be_bytes().to_vec(); - trailing.extend_from_slice(&version); - let run = LeafRun::from_encoded(trailing).unwrap(); + let mut content = record_content(&Version::new(), &Message::new(0u64)); + content.push(u8::MIN); + let run = LeafRun::from_encoded(raw_record(&content)).unwrap(); let error = run - .records(Message::deserializer::()) + .records(PayloadCodec::new::(PayloadDepthLimit::default())) .next() .unwrap() .unwrap_err(); @@ -288,6 +455,104 @@ fn supplied_record_errors_are_typed() { assert_eq!(source.kind(), std::io::ErrorKind::InvalidData); } +/// Pins the stated ingress boundary: the version atom's CBOR head is not +/// spelling-judged (the atom's content is, by `Version::decode`). +/// +/// A record whose version byte string wears a widened two-byte-length +/// head still decodes. Flipping this to rejection is a deliberate +/// contract change, not drift. +#[test] +fn widened_version_atom_head_is_not_spelling_judged() { + // The canonical atom bytes: ciborium serializes a version as a byte + // string whose one-byte head's low bits carry the length; strip that + // head to get the content alone. + let mut atom = Vec::new(); + ciborium::ser::into_writer(&Version::new(), &mut atom).unwrap(); + let content_bytes = &atom[1..]; + + let mut content = Vec::new(); + cbor::write_head(&mut content, MAJOR_TAG, crate::tags::VERSION_TAG); + // The same byte string, its length spelled in the widened two-byte + // form (major 2, additional info 25) instead of the shortest head. + content.push(0x59); + content.extend_from_slice(&u16::try_from(content_bytes.len()).unwrap().to_be_bytes()); + content.extend_from_slice(content_bytes); + content.extend_from_slice(Message::new(0u64).as_slice()); + + let run = LeafRun::from_encoded(raw_record(&content)).unwrap(); + let (version, _message) = run + .records(PayloadCodec::new::(PayloadDepthLimit::default())) + .next() + .unwrap() + .expect("a widened version-atom head decodes: spelling is not re-judged here"); + assert_eq!(version, Version::new()); +} + +/// Pins the stated ingress boundary: the version atom's CBOR head is not +/// spelling-judged, indefinite lengths included. +/// +/// A record whose version byte string is spelled indefinite-length (one +/// definite segment of the canonical content, then the break) still +/// decodes. Flipping this to rejection is a deliberate contract change, +/// not drift. +#[test] +fn indefinite_version_atom_head_is_not_spelling_judged() { + // The canonical atom bytes: ciborium serializes a version as a byte + // string whose one-byte head's low bits carry the length; strip that + // head to get the content alone. + let mut atom = Vec::new(); + ciborium::ser::into_writer(&Version::new(), &mut atom).unwrap(); + let content_bytes = &atom[1..]; + + let mut content = Vec::new(); + cbor::write_head(&mut content, MAJOR_TAG, crate::tags::VERSION_TAG); + // The same bytes as an indefinite-length byte string: the start + // marker (major 2, additional info 31), one definite segment holding + // the canonical content, and the break. + content.push(0x5f); + cbor::write_head(&mut content, MAJOR_BSTR, content_bytes.len() as u64); + content.extend_from_slice(content_bytes); + content.push(0xff); + content.extend_from_slice(Message::new(0u64).as_slice()); + + let run = LeafRun::from_encoded(raw_record(&content)).unwrap(); + let (version, _message) = run + .records(PayloadCodec::new::(PayloadDepthLimit::default())) + .next() + .unwrap() + .expect("an indefinite version-atom head decodes: spelling is not re-judged here"); + assert_eq!(version, Version::new()); +} + +/// Pins the stated ingress boundary: the application payload is decoded +/// by a general CBOR reader that does not judge spelling. +/// +/// A record whose payload is the indefinite-length empty map (a spelling +/// the emitter never writes) still decodes. Flipping this to rejection +/// is a deliberate contract change, not drift. +#[test] +fn indefinite_payload_spelling_is_not_spelling_judged() { + let mut content = Vec::new(); + cbor::write_head(&mut content, MAJOR_TAG, crate::tags::VERSION_TAG); + ciborium::ser::into_writer(&Version::new(), &mut content).unwrap(); + // The indefinite-length empty map: start marker, then the break. + content.extend_from_slice(&[0xbf, 0xff]); + + let run = LeafRun::from_encoded(raw_record(&content)).unwrap(); + let (version, message) = run + .records(PayloadCodec::new::>( + PayloadDepthLimit::default(), + )) + .next() + .unwrap() + .expect("an indefinite-length payload decodes: spelling is not judged here"); + assert_eq!(version, Version::new()); + assert_eq!( + *message.arc::>(), + std::collections::BTreeMap::new() + ); +} + proptest! { /// Every adjacent non-ascending pair reports its values and origin. #[test] @@ -300,16 +565,7 @@ proptest! { prop_assume!(previous >= radix); let stream = stream(index); let children = vec![(previous, Hash::default()), (radix, Hash::default())]; - let encoded_count = u8::try_from(children.len() - QUERY_COUNT_BIAS).unwrap(); - let mut encoded = Vec::with_capacity(WireSignal::ENCODED_LEN + QUERY_COUNT_LEN); - encoded.extend_from_slice(&[ - signal(stream, Signal::Query(Flow::Continue)), - encoded_count, - ]); - for (radix, hash) in &children { - encoded.push(*radix); - encoded.extend_from_slice(hash.as_bytes()); - } + let encoded = query(stream, Flow::Continue, &children); let error = decode_exact(speaker, RunBudget::default(), &encoded).unwrap_err(); prop_assert_eq!(error.origin, Origin::stream(speaker, stream)); let correct = matches!( @@ -321,23 +577,87 @@ proptest! { ); prop_assert!(correct); } + + /// An arbitrary canonical query round-trips through the decoder. + #[test] + fn canonical_queries_decode( + index in 1_u8..Stream::MAX, + speaker in arb_speaker(), + flow in arb_flow(), + radixes in proptest::collection::btree_set(any::(), 1..=32), + ) { + let stream = stream(index); + let children: Vec<(u8, Hash)> = radixes + .iter() + .map(|&radix| (radix, Hash([radix; MERKLE_HASH_LEN]))) + .collect(); + let encoded = query(stream, flow, &children); + prop_assert_eq!( + decode_exact(speaker, RunBudget::default(), &encoded).unwrap(), + (stream, Frame::Reaction(Reaction::Query(children), flow)) + ); + } +} + +/// A query body whose listing map is empty is rejected: an empty query +/// travels as its own signal, so the map spelling requires at least one +/// child (the upper bound is pinned by +/// `oversized_query_listing_is_rejected`). +#[test] +fn empty_query_listing_is_rejected() { + let stream = stream(5); + let encoded = query(stream, Flow::Continue, &[]); + for speaker in SPEAKERS { + let error = decode_exact(speaker, RunBudget::default(), &encoded).unwrap_err(); + assert!(matches!( + error.kind, + DecodeErrorKind::Malformed { + part: FramePart::QueryChildren, + .. + } + )); + } +} + +/// A query listing declaring more children than the radix space holds is +/// rejected at its map head, before any entry is read: the map spelling +/// admits at most one child per radix value. +#[test] +fn oversized_query_listing_is_rejected() { + let stream = stream(5); + let mut encoded = frame_head(2, signal(stream, Signal::Query(Flow::Continue))); + // A map head declaring one entry past the radix space, with no + // entries behind it: the rejection is decided on the head alone, in + // both decoders. + cbor::write_head(&mut encoded, MAJOR_MAP, MAX_QUERY_CHILDREN as u64 + 1); + for speaker in SPEAKERS { + let error = decode_both(speaker, RunBudget::default(), &encoded) + .expect_err("a listing past the radix space cannot decode"); + assert_eq!(error.origin, Origin::stream(speaker, stream)); + assert!(matches!( + error.kind, + DecodeErrorKind::Malformed { + part: FramePart::QueryChildren, + detail: "listing exceeds the radix space", + } + )); + } } /// Exact decoding rejects a trailing frame while incremental decoding preserves it. #[test] fn exact_decode_rejects_trailing_frame() { let stream = stream(10); - let first = signal(stream, Signal::Match(Flow::Continue)); - let second = signal(stream, Signal::End(End::Reply)); - let encoded = [first, second]; + let first = bare_frame(stream, Signal::Match(Flow::Continue)); + let second = bare_frame(stream, Signal::End(End::Reply)); + let mut encoded = first.clone(); + encoded.extend_from_slice(&second); for speaker in SPEAKERS { let error = decode_exact(speaker, RunBudget::default(), &encoded).unwrap_err(); assert_eq!(error.origin, Origin::stream(speaker, stream)); assert!(matches!( error.kind, - DecodeErrorKind::TrailingBytes { - count: WireSignal::ENCODED_LEN - } + DecodeErrorKind::TrailingBytes { count } if count == second.len() )); let mut rest = encoded.as_slice(); @@ -346,12 +666,15 @@ fn exact_decode_rejects_trailing_frame() { frame, (stream, Frame::Reaction(Reaction::Match, Flow::Continue)) ); - assert_eq!(rest, &[second]); + assert_eq!(rest, second.as_slice()); } } -/// Async EOF is clean only before a signal; every partial body reports the -/// same missing part and stream context as synchronous decoding. +/// Async EOF is clean only before a frame head; every partial body reports +/// the same missing part and stream context in both decoders. +/// +/// The clean close is checked async-only: the sync oracle's callers always +/// expect a frame, so it deliberately treats a clean close as a truncation. #[test] fn async_eof_distinguishes_close_from_truncation() { let stream = stream(4); @@ -359,31 +682,34 @@ fn async_eof_distinguishes_close_from_truncation() { let mut closed = FrameRead::new(speaker, RunBudget::default(), &[][..]); assert_eq!(pollster::block_on(closed.frame()).unwrap(), None); + let supply_head = frame_head(2, signal(stream, Signal::Supply(Flow::Continue))); let cases = [ ( - vec![signal(stream, Signal::Query(Flow::Continue))], - FramePart::QueryCount, - ), - ( - vec![signal(stream, Signal::Query(Flow::Continue)), u8::MIN], + frame_head(2, signal(stream, Signal::Query(Flow::Continue))), FramePart::QueryChildren, ), ( - vec![signal(stream, Signal::Supply(Flow::Continue))], - FramePart::SupplyLength, + { + let mut frame = frame_head(2, signal(stream, Signal::Query(Flow::Continue))); + cbor::write_head(&mut frame, MAJOR_MAP, 1); + frame + }, + FramePart::QueryChildren, ), + (supply_head.clone(), FramePart::SupplyLength), ( { - let mut frame = vec![signal(stream, Signal::Supply(Flow::Continue))]; - frame.extend_from_slice(&1_u32.to_be_bytes()); + let mut frame = supply_head; + cbor::write_head(&mut frame, MAJOR_TAG, TAG_CBOR_SEQUENCE); + cbor::write_head(&mut frame, MAJOR_BSTR, 4); frame }, FramePart::SupplyRun, ), ]; for (encoded, missing) in cases { - let mut reader = FrameRead::new(speaker, RunBudget::default(), encoded.as_slice()); - let error = pollster::block_on(reader.frame()).unwrap_err(); + let error = decode_both(speaker, RunBudget::default(), &encoded) + .expect_err("a truncated frame cannot decode"); assert_eq!(error.origin, Origin::stream(speaker, stream)); assert!(matches!( error.kind, @@ -396,8 +722,8 @@ fn async_eof_distinguishes_close_from_truncation() { } } -/// An invalid async signal consumes only itself, leaving the following valid -/// frame at the next exact boundary. +/// An invalid async signal consumes only its own frame, leaving the +/// following valid frame at the next exact boundary. #[test] fn async_invalid_signal_does_not_consume_a_body() { for speaker in SPEAKERS { @@ -423,7 +749,8 @@ fn async_invalid_signal_does_not_consume_a_body() { let valid = WireSignal::new(speaker, stream, valid_signal) .unwrap() .to_byte(); - let bytes = [invalid, valid]; + let mut bytes = frame_head(1, invalid); + bytes.extend_from_slice(&frame_head(1, valid)); let mut reader = FrameRead::new(speaker, RunBudget::default(), bytes.as_slice()); let error = pollster::block_on(reader.frame()).unwrap_err(); @@ -456,19 +783,22 @@ fn reader_errors_are_contextual() { assert!(matches!( error.kind, DecodeErrorKind::Read { - part: FramePart::Signal, + part: FramePart::FrameHead, source, } if source.kind() == std::io::ErrorKind::Other )); } } -/// Supply-body truncation cuts landing one byte short of, exactly on, and -/// one byte past each payload chunk boundary all classify as a truncated -/// `SupplyRun` with an `UnexpectedEof` source. +/// Supply-body truncation cuts at every seeded offset all classify as a +/// truncated `SupplyRun` with an `UnexpectedEof` source. /// -/// The chunked body read preserves the typed truncation contract at every -/// seam. +/// The seeded offsets are one byte short of, exactly on, and one byte +/// past each payload chunk boundary, plus the zero-byte, one-byte, and +/// one-short-of-total cuts. The chunked body read preserves the typed +/// truncation contract at every seam, the zero- and one-byte cuts +/// exercising the earliest possible ones — where a record's leading +/// heads would sit. #[test] fn supply_truncation_at_chunk_boundaries_is_typed() { use crate::tree::mirror::framing::{PAYLOAD_CHUNK_LEN, chunk_boundary_cuts}; @@ -477,9 +807,8 @@ fn supply_truncation_at_chunk_boundaries_is_typed() { let stream = stream(6); for speaker in SPEAKERS { for delivered in chunk_boundary_cuts(declared) { - let mut encoded = vec![signal(stream, Signal::Supply(Flow::Continue))]; - encoded.extend_from_slice(&u32::try_from(declared).unwrap().to_be_bytes()); - encoded.extend(vec![0xA5; delivered]); + let body = vec![0xA5; delivered]; + let encoded = supply_declaring(stream, Flow::Continue, declared, &body); let mut reader = FrameRead::new(speaker, RunBudget::default(), encoded.as_slice()); let error = pollster::block_on(reader.frame()).unwrap_err(); assert_eq!(error.origin, Origin::stream(speaker, stream)); @@ -497,7 +826,9 @@ fn supply_truncation_at_chunk_boundaries_is_typed() { } } -/// The full wire size of a supply frame carrying `body` run bytes. +/// The charged wire size of a supply frame carrying `body` run bytes: the +/// budget envelope constant plus the body, the exact quantity `covers` +/// prices and `OverbatchedRun` reports. fn frame_wire_size(body: &[u8]) -> usize { super::super::SUPPLY_FRAME_OVERHEAD + body.len() } @@ -545,14 +876,14 @@ fn decode_both( } proptest! { - /// Ingress enforces the run budget as the exact complement of the - /// encoder's flush rule, deciding before any body byte is read. + /// Ingress enforces the run budget, deciding from at most the first + /// record's heads. /// - /// A multi-record supply frame decodes when its full wire size is + /// A multi-record supply frame decodes when its charged wire size is /// within the budget and fails typed as `OverbatchedRun` — carrying /// that wire size and the budget — when it is past it. The rejection - /// is decided ahead of the body: a stream ending right after the - /// first record's length header still classifies as the budget + /// is decided ahead of the rest of the body: a stream ending right + /// after the first record's heads still classifies as the budget /// violation, never as a truncation. Both decoders (the async reader /// and the sync oracle) agree throughout. #[test] @@ -566,8 +897,19 @@ proptest! { ) { let stream = stream(index); let mut body = Vec::new(); - for (version, value) in &records { - body.extend_from_slice(&record(version, &Message::new(*value))); + let mut first_record_heads = 0; + for (at, (version, value)) in records.iter().enumerate() { + let record = record(version, &Message::new(*value)); + if at == 0 { + let content = { + let mut input = record.as_slice(); + super::super::frame::record_head(&mut input) + .expect("a built record has record heads"); + record.len() - input.len() + }; + first_record_heads = content; + } + body.extend_from_slice(&record); } let encoded = supply(stream, flow, &body); let wire_size = frame_wire_size(&body); @@ -594,11 +936,11 @@ proptest! { ); prop_assert!(typed, "mistyped over-budget batching: {:?}", error.kind); - // Before the body read: the same rejection from only the signal, - // the run length header, and the first record's length header — no - // body byte exists to read, so a decoder that buffered the body - // first would classify this as a truncation instead. - let prefix = &encoded[..1 + LENGTH_HEADER_LEN + LENGTH_HEADER_LEN]; + // From at most the first record's heads: the same rejection when + // the stream ends right after them (the heads are the leading + // body bytes the check reads) — a decoder that buffered the whole + // body first would classify this as a truncation instead. + let prefix = &encoded[..encoded.len() - body.len() + first_record_heads]; let error = decode_both(speaker, over, prefix).expect_err( "undetected over-budget batching: the violation must be decided \ ahead of the body", @@ -645,23 +987,22 @@ proptest! { /// Corner classifications of the run-budget ingress check, under a zero /// budget so every frame overhangs. /// -/// An over-budget body too short to hold a record header is the +/// An over-budget body too short to hold a record's heads is the /// violation, decided on the declared length alone (no body byte follows, -/// yet the error is not a truncation); a first record header that falls -/// short of the body or overruns it is the violation; a stream ending -/// inside the first record header, or inside an admitted lone record's -/// body, is a truncated supply run. +/// yet the error is not a truncation); a first record that falls short of +/// the body or overruns it is the violation; a stream ending inside the +/// first record's heads, or inside an admitted lone record's body, is a +/// truncated supply run. #[test] fn overbatched_corners_classify_exactly() { let stream = stream(9); let zero = RunBudget::from_bytes(0); for speaker in SPEAKERS { - // Declared bodies too short for a record header, none delivered. - for declared in 0..LENGTH_HEADER_LEN { - let mut encoded = vec![signal(stream, Signal::Supply(Flow::End))]; - encoded.extend_from_slice(&(declared as u32).to_be_bytes()); + // Declared bodies too short for a record's heads, none delivered. + for declared in 0..RECORD_TAG_LEN + 1 { + let encoded = supply_declaring(stream, Flow::End, declared, &[]); let error = decode_both(speaker, zero, &encoded) - .expect_err("a headerless over-budget body cannot decode"); + .expect_err("a headless over-budget body cannot decode"); assert!( matches!(error.kind, DecodeErrorKind::OverbatchedRun { .. }), "declared {declared}: {:?}", @@ -669,13 +1010,16 @@ fn overbatched_corners_classify_exactly() { ); } - // A first record header falling short of the body (two records' - // shapes) and one overrunning it: both are the violation. - let two_records = [record(&Version::new(), &Message::new(1))] - .concat() - .repeat(2); - let mut overrun = (200_u32).to_be_bytes().to_vec(); - overrun.extend_from_slice(&[0; 8]); + // A first record falling short of the body (two records' shapes) + // and one overrunning it: both are the violation. + let two_records = record(&Version::new(), &Message::new(1)).repeat(2); + let overrun = { + let mut record = Vec::new(); + cbor::write_head(&mut record, MAJOR_TAG, TAG_CBOR_SEQUENCE); + cbor::write_head(&mut record, MAJOR_BSTR, 200); + record.extend_from_slice(&[0; 8]); + record + }; for body in [two_records, overrun] { let error = decode_both(speaker, zero, &supply(stream, Flow::End, &body)) .expect_err("a non-spanning first record cannot decode over budget"); @@ -686,12 +1030,12 @@ fn overbatched_corners_classify_exactly() { ); } - // Ends inside the first record header, and inside an admitted lone + // Ends inside the first record's heads, and inside an admitted lone // record's body: truncations of the supply run, not violations. let lone = record(&Version::new(), &Message::new(1)); let encoded = supply(stream, Flow::End, &lone); - let header_end = 1 + LENGTH_HEADER_LEN; - for cut in [header_end + 2, encoded.len() - 1] { + let heads_end = encoded.len() - lone.len() + 1; + for cut in [heads_end, encoded.len() - 1] { let error = decode_both(speaker, zero, &encoded[..cut]) .expect_err("a truncated frame cannot decode"); assert!( @@ -708,3 +1052,52 @@ fn overbatched_corners_classify_exactly() { } } } + +/// A hand-crafted record whose payload nests one scope past the peer's +/// depth limit dies typed at wire ingress, while the same shape at +/// exactly the limit decodes clean, pinning the boundary. +/// +/// Send-side admission binds only this crate's own senders, so a +/// nonconforming implementation's over-deep supply must still surface as +/// `DecodeLeafError::Message` (invalid data), never as a panic or an +/// untyped abort. +#[test] +fn an_over_deep_supplied_payload_dies_typed_at_ingress() { + /// The receiving payload type: pure array nesting, the innermost + /// array empty, matching the hand-crafted bytes below. + #[derive(Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize)] + struct Arr(Vec); + let limit = PayloadDepthLimit::default(); + let deep_payload = |depth: usize| -> Vec { + // `depth - 1` single-element array heads around one empty array: + // nesting depth is exactly `depth` scopes. + let mut bytes = vec![0x81; depth - 1]; + bytes.push(0x80); + bytes + }; + let record_with_payload = |payload: &[u8]| -> Vec { + let mut content = Vec::new(); + cbor::write_head(&mut content, MAJOR_TAG, crate::tags::VERSION_TAG); + ciborium::ser::into_writer(&Version::new(), &mut content).unwrap(); + content.extend_from_slice(payload); + content + }; + let codec = PayloadCodec::new::(limit); + + // One scope past the limit: typed rejection at the record iterator. + let over = record_with_payload(&deep_payload(limit.get() as usize + 1)); + let run = LeafRun::from_encoded(raw_record(&over)).unwrap(); + let error = run.records(codec).next().unwrap().unwrap_err(); + let DecodeLeafError::Message(source) = error else { + panic!("an over-deep payload must fail as a message decode error"); + }; + assert_eq!(source.kind(), std::io::ErrorKind::InvalidData); + + // Exactly at the limit: the same shape decodes clean. + let at = record_with_payload(&deep_payload(limit.get() as usize)); + let run = LeafRun::from_encoded(raw_record(&at)).unwrap(); + run.records(codec) + .next() + .unwrap() + .expect("a payload at exactly the limit decodes"); +} diff --git a/src/tree/mirror/streaming/remote/codec/encode.rs b/src/tree/mirror/streaming/remote/codec/encode.rs index f62bf7326..654e3f3f9 100644 --- a/src/tree/mirror/streaming/remote/codec/encode.rs +++ b/src/tree/mirror/streaming/remote/codec/encode.rs @@ -3,10 +3,7 @@ #[cfg(test)] use std::io::Write; -use crate::tree::{ - mirror::framing::{LENGTH_HEADER_LEN, length_header}, - typed::Hash, -}; +use crate::tree::mirror::cbor::{self, MAJOR_ARRAY, MAJOR_BSTR, MAJOR_UINT, TAG_CBOR_SEQUENCE}; mod async_io; @@ -14,7 +11,7 @@ pub use async_io::FrameWrite; use super::{ error::EncodeErrorKind, - frame::{Frame, LeafRun, QUERY_COUNT_BIAS, Reaction}, + frame::{Frame, LeafRun, Reaction, write_listing}, signal::{Signal, Stream, WireSignal}, }; #[cfg(test)] @@ -42,20 +39,22 @@ pub fn encode( /// The encoder is not a trust boundary: phase placement, query ordering, and /// run record framing are guaranteed by its callers and checked only when /// bytes enter from the wire. Construction performs only the -/// representational checks needed before any byte can be emitted. +/// representational checks needed before any byte can be emitted, and +/// renders every head — so the write paths move bytes without measuring +/// anything. struct FrameEncoding<'a> { - signal: [u8; WireSignal::ENCODED_LEN], + /// The frame's array head and signal head. + head: Vec, body: BodyEncoding<'a>, } enum BodyEncoding<'a> { Empty, - Query { - count: [u8; 1], - children: &'a [(u8, Hash)], - }, + /// A nonempty query's child-listing map, fully rendered. + Listing(Vec), + /// A supply run behind its rendered embedded-sequence head. Supply { - header: [u8; LENGTH_HEADER_LEN], + head: Vec, run: &'a LeafRun, }, } @@ -68,40 +67,65 @@ impl<'a> FrameEncoding<'a> { (Signal::QueryEmpty(*flow), BodyEncoding::Empty) } Frame::Reaction(Reaction::Query(children), flow) => { - let count = u8::try_from(children.len() - QUERY_COUNT_BIAS) - .expect("a protocol query never exceeds the radix fan"); - ( - Signal::Query(*flow), - BodyEncoding::Query { - count: [count], - children, - }, - ) + let mut listing = Vec::new(); + write_listing(&mut listing, children); + (Signal::Query(*flow), BodyEncoding::Listing(listing)) } Frame::Reaction(Reaction::Supply(run), flow) => { - let header = length_header(run.encoded_len())?; - (Signal::Supply(*flow), BodyEncoding::Supply { header, run }) + let len = super::frame::checked_run_len(run.encoded_len())?; + let mut head = Vec::new(); + cbor::write_tag(&mut head, TAG_CBOR_SEQUENCE); + cbor::write_head(&mut head, MAJOR_BSTR, len); + (Signal::Supply(*flow), BodyEncoding::Supply { head, run }) } Frame::End(end) => (Signal::End(*end), BodyEncoding::Empty), }; - let signal = [WireSignal::encode(stream, signal)]; - Ok(Self { signal, body }) + let arity = match &body { + BodyEncoding::Empty => 1, + BodyEncoding::Listing(_) | BodyEncoding::Supply { .. } => 2, + }; + let mut head = Vec::new(); + cbor::write_head(&mut head, MAJOR_ARRAY, arity); + cbor::write_head( + &mut head, + MAJOR_UINT, + u64::from(WireSignal::encode(stream, signal)), + ); + Ok(Self { head, body }) + } + + /// Render the whole frame into one contiguous buffer: byte for byte + /// what the piece-wise writers emit, materialized only for an + /// attached observer's one-item view. + fn to_vec(&self) -> Vec { + let body_len = match &self.body { + BodyEncoding::Empty => 0, + BodyEncoding::Listing(listing) => listing.len(), + BodyEncoding::Supply { head, run } => head.len() + run.as_bytes().len(), + }; + let mut bytes = Vec::with_capacity(self.head.len() + body_len); + bytes.extend_from_slice(&self.head); + match &self.body { + BodyEncoding::Empty => {} + BodyEncoding::Listing(listing) => bytes.extend_from_slice(listing), + BodyEncoding::Supply { head, run } => { + bytes.extend_from_slice(head); + bytes.extend_from_slice(run.as_bytes()); + } + } + bytes } #[cfg(test)] fn write(&self, out: &mut impl Write) -> Result<(), EncodeErrorKind> { - write(out, FramePart::Signal, &self.signal)?; + write(out, FramePart::FrameHead, &self.head)?; match &self.body { BodyEncoding::Empty => {} - BodyEncoding::Query { count, children } => { - write(out, FramePart::QueryCount, count)?; - for (radix, hash) in *children { - write(out, FramePart::QueryChildren, std::slice::from_ref(radix))?; - write(out, FramePart::QueryChildren, hash.as_bytes())?; - } + BodyEncoding::Listing(listing) => { + write(out, FramePart::QueryChildren, listing)?; } - BodyEncoding::Supply { header, run } => { - write(out, FramePart::SupplyLength, header)?; + BodyEncoding::Supply { head, run } => { + write(out, FramePart::SupplyLength, head)?; write(out, FramePart::SupplyRun, run.as_bytes())?; } } diff --git a/src/tree/mirror/streaming/remote/codec/encode/async_io.rs b/src/tree/mirror/streaming/remote/codec/encode/async_io.rs index fa834f2bc..9b88cae22 100644 --- a/src/tree/mirror/streaming/remote/codec/encode/async_io.rs +++ b/src/tree/mirror/streaming/remote/codec/encode/async_io.rs @@ -2,6 +2,8 @@ use tokio::io::{AsyncWrite, AsyncWriteExt}; +use crate::observe::StreamObserver; + use super::super::{ error::{EncodeError, EncodeErrorKind, FramePart}, frame::WireFrame, @@ -17,12 +19,25 @@ use super::{BodyEncoding, FrameEncoding}; pub struct FrameWrite { speaker: Speaker, write: W, + /// The directed stream's observer, if any: handed each flushed + /// frame's contiguous bytes, and costing one branch when absent. + observe: Option>, } impl FrameWrite { /// Bind `write` to the direction spoken by `speaker`. pub fn new(speaker: Speaker, write: W) -> Self { - Self { speaker, write } + Self { + speaker, + write, + observe: None, + } + } + + /// Deliver every flushed frame to `observe`, when one is attached. + pub fn observed(mut self, observe: Option>) -> Self { + self.observe = observe; + self } /// Recover the transport writer without buffered frame state. Every @@ -48,7 +63,14 @@ impl FrameWrite { let result = async { let encoding = FrameEncoding::new(*stream, frame)?; write_encoding(&mut self.write, &encoding).await?; - self.write.flush().await.map_err(EncodeErrorKind::Flush) + self.write.flush().await.map_err(EncodeErrorKind::Flush)?; + // The frame is on the wire: deliver its one-item view. The + // pieces are the bytes just written, so the materialized + // buffer equals the wire by construction. + if let Some(observe) = &mut self.observe { + observe.message(&encoding.to_vec()); + } + Ok(()) } .await; result.map_err(|kind| EncodeError::new(self.speaker, *stream, kind)) @@ -59,18 +81,14 @@ async fn write_encoding( out: &mut (impl AsyncWrite + Unpin), encoding: &FrameEncoding<'_>, ) -> Result<(), EncodeErrorKind> { - write(out, FramePart::Signal, &encoding.signal).await?; + write(out, FramePart::FrameHead, &encoding.head).await?; match &encoding.body { BodyEncoding::Empty => {} - BodyEncoding::Query { count, children } => { - write(out, FramePart::QueryCount, count).await?; - for (radix, hash) in *children { - write(out, FramePart::QueryChildren, std::slice::from_ref(radix)).await?; - write(out, FramePart::QueryChildren, hash.as_bytes()).await?; - } + BodyEncoding::Listing(listing) => { + write(out, FramePart::QueryChildren, listing).await?; } - BodyEncoding::Supply { header, run } => { - write(out, FramePart::SupplyLength, header).await?; + BodyEncoding::Supply { head, run } => { + write(out, FramePart::SupplyLength, head).await?; write(out, FramePart::SupplyRun, run.as_bytes()).await?; } } diff --git a/src/tree/mirror/streaming/remote/codec/encode/tests.rs b/src/tree/mirror/streaming/remote/codec/encode/tests.rs index 4be2237e7..0f20b7443 100644 --- a/src/tree/mirror/streaming/remote/codec/encode/tests.rs +++ b/src/tree/mirror/streaming/remote/codec/encode/tests.rs @@ -16,9 +16,10 @@ use crate::{ use super::super::{ error::Origin, - frame::{MAX_QUERY_CHILDREN, QUERY_CHILD_LEN, QUERY_COUNT_BIAS, QUERY_COUNT_LEN}, + frame::{MAX_QUERY_CHILDREN, listing_len}, signal::{End, Flow, Speaker, Stream}, }; +use crate::tree::mirror::cbor::{MAJOR_TAG, MAJOR_UINT, TAG_CBOR_SEQUENCE}; const SPEAKERS: [Speaker; 2] = [Speaker::Initiator, Speaker::Responder]; const FLOWS: [Flow; 2] = [Flow::Continue, Flow::End]; @@ -33,6 +34,14 @@ fn signal(stream: Stream, signal: Signal) -> u8 { .to_byte() } +/// The frame head of a `arity`-item frame carrying `code`. +fn frame_head(arity: u64, code: u8) -> Vec { + let mut head = Vec::new(); + cbor::write_head(&mut head, MAJOR_ARRAY, arity); + cbor::write_head(&mut head, MAJOR_UINT, u64::from(code)); + head +} + fn arb_speaker() -> impl Strategy { prop_oneof![Just(Speaker::Initiator), Just(Speaker::Responder)] } @@ -41,7 +50,8 @@ fn arb_flow() -> impl Strategy { prop_oneof![Just(Flow::Continue), Just(Flow::End)] } -/// Every query fan and flow state has one canonical count representation. +/// Every query fan and flow state has one canonical map representation, +/// its length priced exactly by the listing closed form. #[test] fn query_count_covers_every_fan_and_flow() { let stream = stream(7); @@ -61,23 +71,24 @@ fn query_count_covers_every_fan_and_flow() { let mut encoded = Vec::new(); encode(speaker, &frame, &mut encoded).unwrap(); if count == 0 { - assert_eq!(encoded, [signal(stream, Signal::QueryEmpty(flow))]); - } else { - assert_eq!(encoded[0], signal(stream, Signal::Query(flow))); - assert_eq!(encoded[1], (count - QUERY_COUNT_BIAS) as u8); assert_eq!( - encoded.len(), - WireSignal::ENCODED_LEN + QUERY_COUNT_LEN + count * QUERY_CHILD_LEN + encoded, + frame_head(1, signal(stream, Signal::QueryEmpty(flow))) ); + } else { + let head = frame_head(2, signal(stream, Signal::Query(flow))); + assert_eq!(&encoded[..head.len()], head.as_slice()); + assert_eq!(encoded.len(), head.len() + listing_len(&children)); } } } } } -/// Match flow and both bare ends exhaust their one-byte representations. +/// Match flow and both bare ends exhaust the body-free representations: +/// each is exactly its one-item array head and signal. #[test] -fn one_byte_frames_are_exhaustive() { +fn body_free_frames_are_exhaustive() { let stream = stream(4); let cases: Vec<(WireFrame, u8)> = vec![ ( @@ -101,15 +112,18 @@ fn one_byte_frames_are_exhaustive() { for (frame, expected) in &cases { let mut encoded = Vec::new(); encode(speaker, frame, &mut encoded).unwrap(); - assert_eq!(encoded, [*expected]); + assert_eq!(encoded, frame_head(1, *expected)); } } } proptest! { - /// Supply framing is exact for an arbitrary run of backend-neutral leaf - /// records: one run length header, then one length-prefixed record per - /// leaf, in push order. + /// Supply framing is exact for an arbitrary run of backend-neutral + /// leaf records. + /// + /// The layout: the run's embedded-sequence heads, then one record item + /// per leaf, in push order — each record the tagged version atom and + /// bare payload behind its own embedded-sequence heads. #[test] fn supplied_run_is_framed_exactly( index in 1_u8..Stream::MAX, @@ -123,18 +137,21 @@ proptest! { for (version, value) in &records { let message = Message::new(*value); run.push(version, &message).unwrap(); - let mut record = Vec::new(); - ciborium::ser::into_writer(version, &mut record).unwrap(); - record.extend_from_slice(message.as_slice()); - body.extend_from_slice(&(record.len() as u32).to_be_bytes()); - body.extend_from_slice(&record); + let mut content = Vec::new(); + cbor::write_head(&mut content, MAJOR_TAG, crate::tags::VERSION_TAG); + ciborium::ser::into_writer(version, &mut content).unwrap(); + content.extend_from_slice(message.as_slice()); + cbor::write_head(&mut body, MAJOR_TAG, TAG_CBOR_SEQUENCE); + cbor::write_head(&mut body, MAJOR_BSTR, content.len() as u64); + body.extend_from_slice(&content); } let frame = (stream, Frame::Reaction(Reaction::Supply(run), flow)); let mut encoded = Vec::new(); encode(speaker, &frame, &mut encoded).unwrap(); - let mut expected = vec![signal(stream, Signal::Supply(flow))]; - expected.extend_from_slice(&(body.len() as u32).to_be_bytes()); + let mut expected = frame_head(2, signal(stream, Signal::Supply(flow))); + cbor::write_head(&mut expected, MAJOR_TAG, TAG_CBOR_SEQUENCE); + cbor::write_head(&mut expected, MAJOR_BSTR, body.len() as u64); expected.extend_from_slice(&body); prop_assert_eq!(encoded, expected); } @@ -164,7 +181,7 @@ fn writer_errors_are_contextual() { assert!(matches!( error.kind, EncodeErrorKind::Write { - part: FramePart::Signal, + part: FramePart::FrameHead, source, } if source.kind() == std::io::ErrorKind::Other )); @@ -215,7 +232,7 @@ fn async_writer_errors_are_contextual() { assert!(matches!( error.kind, EncodeErrorKind::Write { - part: FramePart::Signal, + part: FramePart::FrameHead, source, } if source.kind() == std::io::ErrorKind::Other )); diff --git a/src/tree/mirror/streaming/remote/codec/error.rs b/src/tree/mirror/streaming/remote/codec/error.rs index 68405f945..091a74957 100644 --- a/src/tree/mirror/streaming/remote/codec/error.rs +++ b/src/tree/mirror/streaming/remote/codec/error.rs @@ -10,7 +10,7 @@ use super::signal::{DecodeSignalError, Speaker, Stream}; /// The speaker and, when known, logical stream which produced an error. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum Origin { - /// The direction is known, but no signal byte supplied a stream yet. + /// The direction is known, but no signal supplied a stream yet. Direction(Speaker), /// Both the direction and logical stream are known. Stream { speaker: Speaker, stream: Stream }, @@ -37,16 +37,16 @@ impl fmt::Display for Origin { } } -/// The absent component of a truncated frame. +/// The absent or malformed component of a frame. #[derive(Debug, Clone, Copy, thiserror::Error, PartialEq, Eq)] pub enum FramePart { - #[error("signal byte")] + #[error("frame head")] + FrameHead, + #[error("signal")] Signal, - #[error("query count")] - QueryCount, #[error("query child listing")] QueryChildren, - #[error("supply run length")] + #[error("supply run head")] SupplyLength, #[error("supply run")] SupplyRun, @@ -128,12 +128,32 @@ pub enum DecodeErrorKind { }, #[error(transparent)] QueryOutOfOrder(#[from] QueryOrderError), + /// The frame item is not a one- or two-element CBOR array. + #[error("frame is not a CBOR reaction array: {detail}")] + FrameShape { detail: &'static str }, + /// The frame array's length contradicts its signal's body arity. + #[error("frame array carries {found} item(s) where its signal takes {expected}")] + FrameArity { expected: u64, found: u64 }, + /// A frame component was present but not canonical CBOR of the + /// expected shape. + #[error("frame's {part} is malformed: {detail}")] + Malformed { + part: FramePart, + detail: &'static str, + }, #[error(transparent)] InvalidRun(#[from] LeafRunError), #[error( - "supply frame occupies {declared} wire bytes, batching records past the {budget}-byte run budget" + "supply frame charges {declared} wire bytes, batching records past the {budget}-byte run budget" )] - OverbatchedRun { declared: usize, budget: usize }, + OverbatchedRun { + /// The frame's charged wire size — its run body plus the + /// `SUPPLY_FRAME_OVERHEAD` envelope at its widest — which may + /// exceed the actual frame by the envelope's head slack (at + /// most 4 bytes). + declared: usize, + budget: usize, + }, #[error("{count} trailing bytes follow the frame")] TrailingBytes { count: usize }, } diff --git a/src/tree/mirror/streaming/remote/codec/frame.rs b/src/tree/mirror/streaming/remote/codec/frame.rs index 9b21f0b2f..6f43ba199 100644 --- a/src/tree/mirror/streaming/remote/codec/frame.rs +++ b/src/tree/mirror/streaming/remote/codec/frame.rs @@ -2,9 +2,12 @@ use crate::{ Version, - message::{Message, PayloadDeserializer}, + message::{Message, PayloadCodec}, tree::{ - mirror::framing::{LENGTH_HEADER_LEN, LengthOverflow, length_header}, + mirror::cbor::{ + self, HeadError, MAJOR_BSTR, MAJOR_MAP, MAJOR_TAG, MAJOR_UINT, TAG_CBOR_SEQUENCE, + }, + mirror::framing::LengthOverflow, typed::{Hash, hash::MERKLE_HASH_LEN}, }, }; @@ -12,20 +15,25 @@ use crate::{ use super::error::{DecodeLeafError, QueryOrderError}; use super::signal::{End, Flow, Stream}; -/// The count byte stores one less than the nonempty query's actual fan. -pub const QUERY_COUNT_BIAS: usize = 1; +/// Largest query fan a listing map can carry: one child per radix value. +pub const MAX_QUERY_CHILDREN: usize = 256; -/// Largest query fan representable by a count-minus-one byte. -pub const MAX_QUERY_CHILDREN: usize = u8::MAX as usize + QUERY_COUNT_BIAS; +/// Bytes of the byte-string head ahead of one listed Merkle hash. +pub const HASH_HEAD_LEN: usize = cbor::head_len(MERKLE_HASH_LEN as u64); -/// Bytes occupied by one query child: its radix followed by its Merkle hash. -pub const QUERY_CHILD_LEN: usize = std::mem::size_of::() + MERKLE_HASH_LEN; +/// Bytes one listed child occupies as a map entry: its radix key's head, +/// then its hash value's head and digest bytes. Radixes of 24 and above +/// take a two-byte key head; smaller radixes take one. +pub const fn listing_entry_len(radix: u8) -> usize { + cbor::head_len(radix as u64) + HASH_HEAD_LEN + MERKLE_HASH_LEN +} -/// Bytes occupied by the count-minus-one field of a nonempty query. -pub const QUERY_COUNT_LEN: usize = std::mem::size_of::(); +/// Head bytes of the embedded-CBOR-sequence tag (63) opening every supply +/// run and every record within one. +pub(super) const RECORD_TAG_LEN: usize = cbor::head_len(TAG_CBOR_SEQUENCE); -/// Items in the adjacent-child window used to validate strict ordering. -const ADJACENT_CHILD_COUNT: usize = 2; +/// Head bytes of the version-atom tag ahead of a record's version. +const VERSION_TAG_LEN: usize = cbor::head_len(crate::tags::VERSION_TAG); /// The body of one complete reaction frame. #[derive(Debug, Clone, PartialEq, Eq)] @@ -44,28 +52,36 @@ pub enum Frame { End(End), } -/// A frame paired with the logical stream named by its signal byte. +/// A frame paired with the logical stream named by its signal. pub type WireFrame = (Stream, Frame); /// One supply frame's run of leaf records, held in encoded form. /// -/// A run is a delimited sequence of one or more `(Version, Message)` -/// records: each record is a [`LENGTH_HEADER_LEN`]-byte big-endian length -/// followed by one CBOR value (a byte string wrapping the version's -/// canonical encoding) and then the message's CBOR payload, back to back — -/// the record header delimits the payload, so it travels bare, and the -/// version's CBOR framing is what lets the decoder split the two without -/// re-measuring. The run stays encoded on both sides of the wire — the encoder -/// appends records copied from borrowed leaf data ([`push`](Self::push)) and -/// the decoder yields them one at a time ([`records`](Self::records)) — so -/// neither side materializes a decoded vector of leaves per frame; the bound -/// is one run's bytes. +/// A run is a CBOR sequence of one or more records. Each record is an +/// embedded-sequence item — tag 63 wrapping a byte string — whose content +/// is itself a two-item CBOR sequence: the version atom (its own tag +/// wrapping a byte string of the version's canonical encoding) followed by +/// the message's CBOR payload. The record's byte-string head delimits the +/// payload, so the payload travels bare, and the version's framing is what +/// lets the decoder split the two without re-measuring. The run stays +/// encoded on both sides of the wire — the encoder appends records copied +/// from borrowed leaf data ([`push`](Self::push)) and the decoder yields +/// them one at a time ([`records`](Self::records)) — so neither side +/// materializes a decoded vector of leaves per frame; the bound is one +/// run's bytes. /// -/// Construction guarantees record framing: [`push`] rejects a record no run -/// body can carry within the wire's `u32` frame header, and -/// [`from_encoded`](Self::from_encoded) rejects wire bytes whose headers do -/// not chain exactly to the end. A [`records`] iterator therefore never -/// fails structurally, only on a record's canonical content. +/// Construction guarantees record framing: [`push`] rejects a record no +/// run body can carry within the wire's run byte cap, and +/// [`from_encoded`](Self::from_encoded) rejects wire bytes whose record +/// items do not chain exactly to the end in canonical form. A [`records`] +/// iterator therefore never fails structurally, only on a record's +/// content: a version-atom tag that is missing, non-canonical, or cut +/// short by the record's end (the tag's head is hand-parsed and +/// spelling-judged), a version item the general CBOR reader cannot +/// decode behind that tag, a version atom whose content bytes fail the +/// strict [`Version`] decoder (the atom's byte-string head is read by +/// that general reader and not re-judged for spelling), or an +/// application payload that does not decode. /// /// [`push`]: Self::push /// [`records`]: Self::records @@ -116,7 +132,8 @@ impl LeafRun { self.bytes.is_empty() } - /// Bytes this run occupies on the wire, excluding signal and run length. + /// Bytes this run occupies on the wire, excluding the frame head and + /// the run's own embedded-sequence head. pub fn encoded_len(&self) -> usize { self.bytes.len() } @@ -128,16 +145,25 @@ impl LeafRun { /// Bytes one record with these components will occupy in a run. /// - /// Exactly what [`push`](Self::push) writes — the record header, the - /// version's CBOR byte-string framing plus its canonical bytes, and - /// the payload — pinned against an actual push by - /// `record_len_matches_an_actual_push`. Saturating: a sum past - /// `usize::MAX` cannot occur for in-memory slices, and an over-large - /// record is rejected by [`push`](Self::push) regardless. + /// Exactly what [`push`](Self::push) writes — the record's + /// embedded-sequence tag and byte-string head, the version atom's tag + /// and byte-string framing plus its canonical bytes, and the payload — + /// pinned against an actual push by `record_len_matches_an_actual_push`. + /// Saturating: a sum past `usize::MAX` cannot occur for in-memory + /// slices, and an over-large record is rejected by [`push`](Self::push) + /// regardless. pub fn record_len(version: &Version, message: &Message) -> usize { + let body = Self::record_body_len(version, message); + RECORD_TAG_LEN + .saturating_add(cbor::head_len(body as u64)) + .saturating_add(body) + } + + /// Bytes of a record's content behind its embedded-sequence head. + fn record_body_len(version: &Version, message: &Message) -> usize { let version = version.as_bytes().len(); - LENGTH_HEADER_LEN - .saturating_add(cbor_bytes_header_len(version)) + VERSION_TAG_LEN + .saturating_add(cbor::head_len(version as u64)) .saturating_add(version) .saturating_add(message.as_slice().len()) } @@ -146,45 +172,57 @@ impl LeafRun { /// /// # Errors /// - /// Rejects a record no run can carry — one whose combined encoding plus - /// its own record header exceeds the `u32` run-body limit — leaving the - /// run untouched. + /// Rejects a record no run can carry — one whose whole record item + /// exceeds the wire's run byte cap — leaving the run untouched. pub fn push(&mut self, version: &Version, message: &Message) -> Result<(), LengthOverflow> { + let body = Self::record_body_len(version, message); + let item = RECORD_TAG_LEN + .saturating_add(cbor::head_len(body as u64)) + .saturating_add(body); + checked_run_len(item)?; let version = version.as_bytes(); let message = message.as_slice(); - let len = cbor_bytes_header_len(version.len()) - .saturating_add(version.len()) - .saturating_add(message.len()); - let header = checked_record_header(len)?; - self.bytes.reserve(LENGTH_HEADER_LEN + len); - self.bytes.extend_from_slice(&header); - write_cbor_bytes_header(&mut self.bytes, version.len()); + self.bytes.reserve(item); + cbor::write_tag(&mut self.bytes, TAG_CBOR_SEQUENCE); + cbor::write_head(&mut self.bytes, MAJOR_BSTR, body as u64); + cbor::write_tag(&mut self.bytes, crate::tags::VERSION_TAG); + cbor::write_head(&mut self.bytes, MAJOR_BSTR, version.len() as u64); self.bytes.extend_from_slice(version); self.bytes.extend_from_slice(message); Ok(()) } - /// Validate wire bytes as a run: nonempty, headers chaining exactly. + /// Validate wire bytes as a run: nonempty, canonical record items + /// chaining exactly to the end. pub fn from_encoded(bytes: Vec) -> Result { if bytes.is_empty() { return Err(LeafRunError::Empty); } let mut rest = bytes.as_slice(); while !rest.is_empty() { - if rest.len() < LENGTH_HEADER_LEN { - return Err(LeafRunError::TruncatedHeader { - remaining: rest.len(), + let remaining = rest.len(); + let len = match record_head(&mut rest) { + Ok(len) => len, + Err(RecordHeadError::Head(source)) => { + return Err(LeafRunError::Head { remaining, source }); + } + Err(RecordHeadError::NotARecord(detail)) => { + return Err(LeafRunError::NotARecord { remaining, detail }); + } + }; + let Ok(len) = usize::try_from(len) else { + return Err(LeafRunError::NotARecord { + remaining, + detail: "record exceeds the run byte cap", }); - } - let (header, body) = rest.split_at(LENGTH_HEADER_LEN); - let len = record_header(header); - if body.len() < len { + }; + if rest.len() < len { return Err(LeafRunError::TruncatedRecord { len, - remaining: body.len(), + remaining: rest.len(), }); } - rest = &body[len..]; + rest = &rest[len..]; } Ok(Self { bytes }) } @@ -197,13 +235,13 @@ impl LeafRun { /// Iterate the run's records, decoding each into its canonical pair. pub fn records( &self, - deserializer: PayloadDeserializer, + codec: PayloadCodec, ) -> impl Iterator> { self.record_slices() - .map(move |record| parse_record(record, deserializer)) + .map(move |record| parse_record(record, codec)) } - /// Split the validated run back into its exact record slices. + /// Split the validated run back into its exact record contents. /// /// `pub(super)` for the capture renderer, which decodes each /// record's version structurally without knowing the leaf type. @@ -212,7 +250,7 @@ impl LeafRun { } } -/// Iterator over the exact record bodies of a structurally valid run. +/// Iterator over the exact record contents of a structurally valid run. pub(super) struct RecordSlices<'a> { rest: &'a [u8], } @@ -224,137 +262,270 @@ impl<'a> Iterator for RecordSlices<'a> { if self.rest.is_empty() { return None; } - let (header, body) = self.rest.split_at(LENGTH_HEADER_LEN); - let (record, rest) = body.split_at(record_header(header)); + let len = record_head(&mut self.rest).expect("a validated run chains canonical records"); + let (record, rest) = self + .rest + .split_at(usize::try_from(len).expect("a validated record fits in memory")); self.rest = rest; Some(record) } } -/// The record header for a `len`-byte record, checked against the outer frame. +/// A record's leading heads were not a canonical embedded-sequence item. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(super) enum RecordHeadError { + Head(HeadError), + NotARecord(&'static str), +} + +/// Parse one record's leading heads — the embedded-sequence tag and its +/// byte-string head — off the front of `input`, returning the record's +/// content length. +pub(super) fn record_head(input: &mut &[u8]) -> Result { + let head = cbor::read_head(input).map_err(RecordHeadError::Head)?; + if head.major != MAJOR_TAG || head.value != TAG_CBOR_SEQUENCE { + return Err(RecordHeadError::NotARecord( + "record does not open with the embedded-sequence tag", + )); + } + let head = cbor::read_head(input).map_err(RecordHeadError::Head)?; + if head.major != MAJOR_BSTR { + return Err(RecordHeadError::NotARecord( + "record tag does not wrap a byte string", + )); + } + Ok(head.value) +} + +/// Check a run body length against the wire's run byte cap. /// -/// A record is only encodable if the smallest run body holding it — the -/// record's bytes plus its own [`LENGTH_HEADER_LEN`]-byte header — fits the -/// wire's `u32` frame header, so the check charges the record header too. -/// [`LeafRun::push`] rejects on this boundary eagerly: an unshippable record -/// fails at record level rather than later at the outer frame. -fn checked_record_header(len: usize) -> Result<[u8; LENGTH_HEADER_LEN], LengthOverflow> { - length_header(len.saturating_add(LENGTH_HEADER_LEN))?; - Ok(length_header(len).expect("bounded by the header-charged check above")) +/// The encoder's boundary: a run the cap rejects was necessarily a single +/// record (the budget saturates below the cap, so a multi-record run never +/// grows here), and [`LeafRun::push`] already rejected any such record — +/// this check is the belt to that suspender, priced identically. +pub(super) fn checked_run_len(len: usize) -> Result { + // The cap is exactly the u32 range, so the failed conversion is the + // overflow witness. + match u32::try_from(len) { + Ok(len) => Ok(u64::from(len)), + Err(source) => Err(LengthOverflow { len, source }), + } } -/// Read one record header; construction guarantees its width. -fn record_header(header: &[u8]) -> usize { - u32::from_be_bytes( - header - .try_into() - .expect("a validated run chunks exact record headers"), - ) as usize +/// Whether a run body of `len` bytes is exactly one record: the first +/// record's heads plus the content they declare span the body. +/// +/// The lone-record test of the run-budget ingress check, shared by the +/// async reader and the sync oracle so the two decoders draw the +/// over-budget legality boundary identically. A body this predicate +/// rejects may also be structurally malformed; over budget, that +/// distinction is moot — either way the frame is not the one legal +/// overhang — so the check does not refine it further. +pub(super) fn lone_record_spans(len: usize, record_content: u64) -> bool { + (RECORD_TAG_LEN as u64) + .saturating_add(cbor::head_len(record_content) as u64) + .saturating_add(record_content) + == len as u64 } -/// Decode one exact record body into its canonical pair. -fn parse_record( - record: &[u8], - deserializer: PayloadDeserializer, -) -> Result<(Version, Message), DecodeLeafError> { - // Both fields are self-delimiting CBOR values, so the exact record - // body parses without retrying, and whatever the payload's parse does - // not consume is trailing. +/// Decode one exact record content into its canonical pair. +fn parse_record(record: &[u8], codec: PayloadCodec) -> Result<(Version, Message), DecodeLeafError> { + // The version atom's tag is protocol vocabulary, read here by hand; + // the byte string behind it and the payload are self-delimiting CBOR + // values, so the exact record content parses without retrying, and + // whatever the payload's parse does not consume is trailing. fn de_error(e: ciborium::de::Error) -> std::io::Error { match e { ciborium::de::Error::Io(e) => e, e => std::io::Error::new(std::io::ErrorKind::InvalidData, e.to_string()), } } + fn invalid(message: &str) -> std::io::Error { + std::io::Error::new(std::io::ErrorKind::InvalidData, message.to_string()) + } let mut input = record; + match cbor::read_head(&mut input) { + Ok(head) if head.major == MAJOR_TAG && head.value == crate::tags::VERSION_TAG => {} + Ok(_) => { + return Err(DecodeLeafError::Version(invalid( + "supplied version does not carry the version-atom tag", + ))); + } + // A record too short to hold the version's tag ran out of bytes, + // the same class as a version cut mid-encoding. + Err(HeadError::Truncated) => { + return Err(DecodeLeafError::Version(std::io::Error::new( + std::io::ErrorKind::UnexpectedEof, + "record ends inside the version atom's tag", + ))); + } + Err(e) => return Err(DecodeLeafError::Version(invalid(&e.to_string()))), + } let version: Version = ciborium::de::from_reader(&mut input).map_err(|e| DecodeLeafError::Version(de_error(e)))?; - // The deserializer owns the payload parse, including the + // The payload codec owns the payload parse, including the // exactly-one-value check the record framing otherwise cannot make // (the payload runs to the record's end), so trailing bytes surface // as its InvalidData. - let message = Message::from_wire(bytes::Bytes::copy_from_slice(input), deserializer) + let message = Message::from_wire(bytes::Bytes::copy_from_slice(input), codec) .map_err(DecodeLeafError::Message)?; Ok((version, message)) } -/// Bytes of the CBOR definite-length byte-string header for a `len`-byte -/// payload: the major-type-2 initial byte, plus the argument's width. -/// -/// The dual of [`write_cbor_bytes_header`]; `record_len` prices with one -/// and `push` writes with the other, and the -/// `record_len_matches_an_actual_push` pin holds them together. -fn cbor_bytes_header_len(len: usize) -> usize { - match len { - 0..=23 => 1, - 24..=0xff => 2, - 0x100..=0xffff => 3, - 0x1_0000..=0xffff_ffff => 5, - _ => 9, - } -} - -/// Append the CBOR definite-length byte-string header for a `len`-byte -/// payload: exactly what [`ciborium`] emits for `serialize_bytes`. -fn write_cbor_bytes_header(out: &mut Vec, len: usize) { - const MAJOR_BYTES: u8 = 2 << 5; - match len { - 0..=23 => out.push(MAJOR_BYTES | len as u8), - 24..=0xff => out.extend_from_slice(&[MAJOR_BYTES | 24, len as u8]), - 0x100..=0xffff => { - out.push(MAJOR_BYTES | 25); - out.extend_from_slice(&(len as u16).to_be_bytes()); - } - 0x1_0000..=0xffff_ffff => { - out.push(MAJOR_BYTES | 26); - out.extend_from_slice(&(len as u32).to_be_bytes()); - } - _ => { - out.push(MAJOR_BYTES | 27); - out.extend_from_slice(&(len as u64).to_be_bytes()); - } - } -} - /// A supply run whose record framing is structurally invalid. #[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)] pub enum LeafRunError { /// Every supply frame carries at least one record. #[error("a supply run carries no leaf records")] Empty, - /// A record header overruns the run's declared length. - #[error("a leaf record header overruns the {remaining} bytes left in its run")] - TruncatedHeader { remaining: usize }, - /// A record body overruns the run's declared length. + /// A record's leading heads are truncated or non-canonical. + #[error("a leaf record's heads are invalid in the {remaining} bytes left in its run: {source}")] + Head { + remaining: usize, + #[source] + source: HeadError, + }, + /// The bytes where a record belongs are some other CBOR item. + #[error("a {remaining}-byte run tail is not a leaf record: {detail}")] + NotARecord { + remaining: usize, + detail: &'static str, + }, + /// A record's content overruns the run's declared length. #[error("a leaf record of {len} bytes overruns the {remaining} bytes left in its run")] TruncatedRecord { len: usize, remaining: usize }, } -/// Validate that a radix listing is in canonical order: strictly ascending. -/// -/// This is the one gate every child listing entering from the wire passes, -/// whichever surface carries it — a query frame's body or the greeting's -/// root-fan listing. Strictness is the whole invariant: the canonical form -/// admits each radix at most once, so an equal adjacent pair is rejected -/// exactly like a descent. +/// One structural problem in a child-listing map. /// -/// # Errors +/// Every child listing entering from the wire — a query frame's body or +/// the greeting's root-fan listing — passes one structural gate, and +/// this names how a listing failed it. +#[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)] +#[non_exhaustive] +pub enum ListingIssue { + /// A head was truncated, indefinite, reserved, or widened. + #[error("{0}")] + Head(HeadError), + /// An item had the wrong major type, value range, or count. + #[error("{0}")] + Shape(&'static str), + /// The digest bytes behind a value head were cut short. + #[error("listing hash bytes are truncated")] + Truncated, + /// Adjacent keys were not strictly ascending. + #[error("{0}")] + Order(QueryOrderError), +} + +/// Incrementally validated state of one child-listing map's entries. /// -/// The first adjacent non-ascending pair reports both radices as a -/// [`QueryOrderError`]. -pub fn validate_children(children: &[(u8, Hash)]) -> Result<(), QueryOrderError> { - for pair in children.windows(ADJACENT_CHILD_COUNT) { - let [previous, current] = pair else { - unreachable!("an adjacent-child window contains exactly two items") - }; - if previous.0 >= current.0 { - return Err(QueryOrderError { - previous: previous.0, - radix: current.0, - }); +/// This is the one gate every child listing entering from the wire +/// passes, whichever surface carries it — a query frame's body or the +/// greeting's root-fan listing — and whichever reader drives it (the +/// async decoder, the sync oracle, or the slice parser). The map's +/// deterministic-encoding key order and the wire's canonical child order +/// are one discipline: keys must be strictly ascending radixes, so an +/// equal adjacent pair is rejected exactly like a descent +/// ([`ListingIssue::Order`]). +pub(super) struct ListingBuilder { + children: Vec<(u8, Hash)>, + previous: Option, +} + +impl ListingBuilder { + /// Accept a map head of `count` entries within the radix space. + pub(super) fn new(count: u64) -> Result { + if count > MAX_QUERY_CHILDREN as u64 { + return Err(ListingIssue::Shape("listing exceeds the radix space")); + } + Ok(Self { + children: Vec::with_capacity(count as usize), + previous: None, + }) + } + + /// Accept one entry's key head: an unsigned radix, strictly above the + /// previous key. + pub(super) fn key(&mut self, head: cbor::Head) -> Result { + if head.major != MAJOR_UINT || head.value > u64::from(u8::MAX) { + return Err(ListingIssue::Shape("listing key is not a radix")); + } + let radix = head.value as u8; + if let Some(previous) = self.previous + && previous >= radix + { + return Err(ListingIssue::Order(QueryOrderError { previous, radix })); + } + self.previous = Some(radix); + Ok(radix) + } + + /// Accept one entry's value head: a byte string of exactly one digest. + pub(super) fn value_head(head: cbor::Head) -> Result<(), ListingIssue> { + if head.major != MAJOR_BSTR || head.value != MERKLE_HASH_LEN as u64 { + return Err(ListingIssue::Shape("listing value is not a Merkle hash")); + } + Ok(()) + } + + /// Record one entry whose key and value heads were accepted. + pub(super) fn entry(&mut self, radix: u8, hash: [u8; MERKLE_HASH_LEN]) { + self.children.push((radix, Hash(hash))); + } + + /// Yield the validated children. + pub(super) fn finish(self) -> Vec<(u8, Hash)> { + self.children + } +} + +/// Parse one complete child-listing map off the front of `input`, +/// advancing past it. +pub(crate) fn parse_listing_map(input: &mut &[u8]) -> Result, ListingIssue> { + let head = cbor::read_head(input).map_err(ListingIssue::Head)?; + if head.major != MAJOR_MAP { + return Err(ListingIssue::Shape("listing is not a map")); + } + let count = head.value; + let mut listing = ListingBuilder::new(count)?; + for _ in 0..count { + let key = cbor::read_head(input).map_err(ListingIssue::Head)?; + let radix = listing.key(key)?; + let value = cbor::read_head(input).map_err(ListingIssue::Head)?; + ListingBuilder::value_head(value)?; + if input.len() < MERKLE_HASH_LEN { + return Err(ListingIssue::Truncated); } + let (digest, rest) = input.split_at(MERKLE_HASH_LEN); + *input = rest; + listing.entry(radix, digest.try_into().expect("split at the digest width")); + } + Ok(listing.finish()) +} + +/// Append one child listing as a canonical map: ascending radix keys, +/// each hash a definite-length byte string. +/// +/// The encoder is not a trust boundary — callers guarantee canonical +/// child order — so this writes without revalidating it. +pub(crate) fn write_listing(out: &mut Vec, children: &[(u8, Hash)]) { + cbor::write_head(out, MAJOR_MAP, children.len() as u64); + for (radix, hash) in children { + cbor::write_head(out, MAJOR_UINT, u64::from(*radix)); + cbor::write_head(out, MAJOR_BSTR, MERKLE_HASH_LEN as u64); + out.extend_from_slice(hash.as_bytes()); + } +} + +/// Bytes a whole child listing occupies as a map: its head plus entries. +#[cfg(test)] +pub fn listing_len(children: &[(u8, Hash)]) -> usize { + let mut total = cbor::head_len(children.len() as u64); + for (radix, _) in children { + total += listing_entry_len(*radix); } - Ok(()) + total } #[cfg(test)] diff --git a/src/tree/mirror/streaming/remote/codec/frame/tests.rs b/src/tree/mirror/streaming/remote/codec/frame/tests.rs index f21de3348..626d0a7e5 100644 --- a/src/tree/mirror/streaming/remote/codec/frame/tests.rs +++ b/src/tree/mirror/streaming/remote/codec/frame/tests.rs @@ -1,48 +1,36 @@ use super::*; +use crate::message::{PayloadCodec, PayloadDepthLimit}; -/// The largest record `push` admits: with its own record header charged, the -/// smallest run body holding it exactly fills the outer `u32` frame header. -const LARGEST_ENCODABLE_RECORD: usize = u32::MAX as usize - LENGTH_HEADER_LEN; +use crate::tree::mirror::cbor::HeadError; -/// Push's capacity check is eager and charges the record header. -/// -/// A record is admitted exactly when its bytes plus its own header fit the -/// outer `u32` frame header, so a record with length in -/// `(u32::MAX - 4, u32::MAX]` fails at record level rather than later at -/// the outer frame. -#[test] -fn record_capacity_charges_the_record_header() { - assert!(checked_record_header(LARGEST_ENCODABLE_RECORD).is_ok()); - for unshippable in [ - LARGEST_ENCODABLE_RECORD + 1, - u32::MAX as usize, - u32::MAX as usize + 1, - ] { - let error = checked_record_header(unshippable) - .expect_err("a record past the header-charged boundary must fail"); - assert_eq!(error.len, unshippable.saturating_add(LENGTH_HEADER_LEN)); - } +/// The record heads ahead of a record's content: the embedded-sequence +/// tag plus the byte-string head for `content` bytes. +fn record_heads(content: usize) -> usize { + RECORD_TAG_LEN + cbor::head_len(content as u64) } -/// The checked header encodes the record's own length, not the charged sum: -/// the header-charged boundary changes only admission, never the wire bytes -/// of an admitted record. +/// Push's capacity check is eager and charges the record's whole item. +/// +/// A record is admitted exactly when its heads plus its content fit the +/// wire's run byte cap, so a record item with length past `u32::MAX` +/// fails at record level rather than later at the run head. #[test] -fn checked_header_encodes_the_bare_record_length() { - let len = 7; - assert_eq!( - checked_record_header(len).expect("a small record is admitted"), - (len as u32).to_be_bytes(), - ); +fn record_capacity_charges_the_whole_item() { + assert!(checked_run_len(u32::MAX as usize).is_ok()); + for unshippable in [u32::MAX as usize + 1, u32::MAX as usize + 2] { + let error = checked_run_len(unshippable) + .expect_err("a record item past the run byte cap must fail"); + assert_eq!(error.len, unshippable); + } } /// `record_len` prices exactly what `push` writes, at every CBOR -/// byte-string header width a version can occupy. +/// byte-string head width a version can occupy. /// /// The two are the same quantity computed two ways — arithmetic against /// actual encoding — so the run-budget math can trust the closed form. /// Deep version chains grow the canonical encoding through the 1-byte -/// (< 24), 2-byte (< 256), and 3-byte (< 65536) CBOR header regimes; the +/// (< 24), 2-byte (< 256), and 3-byte (< 65536) CBOR head regimes; the /// chain lengths below land encodings in the first two and the message /// sizes sweep the payload term. #[test] @@ -52,12 +40,12 @@ fn record_len_matches_an_actual_push() { for parties in 1..=128u32 { // One tick on a fresh disjoint party per step: each new party's // event widens the canonical encoding, marching it through the - // CBOR header-width regimes. + // CBOR head-width regimes. version.tick(&crate::tree::arb::nth_party(parties as usize)); if !(parties == 1 || parties % 17 == 0) { continue; } - checked_regimes.insert(super::cbor_bytes_header_len(version.as_bytes().len())); + checked_regimes.insert(cbor::head_len(version.as_bytes().len() as u64)); for message in [Message::new(0u64), Message::new(u64::MAX)] { let mut run = LeafRun::new(); run.push(&version, &message).expect("test records fit"); @@ -66,19 +54,129 @@ fn record_len_matches_an_actual_push() { LeafRun::record_len(&version, &message), "record_len must price exactly one pushed record", ); - // The version atom `push` writes is byte-identical to the - // serde form the decoder parses (ciborium's byte string). + // Behind the record's heads and the version-atom tag, the + // version `push` writes is byte-identical to the serde form + // the decoder parses (ciborium's byte string). let mut serde_form = Vec::new(); ciborium::ser::into_writer(&version, &mut serde_form).unwrap(); + let content = LeafRun::record_body_len(&version, &message); + let at = record_heads(content) + VERSION_TAG_LEN; assert_eq!( - &run.as_bytes()[LENGTH_HEADER_LEN..LENGTH_HEADER_LEN + serde_form.len()], + &run.as_bytes()[at..at + serde_form.len()], serde_form.as_slice(), - "push's hand-written CBOR header must match ciborium's", + "push's hand-written version framing must match ciborium's", ); } } assert!( checked_regimes.len() >= 2, - "the sweep must cross at least two CBOR header-width regimes, got {checked_regimes:?}", + "the sweep must cross at least two CBOR head-width regimes, got {checked_regimes:?}", ); } + +/// A pushed run round-trips through `from_encoded` and yields the same +/// records: the writer's record heads are exactly what the validator +/// chains over, one record at a time. +#[test] +fn pushed_runs_validate_and_iterate() { + let mut version = crate::Version::new(); + version.tick(&crate::tree::arb::nth_party(1)); + let mut run = LeafRun::new(); + for payload in [1u64, 2, 3] { + run.push(&version, &Message::new(payload)) + .expect("test records fit"); + } + let decoded = + LeafRun::from_encoded(run.as_bytes().to_vec()).expect("a pushed run is structurally valid"); + let payloads: Vec = decoded + .records(PayloadCodec::new::(PayloadDepthLimit::default())) + .map(|record| *record.expect("a pushed record decodes").1.arc::()) + .collect(); + assert_eq!(payloads, vec![1, 2, 3]); + assert_eq!(decoded.record_count(), 3); +} + +/// A run whose record opens with anything but the embedded-sequence tag, +/// or whose head is widened past shortest form, is rejected typed: the +/// deterministic contract holds inside runs too. +#[test] +fn malformed_record_heads_are_typed() { + // A bare byte string where a tagged record belongs. + let mut bytes = Vec::new(); + cbor::write_head(&mut bytes, MAJOR_BSTR, 1); + bytes.push(0); + assert!(matches!( + LeafRun::from_encoded(bytes), + Err(LeafRunError::NotARecord { .. }) + )); + // A widened (non-shortest) byte-string head behind a valid tag. + let mut bytes = Vec::new(); + cbor::write_tag(&mut bytes, TAG_CBOR_SEQUENCE); + bytes.extend_from_slice(&[0x58, 0x01, 0x00]); // 1 spelled wide + assert!(matches!( + LeafRun::from_encoded(bytes), + Err(LeafRunError::Head { + source: HeadError::NotShortest, + .. + }) + )); + // A record whose declared content overruns the run. + let mut bytes = Vec::new(); + cbor::write_tag(&mut bytes, TAG_CBOR_SEQUENCE); + cbor::write_head(&mut bytes, MAJOR_BSTR, 4); + bytes.push(0); + assert!(matches!( + LeafRun::from_encoded(bytes), + Err(LeafRunError::TruncatedRecord { + len: 4, + remaining: 1 + }) + )); + // The empty run. + assert!(matches!( + LeafRun::from_encoded(Vec::new()), + Err(LeafRunError::Empty) + )); +} + +/// The listing writer and the listing parser are inverses on every +/// canonical listing, and the parser holds keys strictly ascending: +/// the map's deterministic key order and the wire's canonical child +/// order are one rule. +#[test] +fn listings_round_trip_and_hold_canonical_order() { + use proptest::prelude::*; + proptest!(|(radixes in proptest::collection::btree_set(any::(), 0..=64))| { + let children: Vec<(u8, Hash)> = radixes + .iter() + .map(|&radix| (radix, Hash([radix; MERKLE_HASH_LEN]))) + .collect(); + let mut bytes = Vec::new(); + write_listing(&mut bytes, &children); + prop_assert_eq!(bytes.len(), listing_len(&children)); + let mut input = bytes.as_slice(); + let parsed = parse_listing_map(&mut input).expect("a written listing is canonical"); + prop_assert_eq!(parsed, children); + prop_assert!(input.is_empty()); + }); +} + +/// A listing with a descending or repeated key is rejected with the +/// order violation, exactly like a wire query: an equal adjacent pair is +/// as non-canonical as a descent. +#[test] +fn unordered_listings_are_rejected() { + for (previous, radix) in [(3u8, 3u8), (5, 2)] { + let children = [ + (previous, Hash([0; MERKLE_HASH_LEN])), + (radix, Hash([1; MERKLE_HASH_LEN])), + ]; + let mut bytes = Vec::new(); + write_listing(&mut bytes, &children); + let mut input = bytes.as_slice(); + assert_eq!( + parse_listing_map(&mut input), + Err(ListingIssue::Order(QueryOrderError { previous, radix })), + ); + } +} diff --git a/src/tree/mirror/streaming/remote/codec/greeting.rs b/src/tree/mirror/streaming/remote/codec/greeting.rs new file mode 100644 index 000000000..bdd2e2602 --- /dev/null +++ b/src/tree/mirror/streaming/remote/codec/greeting.rs @@ -0,0 +1,306 @@ +//! The V2 greeting's wire spelling. +//! +//! One control-stream item: an embedded-CBOR-item tag (24) wrapping a +//! byte string whose content is a text-keyed map. The embedding is what +//! keeps the control stream's reader trivial — the byte string's head +//! declares the whole greeting's length up front, so no incremental map +//! walk happens against the transport — while a generic tool unwraps +//! tag 24 as part of the standard vocabulary and sees the map. +//! +//! The map's keys ride in CBOR deterministic order (bytewise +//! lexicographic over their encodings), and the decoder requires exactly +//! this key set in exactly that order: one spelling per greeting. +//! +//! - `"listing"`: the sender's root-fan listing, the same +//! `{radix: hash}` map spelling a query frame carries. +//! - `"set_len"`: the sender's declared set size. +//! - `"version"`: the sender's causal version — the version-atom tag +//! wrapping a byte string of the version's canonical encoding. +//! - `"max_version_bytes"`: the sender's version-size bound. +//! - `"payload_depth_limit"`: the sender's payload nesting-depth limit, +//! which the counterparty's must equal for the session to proceed. +//! - `"target_message_size"`: the sender's supply-run byte target. + +use crate::{ + Version, + tree::mirror::cbor::{ + self, HeadError, MAJOR_BSTR, MAJOR_TAG, MAJOR_TEXT, MAJOR_UINT, TAG_EMBEDDED_ITEM, + }, + tree::mirror::streaming::message::Greeting, +}; + +use super::error::QueryOrderError; +use super::frame::{ListingIssue, parse_listing_map, write_listing}; + +/// The greeting map's keys, in the deterministic (bytewise lexicographic) +/// order the wire requires. +const KEYS: [&str; 6] = [ + "listing", + "set_len", + "version", + "max_version_bytes", + "payload_depth_limit", + "target_message_size", +]; + +/// Render one greeting as its complete control-stream item: +/// tag 24 wrapping a byte string of the greeting map. +pub(crate) fn encode_greeting(greeting: &Greeting) -> Vec { + let map = greeting_map(greeting); + let mut item = Vec::with_capacity( + cbor::head_len(TAG_EMBEDDED_ITEM) + cbor::head_len(map.len() as u64) + map.len(), + ); + cbor::write_tag(&mut item, TAG_EMBEDDED_ITEM); + cbor::write_head(&mut item, MAJOR_BSTR, map.len() as u64); + item.extend_from_slice(&map); + item +} + +/// Render the greeting map alone. +fn greeting_map(greeting: &Greeting) -> Vec { + let mut map = Vec::new(); + cbor::write_head(&mut map, cbor::MAJOR_MAP, KEYS.len() as u64); + for key in KEYS { + cbor::write_head(&mut map, MAJOR_TEXT, key.len() as u64); + map.extend_from_slice(key.as_bytes()); + match key { + "listing" => write_listing(&mut map, &greeting.listing), + "set_len" => cbor::write_head(&mut map, MAJOR_UINT, greeting.set_len), + "version" => { + let version = greeting.version.as_bytes(); + cbor::write_tag(&mut map, crate::tags::VERSION_TAG); + cbor::write_head(&mut map, MAJOR_BSTR, version.len() as u64); + map.extend_from_slice(version); + } + "max_version_bytes" => { + cbor::write_head(&mut map, MAJOR_UINT, greeting.max_version_bytes); + } + "payload_depth_limit" => { + cbor::write_head(&mut map, MAJOR_UINT, greeting.payload_depth_limit); + } + "target_message_size" => { + cbor::write_head(&mut map, MAJOR_UINT, greeting.target_message_size); + } + _ => unreachable!("the key roster is exhaustive"), + } + } + map +} + +/// A greeting that is not canonical rumors CBOR. +/// +/// Carried by [`RemoteError::HandshakeDecode`]: the greeting item +/// arrived, but its spelling or content violates the wire's +/// deterministic-encoding contract. The greeting admits one spelling per +/// value, so every variant here is a counterparty bug, never an +/// alternate encoding. +/// +/// [`RemoteError::HandshakeDecode`]: super::super::RemoteError::HandshakeDecode +#[derive(Debug, thiserror::Error)] +#[non_exhaustive] +pub enum GreetingError { + /// A head was truncated, indefinite, reserved, or widened. + #[error("greeting head is not canonical: {0}")] + Head(HeadError), + /// An item had the wrong major type, value, or position. + #[error("greeting is malformed: {0}")] + Shape(&'static str), + /// The listing map violated a structural rule. + #[error("greeting listing is malformed: {0}")] + Listing(ListingIssue), + /// The listing's keys were not in canonical strictly ascending order. + #[error(transparent)] + Order(QueryOrderError), + /// The version atom's bytes are not one canonical version encoding. + #[error("greeting version does not decode: {0}")] + Version(before::error::Decode), +} + +/// Parse a greeting map from the embedded byte string's exact content. +pub(crate) fn parse_greeting(bytes: &[u8]) -> Result { + let mut input = bytes; + let head = cbor::read_head(&mut input).map_err(GreetingError::Head)?; + if head.major != cbor::MAJOR_MAP || head.value != KEYS.len() as u64 { + return Err(GreetingError::Shape( + "greeting is not a map of one entry per roster key", + )); + } + let mut version = None; + let mut set_len = None; + let mut max_version_bytes = None; + let mut payload_depth_limit = None; + let mut target_message_size = None; + let mut listing = None; + for key in KEYS { + let head = cbor::read_head(&mut input).map_err(GreetingError::Head)?; + if head.major != MAJOR_TEXT || head.value != key.len() as u64 { + return Err(GreetingError::Shape( + "greeting keys are not the deterministic roster", + )); + } + let Some((text, rest)) = split(input, key.len()) else { + return Err(GreetingError::Shape("greeting key is truncated")); + }; + input = rest; + if text != key.as_bytes() { + return Err(GreetingError::Shape( + "greeting keys are not the deterministic roster", + )); + } + match key { + "listing" => { + listing = Some(parse_listing_map(&mut input).map_err(|issue| match issue { + ListingIssue::Order(order) => GreetingError::Order(order), + issue => GreetingError::Listing(issue), + })?); + } + "set_len" => set_len = Some(uint(&mut input, "set_len is not an unsigned int")?), + "version" => { + let head = cbor::read_head(&mut input).map_err(GreetingError::Head)?; + if head.major != MAJOR_TAG || head.value != crate::tags::VERSION_TAG { + return Err(GreetingError::Shape( + "greeting version does not carry the version-atom tag", + )); + } + let head = cbor::read_head(&mut input).map_err(GreetingError::Head)?; + if head.major != MAJOR_BSTR { + return Err(GreetingError::Shape( + "greeting version tag does not wrap a byte string", + )); + } + let Ok(len) = usize::try_from(head.value) else { + return Err(GreetingError::Shape("greeting version outsizes memory")); + }; + let Some((atom, rest)) = split(input, len) else { + return Err(GreetingError::Shape("greeting version is truncated")); + }; + input = rest; + version = Some(Version::decode(atom).map_err(GreetingError::Version)?); + } + "max_version_bytes" => { + max_version_bytes = Some(uint( + &mut input, + "max_version_bytes is not an unsigned int", + )?); + } + "payload_depth_limit" => { + payload_depth_limit = Some(uint( + &mut input, + "payload_depth_limit is not an unsigned int", + )?); + } + "target_message_size" => { + target_message_size = Some(uint( + &mut input, + "target_message_size is not an unsigned int", + )?); + } + _ => unreachable!("the key roster is exhaustive"), + } + } + if !input.is_empty() { + return Err(GreetingError::Shape("greeting carries trailing bytes")); + } + Ok(Greeting { + version: version.expect("the roster visits version"), + set_len: set_len.expect("the roster visits set_len"), + max_version_bytes: max_version_bytes.expect("the roster visits max_version_bytes"), + payload_depth_limit: payload_depth_limit.expect("the roster visits payload_depth_limit"), + target_message_size: target_message_size.expect("the roster visits target_message_size"), + listing: listing.expect("the roster visits listing"), + }) +} + +/// Read one unsigned-int value, returning `detail` as the shape +/// diagnostic when the item is not an unsigned int. +fn uint(input: &mut &[u8], detail: &'static str) -> Result { + let head = cbor::read_head(input).map_err(GreetingError::Head)?; + if head.major != MAJOR_UINT { + return Err(GreetingError::Shape(detail)); + } + Ok(head.value) +} + +/// Split `len` leading bytes off `input`, or `None` when it is shorter. +fn split(input: &[u8], len: usize) -> Option<(&[u8], &[u8])> { + (input.len() >= len).then(|| input.split_at(len)) +} + +/// Read one complete greeting item from the control stream. +/// +/// Transport failures pass through as `Err(Ok-side io)`; a malformed or +/// non-canonical greeting is a typed [`GreetingError`], except a +/// non-canonical listing order, surfaced separately so the handshake can +/// report it as the codec's own violation class. +pub(crate) async fn read_greeting(read: &mut R) -> Result +where + R: tokio::io::AsyncRead + Unpin, +{ + use crate::tree::mirror::framing::read_payload; + let head = cbor::read_head_async(read) + .await + .map_err(head_read_error)? + .ok_or_else(|| { + ReadGreetingError::Io(std::io::Error::new( + std::io::ErrorKind::UnexpectedEof, + "peer closed before its greeting", + )) + })?; + if head.major != MAJOR_TAG || head.value != TAG_EMBEDDED_ITEM { + return Err(ReadGreetingError::Decode(GreetingError::Shape( + "greeting does not open with the embedded-item tag", + ))); + } + let head = cbor::read_head_async(read) + .await + .map_err(head_read_error)? + .ok_or_else(|| { + ReadGreetingError::Io(std::io::Error::new( + std::io::ErrorKind::UnexpectedEof, + "peer closed inside its greeting", + )) + })?; + if head.major != MAJOR_BSTR { + return Err(ReadGreetingError::Decode(GreetingError::Shape( + "greeting tag does not wrap a byte string", + ))); + } + let Ok(len) = usize::try_from(head.value) else { + return Err(ReadGreetingError::Decode(GreetingError::Shape( + "greeting declares an unaddressable length", + ))); + }; + let bytes = read_payload(read, len) + .await + .map_err(ReadGreetingError::Io)?; + parse_greeting(&bytes).map_err(|e| match e { + GreetingError::Order(order) => ReadGreetingError::Listing(order), + e => ReadGreetingError::Decode(e), + }) +} + +/// How reading a greeting from the control stream failed. +#[derive(Debug, thiserror::Error)] +pub(crate) enum ReadGreetingError { + /// The transport failed before the greeting arrived whole. + #[error(transparent)] + Io(std::io::Error), + /// The greeting arrived but is not canonical rumors CBOR. + #[error(transparent)] + Decode(GreetingError), + /// The greeting's listing violated canonical child order. + #[error(transparent)] + Listing(QueryOrderError), +} + +fn head_read_error(e: cbor::HeadReadError) -> ReadGreetingError { + match e { + cbor::HeadReadError::Io(io) => ReadGreetingError::Io(io), + cbor::HeadReadError::Malformed(head) => { + ReadGreetingError::Decode(GreetingError::Head(head)) + } + } +} + +#[cfg(test)] +mod tests; diff --git a/src/tree/mirror/streaming/remote/codec/greeting/tests.rs b/src/tree/mirror/streaming/remote/codec/greeting/tests.rs new file mode 100644 index 000000000..1d8f22591 --- /dev/null +++ b/src/tree/mirror/streaming/remote/codec/greeting/tests.rs @@ -0,0 +1,213 @@ +use super::*; + +use crate::tree::typed::{Hash, hash::MERKLE_HASH_LEN}; + +fn sample(listing: Vec<(u8, Hash)>) -> Greeting { + let mut version = crate::Version::new(); + version.tick(&crate::tree::arb::nth_party(1)); + Greeting { + version, + set_len: 7, + max_version_bytes: 4096, + payload_depth_limit: 300, + target_message_size: 1 << 20, + listing, + } +} + +/// Greeting encode and parse are inverses, listing shapes included: +/// empty, small-radix, and large-radix listings all round-trip through +/// the one wire spelling. +#[test] +fn greetings_round_trip() { + for listing in [ + Vec::new(), + vec![(0, Hash([1; MERKLE_HASH_LEN]))], + vec![ + (3, Hash([1; MERKLE_HASH_LEN])), + (24, Hash([2; MERKLE_HASH_LEN])), + (255, Hash([3; MERKLE_HASH_LEN])), + ], + ] { + let greeting = sample(listing); + let item = encode_greeting(&greeting); + // Strip the embedded-item tag and byte-string head, the layer the + // async reader consumes. + let mut input = item.as_slice(); + let head = cbor::read_head(&mut input).expect("the item opens with a head"); + assert_eq!((head.major, head.value), (MAJOR_TAG, TAG_EMBEDDED_ITEM)); + let head = cbor::read_head(&mut input).expect("the tag wraps a byte string"); + assert_eq!(head.major, MAJOR_BSTR); + assert_eq!(head.value as usize, input.len()); + let parsed = parse_greeting(input).expect("a written greeting parses"); + assert_eq!(parsed.version, greeting.version); + assert_eq!(parsed.set_len, greeting.set_len); + assert_eq!(parsed.max_version_bytes, greeting.max_version_bytes); + assert_eq!(parsed.payload_depth_limit, greeting.payload_depth_limit); + assert_eq!(parsed.target_message_size, greeting.target_message_size); + assert_eq!(parsed.listing, greeting.listing); + } +} + +/// The greeting's map admits exactly one spelling: a missing or +/// out-of-order key, or trailing bytes, are each rejected — one +/// spelling per greeting is the deterministic contract. +#[test] +fn greeting_key_roster_is_exact() { + let greeting = sample(Vec::new()); + let item = encode_greeting(&greeting); + let mut input = item.as_slice(); + cbor::read_head(&mut input).expect("tag head"); + cbor::read_head(&mut input).expect("bstr head"); + let map = input.to_vec(); + + // Renaming a key breaks the roster. + let mut wrong_key = map.clone(); + let at = find(&wrong_key, b"set_len", 0).expect("the key is present"); + wrong_key[at] = b'x'; + assert!(matches!( + parse_greeting(&wrong_key), + Err(GreetingError::Shape(_)) + )); + + // Trailing bytes are rejected. + let mut trailing = map.clone(); + trailing.push(0); + assert!(matches!( + parse_greeting(&trailing), + Err(GreetingError::Shape(_)) + )); +} + +/// A listing whose content ends inside a hash is rejected as the typed +/// listing issue. +/// +/// The map declares its listing entries up front; content that runs out +/// inside an entry's digest bytes must surface +/// [`GreetingError::Listing`] with the listing's own truncation, never a +/// panic and never a partial listing. +#[test] +fn truncated_listing_hash_is_a_typed_listing_issue() { + let greeting = sample(vec![(4, Hash([7; MERKLE_HASH_LEN]))]); + let item = encode_greeting(&greeting); + let mut input = item.as_slice(); + cbor::read_head(&mut input).expect("tag head"); + cbor::read_head(&mut input).expect("bstr head"); + // Cut one byte deeper than the listing's end (the next key's one-byte + // text head sits just before the key text): the kept bytes stop + // inside the entry's digest. + let at = find(input, b"set_len", 0).expect("the key is present"); + let cut = &input[..at - 2]; + assert!(matches!( + parse_greeting(cut), + Err(GreetingError::Listing(ListingIssue::Truncated)) + )); +} + +/// A version atom whose bytes are not one canonical version encoding is +/// rejected as the typed version defect. +/// +/// The atom's tag and byte string parse, so the failure is the content's +/// own: [`GreetingError::Version`] carrying the decoder's verdict. +#[test] +fn undecodable_version_atom_is_a_typed_version_defect() { + let greeting = sample(Vec::new()); + let item = encode_greeting(&greeting); + let mut input = item.as_slice(); + cbor::read_head(&mut input).expect("tag head"); + cbor::read_head(&mut input).expect("bstr head"); + let mut map = input.to_vec(); + + // Locate the version atom's content: after the "version" key text + // ride the version tag's head, the byte string's head, and then the + // encoded version itself; saturate those bytes. + let at = find(&map, b"version", 0).expect("the key is present"); + let mut cursor = &map[at + b"version".len()..]; + let before_heads = cursor.len(); + cbor::read_head(&mut cursor).expect("the version tag's head"); + let head = cbor::read_head(&mut cursor).expect("the version string's head"); + let content_at = at + b"version".len() + (before_heads - cursor.len()); + assert!(head.value > 0, "a ticked version encodes to content bytes"); + for byte in &mut map[content_at..content_at + head.value as usize] { + *byte = 0xFF; + } + assert!(matches!( + parse_greeting(&map), + Err(GreetingError::Version(_)) + )); +} + +/// A widened spelling of a greeting value head is rejected as the +/// codec's own shortest-form violation. +/// +/// The greeting is deterministic-encoding CBOR, so a head wider than +/// its value requires is a spelling the encoder never writes, even +/// though the value it carries is the right one. +#[test] +fn widened_value_spelling_is_rejected() { + // Build the malformed map directly: copy the canonical map bytes + // and re-spell the one-byte `set_len` value as the widened + // two-byte `0x18 ` form. Operating on the bare map (the layer + // `parse_greeting` consumes) needs no fix-up of an embedding + // byte-string head. + let greeting = sample(Vec::new()); + let map = greeting_map(&greeting); + let at = find(&map, b"set_len", 0).expect("the key is present"); + // The value head follows the key's text bytes. + let value_at = at + b"set_len".len(); + assert_eq!( + map[value_at], + u8::try_from(greeting.set_len).expect("the fixture's set_len is small"), + "the fixture's set_len spells as one canonical head byte" + ); + let mut widened = Vec::with_capacity(map.len() + 1); + widened.extend_from_slice(&map[..value_at]); + widened.extend_from_slice(&[0x18, map[value_at]]); + widened.extend_from_slice(&map[value_at + 1..]); + assert!(matches!( + parse_greeting(&widened), + Err(GreetingError::Head(HeadError::NotShortest)) + )); +} + +/// A greeting listing violating strictly ascending radix order is +/// rejected as the codec's own order violation, the same class a wire +/// query reports. +#[test] +fn greeting_listing_order_is_enforced() { + // The encoder trusts its caller, so an unsorted listing synthesizes + // the wire violation directly. + let greeting = sample(vec![ + (9, Hash([1; MERKLE_HASH_LEN])), + (5, Hash([2; MERKLE_HASH_LEN])), + ]); + let item = encode_greeting(&greeting); + let mut input = item.as_slice(); + cbor::read_head(&mut input).expect("tag head"); + cbor::read_head(&mut input).expect("bstr head"); + assert!(matches!( + parse_greeting(input), + Err(GreetingError::Order(QueryOrderError { + previous: 9, + radix: 5 + })) + )); +} + +// Defensive-variant exemption: the two unaddressable-length shapes — +// `Shape("greeting declares an unaddressable length")` in the greeting +// reader and `Shape("greeting version outsizes memory")` in the map +// parser — deliberately have no construction tests. Each guards a +// u64-to-usize length conversion that cannot fail on a 64-bit host; only +// a 32-bit target (e.g. wasm32) can present a declarable length past +// `usize::MAX`, and this suite has no 32-bit test host. + +/// Find the `skip`-th occurrence of `needle` in `haystack`. +fn find(haystack: &[u8], needle: &[u8], skip: usize) -> Option { + haystack + .windows(needle.len()) + .enumerate() + .filter(|(_, window)| *window == needle) + .map(|(at, _)| at) + .nth(skip) +} diff --git a/src/tree/mirror/streaming/remote/codec/signal.rs b/src/tree/mirror/streaming/remote/codec/signal.rs index cd93bb6b2..6019689ff 100644 --- a/src/tree/mirror/streaming/remote/codec/signal.rs +++ b/src/tree/mirror/streaming/remote/codec/signal.rs @@ -1,5 +1,6 @@ -//! The dense signal byte and its semantic components. +//! The dense signal code and its semantic components. +use crate::observe::Role; use crate::tree::typed::height::{Height, Root, UnderRoot, Z}; /// Lowest node height carried by a logical stream. @@ -121,6 +122,14 @@ impl Speaker { Speaker::Responder => Speaker::Initiator, } } + + /// This role in the observation hook's public vocabulary. + pub fn role(self) -> Role { + match self { + Speaker::Initiator => Role::Initiator, + Speaker::Responder => Role::Responder, + } + } } /// The phase-specific signal grammar of a logical stream. @@ -178,7 +187,7 @@ impl Flow { } } -/// The semantic state carried alongside a stream id in one signal byte. +/// The semantic state carried alongside a stream id in one signal code. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum Signal { Match(Flow), @@ -271,8 +280,11 @@ pub struct WireSignal { } impl WireSignal { - /// Bytes occupied by a densely encoded signal. - pub const ENCODED_LEN: usize = std::mem::size_of::(); + /// Widest head a dense signal code occupies as a CBOR unsigned int + /// item: every code of 24 and above takes a two-byte head, and the + /// code space tops out at 169. + pub const MAX_ENCODED_LEN: usize = + crate::tree::mirror::cbor::head_len((Signal::STATE_COUNT * Stream::COUNT - 1) as u64); /// Byte values occupied by the syntactic `(signal state, stream)` product. #[cfg(test)] @@ -288,7 +300,7 @@ impl WireSignal { Self::pair(stream, signal).validate(speaker) } - /// Parse and validate a dense wire byte for its speaker's protocol phase. + /// Parse and validate a dense code for its speaker's protocol phase. pub fn from_byte(speaker: Speaker, byte: u8) -> Result { Self::parse(byte)?.validate(speaker).map_err(Into::into) } @@ -342,7 +354,7 @@ impl WireSignal { } } - /// Render the paired stream and semantic signal as one dense wire byte. + /// Render the paired stream and semantic signal as the dense code. pub fn to_byte(self) -> u8 { self.signal.state() * Stream::COUNT + self.stream.index() } @@ -355,14 +367,14 @@ impl WireSignal { /// A valid signal state placed on a stream where the protocol forbids it. #[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)] -#[error("signal byte {byte:#04x} is invalid for {class}")] +#[error("signal code {byte:#04x} is invalid for {class}")] pub struct InvalidSignalPlacement { byte: u8, class: StreamClass, } impl InvalidSignalPlacement { - /// Return the rejected dense wire byte. + /// Return the rejected dense code. pub fn byte(self) -> u8 { self.byte } @@ -373,7 +385,7 @@ impl InvalidSignalPlacement { } } -/// A syntactically invalid signal byte or a valid state in an invalid phase. +/// A syntactically invalid signal code or a valid state in an invalid phase. #[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)] pub enum DecodeSignalError { #[error(transparent)] @@ -392,9 +404,9 @@ impl DecodeSignalError { } } -/// A reserved dense signal byte and the stream encoded within it. +/// A reserved dense signal code and the stream encoded within it. #[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)] -#[error("signal byte {byte:#04x} encodes an invalid semantic state")] +#[error("signal code {byte:#04x} encodes an invalid semantic state")] pub struct InvalidWireSignal { byte: u8, stream: Stream, @@ -403,7 +415,7 @@ pub struct InvalidWireSignal { } impl InvalidWireSignal { - /// Return the rejected dense wire byte. + /// Return the rejected dense code. pub fn byte(self) -> u8 { self.byte } diff --git a/src/tree/mirror/streaming/remote/codec/snapshots/rumors__tree__mirror__streaming__remote__codec__tests__bounded_corpus_manifest_snapshot.snap b/src/tree/mirror/streaming/remote/codec/snapshots/rumors__tree__mirror__streaming__remote__codec__tests__bounded_corpus_manifest_snapshot.snap index 501e88d9b..5ee59f0b8 100644 --- a/src/tree/mirror/streaming/remote/codec/snapshots/rumors__tree__mirror__streaming__remote__codec__tests__bounded_corpus_manifest_snapshot.snap +++ b/src/tree/mirror/streaming/remote/codec/snapshots/rumors__tree__mirror__streaming__remote__codec__tests__bounded_corpus_manifest_snapshot.snap @@ -10,363 +10,363 @@ Initiator QueryEmpty(End): cases 1 accepted 0 rejected 1 rejection Some(OpeningSupplies) digest bed1553e944c1f60caad77749acd1505d7b760208777c714f35ffdc0f2de766d Query(Continue): cases 32896 accepted 0 rejected 32896 rejection Some(OpeningSupplies) digest 9ba562d04ead543df788328565bd2f0dc9d4471e6e2bf7be09eec7d061dee9af Query(End): cases 32896 accepted 0 rejected 32896 rejection Some(OpeningSupplies) digest 387698c6a8f6cd20537f989b769a3033ffa03c71aaf4854dc14f2010d5673555 - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 59b5fd7fcfce8862da5a311e322502cc8fdcd5b1ead61bd8e6b6e532bf774ede - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest e84ed624d4c85c71009ba4d06f28a9b3c251d88711e3ee37f9c5877b511222ad - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest e6c8657a09bbd68e3a4410183521d0dc564776c29489da7eef314c391632057b - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest c6137a2a9996157c93f59583278135d535fbd3d5439da4de7af076a1a699dc2d + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest ef42a1f4f55d67a16c4425c0b892fc4d8ee7ef28b2745d077cd58f0bb5814980 + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest e4409433cc158bfb054b2001e551041927ed7c45e10012eb5d3309c841a78ba9 + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 8f78ad754b008c62ef8f7e0346ece60c634bf8cc48e00d8d8ef118aa8faf4f5a + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 122aa7dc19fb16cfede3a7b7b70eff5651b16ae1f2aba2acd02b4820bfd67d95 stream 01 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 9bbd1097cc86719184aa34340aa3f2c06a63dafb37b86914d45f36563186235d - Match(End): cases 1 accepted 1 rejected 0 rejection None digest 45ee114c5b1f0ed4b48840ede520d9c80af340fd119de086f63b8a976a1ec742 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 0bb26c70fe8f5bf17d6645b50caba9dbea86edef3b92e14f585637a64ed0f548 - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 066871f0d0267b1b42edbb3835aed267fd755c6f7a819c1420def0671f62628d - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest af9a3ba555a3bcdfa2108b28f6539d8620aeb2eda089bfb7896820f98ba0a827 - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest f29eaef5dc2c87a94692e1687365ee51a341ad822204604da619148f4f742247 - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 319b28c9c064036c6f2b1e30f665b32d6061b2f03a826d934a56845924030837 - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 4dfd3a14bb679a3f3c72a3bac874d2ecb167299fcf03f1543fcdd49d76144db8 - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 7efb04b3cd8ae0a43ffaaf027a7d1ae31f033da458ccf85ed6339227bdfce8c2 - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest d210087cbc67d20f09c0fda1159b051ad38c629fe9783dfaf4e0cbbe985ecdcc + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 822e221183582cc39ec93269c2f05600f45f6257060a6d7ae3fa807ffdab3bfe + Match(End): cases 1 accepted 1 rejected 0 rejection None digest c7b891e63ff7694bf97ddae67d2d58f763785ab525dc164923540e714a136b94 + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest b514e4add70536373f31eb2ee2be3e2955c5c830f6e9c2d88368303d08c50c2f + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest dcd78220f73334462fc05c0092732081ce054f8390d6deae9478413ebc51c4b0 + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 492c659c7e07c17f674abb88954a176fcf35038e0c8c2c1552a895a21806986b + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 154a3788c85d4a50eba9328b8351ad401a987d7d6bc2986f937f2afcd880285f + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest f812816df7b5bcc7158910f89f3493fed4e09e28a39fa7ae6e6986e1b6e877a6 + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 6bb66af11167bfe6b41ff77a4eba9a8dc8b77a6079c0818e84c0d99c1f9b5295 + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 3f7595c05aa461b3adfeb2720b3ea00bba3bf16236e5dc8dea498ef648d64d11 + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 5a6f8915e99b97a83b5a0b30ae413f0ebfa61fdfa177f656be1a95ebed9385c6 stream 02 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 8f3624873d4b068a073bda3cf2e280a5ca195bd9066d3b2e0b9c6156a33a5e6a - Match(End): cases 1 accepted 1 rejected 0 rejection None digest 1f286c89dd40d9dd0a323fc7a4aa9222aeeff0d82b0cdc1e337859c7fa57249b - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 7ceb016d4177164a41f5315046fb7a104e7b402b8544076656fba53cdc7c5b8a - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest b232233fad61fdfe489b32005295b43142c3da73df863366f532cf4e04dc0960 - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 6c6a82a4201c2095453ffcc81b644af49180d33c49a958cbbd221bd259a931a5 - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 1d2091676cd8510ae281d0959ba783251c31b696d764c878213a67cdddb2fe00 - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 97b2682d961d4e0fbb9ca8093663428d6c5852e37baa1905f652fde30e31b0b3 - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest fc1d59d32a9266c1a77486e6d76a9bee51d1bb4cd943d380038230f42b9dcf4b - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 5aaec826e8f1aa4bb1dc7f60095baf57a5b3cb9adcb0a4cf688407b0e7b7347e - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 2221d8b7741c597c23a781ae3a4315325660af324d10d667036b1c22c70fd4ff + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 8da023b966a9819fd558c2276909fc87966cced0c324a13db00489c2a1d4db1e + Match(End): cases 1 accepted 1 rejected 0 rejection None digest 3a166443f96064928189f46632d85d880cf414706aff03099c3e5fb7ef706047 + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 0eeb109cb452c2417fe4c9d280f3e435bbe8f7a0ade330d494415fdce47724a8 + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 71d47af427367e34a0420af2c849bb06c38db6ca107c725a2f525038485387a4 + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 00ed3d0aae4f3ec8d63e223e67834db5dfe001390e6fd2db5bf76ad288fb18fb + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 58bd9c9778eff881c835a53fb270cc781bae4331ca767e54b9ddb6c4a629176d + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 217373b0d58da1136bcce08bdc1ed633f939c84ea1d4fa5bc56d69406cb6b26d + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 97688245e619012a49a5043b1f4f45983a0ccf57481e071867d37dd9921ec24d + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest aa8428f399e0244514b082aa56c9fda2510488a978a320028b219050c8e4c7bf + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 8d5ad0ff596aebc75fd7a0435dac0d8e7f521278f3489faccbe2b5bdb7de7be1 stream 03 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 66fad94ed67e1ffe5a3c03b47e195ecfd7fe08e96e933146ab068802c62711de - Match(End): cases 1 accepted 1 rejected 0 rejection None digest c85c9376cd1443e95f4a955c78ce5bc0886a3886744b6c641db8640fa471e6fd - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest ac1261c496af02319a1772ff71e9813a3b4658b27bab4d1610fba06196b9ef51 - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 157b7b3c36e30debeb2364b3e9c29267d936ae5b68ac616aee5fc533a24b52fb - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest c51172601ee20ac3a04e5a32720d521af1af3a90d29b71d6337acbafbe5fc115 - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest f427d3759ade3b024bfc7f57189200fbe0e220ecc832df29864cfccb0283a740 - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest e3525955c5573149fbb36710cf3cca454db1eef8fc2d7dfbfbadd08096b6b14a - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 9f920581eb1636ed6e52b37f50109b03b6010e431e7173a0af29bfda972f1f64 - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 8dc4e0a82ef1dfa3953132cd6cbe54650221b73a98162927793d488162a38fac - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest edc61faa00b781b3d047bab5809faf0b1a74bdb20f202712d9e5dbbf805525b8 + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 18362a39b132ae13a3ef04803937260cd185f761b2147d2506202f206c05a611 + Match(End): cases 1 accepted 1 rejected 0 rejection None digest 2796a215960684c415193f031b414fafd5054a0876fc885d45b2becd4b2333ac + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 93d46dba804ced90724fe3100b67406bb81cb40b17aa5a8e846de9f9723b6a76 + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest a372666b42bde5f3679a47b32d5846a9aa427dd3d3a3b11571c8c5c9bb74159d + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 181528afe22b0e7986dfeb8ff683e1b0c039b03804fb5da51e4a0d0128dde6db + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest b428487914e610ac8d2385e50a29bfe26a08a12bb71c812fedbffd824610103b + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest e3d0d3c889b924fc0d81f7c8cca6ee888baf3d0a9578c79eeedc46938a028d76 + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest a2d2469022d06234ced10ab8ee31795662aed518588c7af109a8f4e41764e968 + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest b9031379ac21c36c75ce35461c068591fa0acfc37dc59e0bdb5c198a51cc2436 + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 624110158fafc1bc489ba666d6f3cfcb1c42e88948baf16de09a942382a9ed00 stream 04 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 0aabcb6009a818cb51cd563f227fc5100ad87c2cc07e8f95ee952d3fd840c3eb - Match(End): cases 1 accepted 1 rejected 0 rejection None digest 43511d45dc52dd6d8f66c218e6bcb8272bdbd57a53a228670dc8e999cb99b8e9 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest b435ac27a6a88d353d269371500517496cf8f3cc386895cc8ac3059abd1151f5 - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest bafaa10bfebbfd58e6dd9555c38d68f074a88cc7714d5cda3a8d4932990cd87f - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest aa3e0335828851588f14358567427c9634d7463286a2264198060a3d9c188e11 - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest dfc58272001597c14dfa88433653257a8a3b12091c74fc18f40c6fe3c74a588d - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 9d263dbb054de831ed5346a3301e63c7e338b02a834d0db1bcd11d1eb77105ad - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 4d2250bf01b9cc3ef793712d67669ccc61ef5c42984841d578bfd2a848b5331b - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 964e4c82824016b610d97562de5eb0d355b4d17cf16acb220581550c2cb08201 - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 09f3d01b5a48fee382807fb80ccfae7e9a2aa167b28ed706e8cd6e536752e14f + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 163d3634fc1418cc68a6632a5bf272b4e4d13752e0ebd8add7928e88d8fe3c84 + Match(End): cases 1 accepted 1 rejected 0 rejection None digest 116b0f61934959f642b115f3a0695318b8097c7e1cfff1ace24c8a94f32fab0a + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest b418d391ea72a381e693c8c7501adaf436976974b307f000547de7f0a4eb056f + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 7b428726940fb374efc9da2c7f1ec5eb99a1eef0d4873b886e9e02e5718b0c24 + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 4bdecfce7e2f1db9dc54b3f203871bfc44be54a7b9a73523b5026e4554377a65 + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest b17a10c5fab248a0a35dc69aee60d3941af501d2b73377be8c9b4212777d917f + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 112ba83fa34002a0d82c9e67586f92a1ee7ed2c801738c666d212549609399cf + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest fbe0e0af3533eb75f617af3aa735816a7a88a3b4043f365dcc771605a1829282 + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 0745e6e4b9d144d37c7dfcb1fd8c3b20bbfb772ee9d7d9524896c836bb7a2c07 + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest c336003c19e3d38c544445fa47a491cc0c0bf38267c67a53205bd0cdefe69bff stream 05 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 775ed8cb20fd4d5405bb3d83b4c81a29738bc999c1341d2c4ec38a96e9c2a478 - Match(End): cases 1 accepted 1 rejected 0 rejection None digest 0d6f8784c353242b50acc993f6af93394b09564469f4976a8ecf3809a4b7c340 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 716e53ce1b990161233ac6562d3672e38cce0a91b108ebe915e2362216af4d79 - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 41fc3692f19ac323b3b2d7c8bd0307425676297f8e0bed6b221753c173461fd0 - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 75abb3accb666e6145d70c2d102fdd6ce5f7ee38cfb7d4a33696051837bedbfd - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest fe90890210e48d3fac30d24e2fb66ad2a6273e6d8b67cff578a161b3062680c0 - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest b528e65d2440e583aead617a94d194f56699e05e71235a9229f907dced0a9b79 - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest df3b29ef01094a59fa27126fbc554570a1b89e4be9cbd9f525279aefa812c774 - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest f42bf364254585a59063ca719e3e466fabdcf8bb4de407f05c44bf8c1b75b0f4 - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 37008b172b7470e27c3df9f915d753b7a1c501bdab556c600eddff42c51e5df1 + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 8b2bcb0aa72efc91a4308a50158458b40c34a5e6298420da3cc4436a1a7438c8 + Match(End): cases 1 accepted 1 rejected 0 rejection None digest fa929e4b8d272af567fcda22b7c09e8cca387a9b4e5eb6cf108d05224e568a54 + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest a8feda23c84986de29557cf75e55cca962a9f421a81bb2fd611d73f272b5e411 + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest e36e489f824ce1a98b5efc940e70fd3d736138760f2bba5f0b69127edbdf77b8 + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 57160278519ca5794d4b9d5d6cc5c4c83112973411c5eadbb4e306de0a578925 + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 1312217b358ae3a9a3ecc386ab11f025d074112cf2bce302422ce08836689d75 + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 4091eab1ac41b4c8a07bcb9daa72e9d0dbfb679978902f531b3ffcde71ee56b9 + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 4798a3f3ed5923953ed9c4bde3eb03ad5cdb6302c1259783695acab7afb17e79 + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 674d77ceb6d82027d98029a77aca2fc7b864800d20c83bffb0e1ffc67c303f6a + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest d2de9dc2f96e7d6b3d5fc7946b7038862714c55cdf872e100e18c7687a460b50 stream 06 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 19dc8b6508fcb25bd215ff98745b10a0226d0b7802317ff23693170f6b69e2bf - Match(End): cases 1 accepted 1 rejected 0 rejection None digest b3f847c2acab7165aed0ab12dbf8095ddedff4da4f27b75719e598bd7a1bc9d5 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 05319d1eba637b4941839be91b827a4e17bbb9ea997a9de62021e8c51d44f28d - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest e11362eed2d9395eae0b5e66d4adf8e7b26d5660c549658c61c500fc61921a97 - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 14a04b8700e66157eb28db77fc19075aec4fb93ac3b4cf7ac8f4a7dc6d195548 - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 9095d15c02ae8055927f39ada986d42a974f1621d7a97a48f22619dab7f51eb3 - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest a2d7ee70ef3c9562a15d7077586fd0d24b4d8f81db5a4fb9869782458de72fb2 - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest a4561e3b2a42f881801ad9ab5d79618254f5157ec1a50043c81d7a53f3245f72 - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 29f62ebb0b44a04c6c29c315771602bf97d0e948fdfc9e889a3ec27572876e1f - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 127c84b8f7a667189b8accc3ead7f9d759231b6d5e9f4d521bf8a27acffd3e20 + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest e4f7848df6ca1afcccab12651d9356f43ace95ed972a996669aec12de64eb8e2 + Match(End): cases 1 accepted 1 rejected 0 rejection None digest a86d5f8b56f9221c2d5f944ee03ae4f5d1cf058ba32b2db7353cc01b2510fe4d + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest a51b552a13d95e1c413876140bc217d5dfb95132fdc83134ff19be18d1329999 + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest ec94b9af7046d9cf87c300043e51fbdaeb46af865e97c0fab3cb09dea22f6cde + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 275db08be66aa5ef32a6747f2ec7bdcb9ef867f8f726ecb3e852dbe2cd9598cc + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 9697014f4cee53799c3890119160281f811c65287adb31221289bf3f184259ce + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 784f02431c5bdb551b5c2185f787bc5f6832212602ef363fd48910a3dcda312f + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 16c13cdfbb93ca755d7304932c5035a1575e7c82c67acc6cfdc377009109daaf + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 3a313910a93bb9c6fce179115ac038716b01ba7e924f61ea4032287c26da7c25 + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest fc52adcaab5665aac8ffc0c4b20decec58547c5cf358cf009917f3dd21c445f5 stream 07 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest dafa9fae71d3877a3a621fcd8bf9f75723bab8f02d0dab6a6bb5ea6d608c2786 - Match(End): cases 1 accepted 1 rejected 0 rejection None digest 75d5f73f826776473c9d2a2560c371f3e7d0092b123648b34240d4f82c4acc36 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 074aedf4174f5bc56a17f56d49dda4403c9bfdf0ab9f7bff581fd3757af72129 - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 46acb5c7c4557b55a31329576477e76c7a9f9a41fb45b8c5907ed7b9ec11a433 - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 66a2f0ee0a7799997880a907305e992f4ac33bf602d96761235310a2149c9bc8 - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 5d04b5006b60a70a1413b657e481c3ed6d0a75da18ecc410b72d19872a2b17eb - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 5e300b4cb071daa8574f20507a7da4472111351f1e740723f9b5bf8097b8e41b - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 0a562f5dd6f838c161f4f48188a8b8783acedd8f3938c965213ef59fb0e6d6e3 - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 0abc151c61d1bf0abdbf8de84f9a373da4dd54f63d5ddde0d3963e68e7a1bf8e - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest ef47eedfe6e578be84ed170e1f5cfb0978893dda7ff8ea95c5e56c76947e5064 + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest c09a0b577898409cda79efcfb32ef0892830769ed6ca0cf360350e8bc2f53704 + Match(End): cases 1 accepted 1 rejected 0 rejection None digest dfd0ae5b28c32c8e65b61656edc45398602de1a83762a2094e07e2aa4c3314bd + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 506b9458de2178528097b78795cb9ccf8099c865801df55cb8be36a28dfdb419 + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 472449d11e02b2cd60077313c13c45acea914d48ce9dcb247f09922494b86bc4 + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 98e3af9d722bedd4b6b2e79a380033b1133fe0653450d2c4cc6734c92f8d0e6d + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest ba112066ab293fb2cfda6a508bbb5962ad7e3de98a38f4e0bfb9c466dd00f6fd + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 4c0fac1a9e9adcb0a6e0702b55c704405e5dbccd11e44d28358c508439272a41 + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest b8a1477fb43bf17a7af0dd27866f013120ef5c4062811ac507d23886e399f945 + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest b9848cca526ae9ad112553ccbf357526d94e7eb024067cd758d3d74d88d25282 + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest a86c8046f0029b1ad34120d3f84167bbde755f794d9955f6af0b29a1aa6dfef7 stream 08 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest bd306ad277db2515cc49b306b36b9c4c499e6a192339b74c44fd61f598215e78 - Match(End): cases 1 accepted 1 rejected 0 rejection None digest 780efde07897d67399fa95cb91d9e08698bb571eb13c07f1dc7c808bea43ded7 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 0e3fbd1149bcf46ed868882fb0fff74fcdd959d671835f6909e71b1ed95845eb - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 17f68790f5b708d77429611d19cade3367733a261a50c4b41037fd88026cbaa3 - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 5851952e464d7e55c7f95efb554bc5613a5bdce4ccec542581674532a90ec418 - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest e286e082de98f67d8a5e12fa2ce5d46de10d524b70764cd9de29d59be3b54c2c - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest f0b52c1605a67276d58f1baeb1abb4b1ec603d3f5d037ce404b8052298b13828 - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 130042645e4d5f9eac83eaa1271585f1147b8dcce58bffd94aecc4cac6e96b96 - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 8bb8f6555c5bdf08019175230fe7e440d3ab688fa49909e75da4ac8c3e091809 - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest f96f9bf58b82edb0d80376acd21f4f8b5458aeed6f63c82f79da08f83eb73601 + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 76a538374c611ae0f915298a269e642bc343015d2c2200076d8fcbd61479dc99 + Match(End): cases 1 accepted 1 rejected 0 rejection None digest a04abe29794ef9764b323839ecf5f7b82025da1b1b981cdadd59096cb619e5fc + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest f2d8b2e7629631594e1e896a108847a2bb8f30b61dfb36c965db44cb089c7fcb + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest a1e8eab09925fabebaa69dae25c80fad8878c464666782d3d06997659ed01b20 + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest e78c3ea97c25284e115d9877aff71a71b456788785542e0e0999d8ffa5476340 + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 879dd07d45e47be9a3e5165a351810fa677600872c012104c228229d913d6a8f + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest b46dc4e98f83be2b7be8186e6acc499da2fd27016fc4a76fca24746d9fb036d3 + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 92a7e230ad7aec2ef7ebd968f4b66855e789b1cddb155783cc626dabd4a1956d + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 50361a2982a505da1553321c4bf5ef892c86047553a5709af75be5ba68acd891 + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 802d3cccb9510a00fff36ea14b439c1585e5a7b548f5492d159f032966dc0594 stream 09 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 6ff2d54cfbad7b12599d13898bcab4184ff9708ff7d135e6c86b86c51a6998bf - Match(End): cases 1 accepted 1 rejected 0 rejection None digest edceddeeeb01fa91218b35507effc6b6eab4ae1fa20d641d07724144ed9613b5 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 17edbc4a9820227254a08fbad0a5d88cc4aed289087687c1abea050b9aec6aa4 - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest f730acb629eef6f410a2464a59b555340285b5a9f4fa7c3c726353ff38c8e538 - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 7bfb131a76928337a39d00599979ddacea1cc88fbea1f251eabc7ce2130a16d1 - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 137950e7ae9ab307972b518682a1a6529819980301e0c6b61869477829624ce4 - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 138fd3f03e645cf6dbce8b55d67dfa3bee2d319611ccbd87fc5b323747ea818b - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest b19854ff51cedbf8a4e761891538cdd8a2466a8111f5a321f530c1c23c685b5b - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest b1b6b3a0adeaf7bdf761e9a23a79f238199e020dd452c21e9f5aab6a1da5d82b - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 27cf3d734488f83f038cb377078fe6cc24925393b66c6c31fe2960a6107a7415 + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest aa3ae45ca390d05aea43866b30f3c9ca3455b1f1b05a99c2547d06ae15874e3a + Match(End): cases 1 accepted 1 rejected 0 rejection None digest 8f85614bfdf9ce7fb9d42500f8c89ae650d80ceb9400e5c5931530be1035ce30 + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 17b32292a1292f52f825bb6dff8f66e938a85f8b123e1e2e423b16d8bea86094 + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest b00698287e44e9fbcbba00c223a58bdacaabaf4b11def029b4decefab932889a + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 09a6d4d83dcae40376a255a845c3d47468f095e621c50122512c1d7bdb8a1271 + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 99b3695a660ddec193b6ea2a212e44f3551115cc49d10f2dce461aed5bd92a02 + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 85bd41349fcd3affdd14f6d8f854e7c8824ddc7cb1d9eac780a4050e53d38543 + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest edb4ebfa31892e0874d8798829a6f5059853820d9988620268fbe41658f01627 + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest ff93c31134942756e47063b116c8f2081f1f6605c90214f6f46b7798774ba604 + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest ca69134ef23f6197177d7e761738644763e386ae540238e7ac2a6adc2b295e63 stream 10 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest ca44c3bce73a5610a872a1d0303b56a23cf24ca37777c13e6a24a87d1cfc24c7 - Match(End): cases 1 accepted 1 rejected 0 rejection None digest b54ad413a5a73d9c58159d8ba395a38e0f10aa9a228850241c4b0a86b3f7c060 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest d74014c3b3088e17e6d053c05d819d7cc2167122a4ae5f892635440d69384884 - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest e74e1793309ec1319e8638cf40ea30ecda33dac3ac34e039080e2c6b0fd9f4cc - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 1a0630fb34fbcf3cd72eb700271d55cb72d1396cf043d7594b4d011e06de3d65 - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 16d0104cd8cc53d64ee80959c406a0b544dcf97efc2a8d4883b8372bdb8107dc - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 09a2b373f6a05b7868f0b62badd9a7b70485c3887d07f3704e17287a495bdd0e - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest cf53b4f42b6df5f38a07ecb6dbae883816b89d0776d27fd781f8fa08105c0409 - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 63c674a610aa789bcab890eb7b995a513ea52f2cb4d35a3e773836850070c932 - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 299a727b9d68956daf0449e6984cb16c4b2164d84eb1458c06226902204b6609 + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 104bdf253fdd338b6dca9ab6c5e85eff63b2d89dcab038fa3b51037b50241dd8 + Match(End): cases 1 accepted 1 rejected 0 rejection None digest 1252adbe0adcb6d80d6c49e5f42328cffca047b1051f0db4156b495a7ab09c64 + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 159d7594519afacb1983e0e1342f0c1d9a1690df506f242f10f325ab913ae6a5 + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 737ac4e0e149b3e6035639cd463ee58b93665b608ded034b0bc6fce2fad7b22c + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest ca92e8a0028ceb29d923afb1756e1b3a2cd2d6612a1b94275505343fde2c9ba6 + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 38e0024a61e3ae9e82824f8cb39f7306170bf83b35ee25735bceb74c3a370f1c + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest d7f15e05b6b502dfc48e1b450586c1a59f45cc752db762871a0f25b60e80aff4 + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 1c86ba62acded2176304804fade5d64040d0bb166865de53ee1be8c782dd5ba3 + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 1980296bd259e73290087b9a647d9fb23fdd89126b703e5a0b787e409172c979 + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 9e61f33b566ce703dafbe5bb907defdd2b953e4fec670e1953541f43a70f4f08 stream 11 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest b686ece8f076664ffa0165733e0500ee836edf985acf0b484edbdfb613e6563b - Match(End): cases 1 accepted 1 rejected 0 rejection None digest 4b32a58c8dcba7682ce0b04152d5184e48f255bcbdde08d728ab1dc6ef5aa719 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest dd82e4cf4daf7c19991aa4c7bd1bb24e55eb7b73cfdaff3444b3b1bbf73a2694 - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 851a1229d333e606ec2aeb7fffe0f817662f5f010010af9b62ae437d2e60f4d2 - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest c930467f2aaeee953347042808ebb417d92cd89f56335f085d0e468f33d61ffa - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 20b37b9884de14193cca46bda29983412944b68c277fd6930dd5f60df04f782d - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 5611b1ec26af613d14ebb77b8a17f0efc88b9c5da0c29776e9692aa12ee2af76 - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest e504ec6c7b648cd9e1a489be7954ed0208ff6cfb264e366ba0a300da9c1820f7 - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest d8157eee870ca226593b505a153b951f657311ca5471860ee29e67aa548dd58b - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest a1351b92d9e69aaaee98b9ecf17a33f6e3a9e0270b149290f2d45cba7c6d6149 + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 672eae71bdd18bb1ed4805dc282787ce7a167b895f0134c9df4e29dad42d0e0e + Match(End): cases 1 accepted 1 rejected 0 rejection None digest f70ab3dfb26e0ff7b11a6fa548081c047fb8c4c7dbdecd07b2eb3997483883be + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 99838e8b827f8033b1aedcbd3350fb3bded51219357fbb6c574a8c39684034fe + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 77127117955c46c913e999393886d24787a4870cb61fe3055fb8f4f2713e40c1 + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 546ccc9d71ca5a4561828595de3f8454fe91877e56eb132f34f421b760ab8961 + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 411c31511a1c718afe1f86682afff7fb59ccd88b1bfdd01ef8fcfd0bc284694e + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 1d28a0b5f2e76df4b27e36706fd7077a5c7d1ff0ea500788e0cfde7051dfebf3 + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 67642881fc7ef3be06590df5430b90511638f1aac15aa74c924e5b92ed80d7a9 + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest a04832f9a32bb370e59ef7bcf4ab2e97b0b621b06af0c84b8f68f37358aad309 + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 609d5452aa7e0a88cb9b8a5534681c062b92167792e1fb4521937b6ce3f3e7be stream 12 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 8d2fb9fe08d4398ba2bdf41a1bfbd61f8a623a94a0997cc110425e2a32089fd4 - Match(End): cases 1 accepted 1 rejected 0 rejection None digest cadfaa6a713d548a63d259d1aeda96768bff2eae82fd1d399370bffc03137613 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest cecec5d2cb55bfcccd6f0d48be9dc21064ff5e0f63d7649e7f26df156d536f2f - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest a31da823b3c1bb429765cc210a258f220aaa8c92df9673eb382fe12821b111ac - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 254975c65f08456de3e7be51da5139736a55a1b5e7e9a6defbaa7a6dd95a0523 - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest c54fb8b09ef9fea57ae284009d05a41b8f993e06114b4ff918d8565b2bb43d3f - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 0d05f27f04bd9868452c564d84baea9c6ce14927eb4ce469ab9a6246fa26d926 - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 638ae22a8f314323aa7353af461430680e1fafa580f11ac1f73f21fc1e8d9d95 - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 5a3f1429c3d154082905a1e51ffd43a59cc80a48fb38e85abd61598fdca8dedc - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 0e99767e69250ac6fcbfb7b4d9c993c83ecf9077dc80bd989d10d8e16687c1a7 + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 6c298f19bca2e8078524cedae611cd2673c62ee97ad06e62243d4db52358f7cb + Match(End): cases 1 accepted 1 rejected 0 rejection None digest 65803d6519e8009a73c56b339a9f7ba46f1285b11c09f5ab4450354c39bf8490 + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest b92a07935202be27bf25b5c9bd51735c2060ebbd6c67c61fc119e8f8acaaf986 + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 8d1ef8402eb7a9aae43d4334f297341e8de2efa2662a53c309fac0053d04b365 + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 4b506718dbf3dbdc49cd1cc003390ae1363ec0d71a7825d979045f44068e9ca5 + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest e6aaff4e35949e4a7ff0a967d344799619b6e170295950c1bda9b267748ade96 + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 7f648e2f9a661f83059182d36099f8c071675e727ba7b9b8d7d0336188695173 + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 229a363b7bde06285d9df455c6eb65af90935aeb893613a98c6f2e485f57c3ae + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest bd9cbd15f7105c97ed69e1c3325d12268e1bc85725d3f944ec6de84fa8270aad + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 32af3bd5bb0b69e0ce179a770269a7418177fa563e8edc44df1c01f7cb2204dd stream 13 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 059088ab453f9a4ea6c1bda93d62172ec77f9e2e27dbf1aa355ebce4064ecfed - Match(End): cases 1 accepted 1 rejected 0 rejection None digest 87a00b397cf046dc00eb333155428d922229a4dbe8c97422902374c399cf0a40 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest f9ac3b457d07b5344335572c99e15dfd8b4ce1dbf74d20cdf45ff9d132f8ec62 - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 4fd9741405c8cc1354022519f755d08d511ea2455235de82b292d0bac0f2da9d - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 3c5d1011b48719d8d6b69ecbf983be0b3fb6757f0c4eea62740babdf03888f0a - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest e5abecd847278d559eb1e90add494e476aafb8b8a7b6da3adc46314c003c3d91 - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest bed90570bb0963ab5526ce6e55abda3be957893a5177c318819af3a7f46e29b0 - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 7d73dd645bbf5331ce6e68dcdde05bf0dd9057592b8d5d02c651e8855abbdd6b - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 785f77fdc47845b843501b702d0260af3ba977a343a8763a874321e7769160f7 - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest b1b179df568a7c34c8d7bf87203c161ef88e1ab38ca0d75e6557fd3527dc11a7 + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest fec35e1cf5737ab3dcd034e66551e32b381cb672291b64f6b0140d616190ce83 + Match(End): cases 1 accepted 1 rejected 0 rejection None digest 4a1861f34feb55ddb21f549739bd5666dd8fc27552f3a3173baaab2eb34df19d + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 7190a38fff28dca89d3aadb7d4f07ab67349fd3448877b469ef32bcf9af7c8ed + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 1406117ea9fa975564ae00ee94b0c99cc479fd72616f417d1b5901b525529cd4 + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 215bf8bb454b8bb072d8e980f85570b12f49a7e3bf889bee8773b54b1aff5184 + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest eca762246a893da6ade59d27aafc0da76e88cd7d83586ca687a0ee6fc689dcef + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest b2a369f93b2e2f1aa801ecedbf76c018cd5e9e54dffdfa368618b7e23ad6099a + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest e4d7c0752bde967a6a6c229128f2473713cac168302411d2068692ed867a4f8b + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 8febe5d88db3eb980d967075e77a4eacd709fadf23104dce8fbdaa4e771f9094 + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 68b3b4adc4d89d691374d0296f39d9667fd4b6206f1a203e20d19a37fa73e93d stream 14 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 412f432fbfc37aa305017091196a1cf08cdcfb58286c5d67a5879a7adfe41a98 - Match(End): cases 1 accepted 1 rejected 0 rejection None digest b926b2fa40df45f59a10eb48153efaba5c5aa388262e69137f571acab8678ad5 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 2d87757635342d0fd27aff56df18325535b7886318a1508f3dadd6a540a30e9f - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 5d537842e1e44c2fb2663485c69818515a5d724e84ab275eeeda1534a66b326f - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest b0adb97cf6e8fd75a3d6eca5d1ba6a81b4e9cfbce99793d9374bf35e6a53f756 - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 1a598804994f12a0bf7db52814ab606b664bb716c6077797d25ca28777067d00 - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest b8637f87ab12074b33dd6ef0cf82da1a1e576167190c225c72f9578963208c00 - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 28a6bede5bd63c5a42a5f30e4f7dbcc3628634af2962a8e8839ae4b9c29f604d - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 2f182802ac4c754db7b1e709ec1e2c62fed5a070e2c068faba47bb7964d3f4e2 - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest f5c41bf5e61ad5831f4929f863cfa6bfe71fa3b6ee9b27a21e7c0a7595e357b7 + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 16131963006b632de280a07deb46ea192c15d2dc8e25ddf274461261ae0ebca3 + Match(End): cases 1 accepted 1 rejected 0 rejection None digest 21a867cfc349c464b9c79771ff5ce115a7fb6c77c69eab4333c654d1617ec112 + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest cf8f33059852e53a2e2e01d7106399ece848983621f1dc666fb7d0c7ea6561e9 + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest f9e69834a2d2b1a86a0bc36b29a8cdd8d6af1c4d59e93c2fab12517ceb7ceb6f + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest b160156964892a2f2f7a872f1c2702fc6a8ed893ea3eade80f4a74931f4043f1 + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 22185f43a8a2836d412f0a7498ecc7dd9194720e2056c8b59bf95334de525cc9 + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 3c85c62b9b1dab4c45bb9c9af63ca1de990e12d5402e2341e92d22703ed27095 + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 67070ac2787a5aec30323b7b1fd7e45c7d018878de75503d80e5e42ffb7f52a4 + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest d95cc1a17f764327b1a5269a0ade95b190f22f0d68b801023394934525457971 + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 71e233055c53dfe2ac77e4d9cf92dee19af492e82c3e32d7eb93dbd28d4eac8c stream 15 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 857b425150218962c0a7548ad7867d0223631dbca974fdcc84cb94019355c7ad - Match(End): cases 1 accepted 1 rejected 0 rejection None digest 95287e0900ba61bf3efa683215b6b74a25af987141de2f338f74b9e9089c9694 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 98956d902b7f9ded95fe6659d503239a457ff8e0f9557ca5e48c5f3f73cafeb9 - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 627c3e7aff204f684040fcf3fd123b5c9cfb945c9aae4167b12eab4723b953f0 - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest b8bef3ff0962d6a257250d4bb4876d9f1d9aab401cd818b3e9a2c9475e1ec67e - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 97fec0191ef2a75b3d6440739172679e85e8c853f502199048fa9c34fe479ad9 - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest e2a690dbe15e9f6276788dee265fa948b186a5904698d55aa72c1ebf49bd7b9f - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest b10ff7406dbceaf57f9edfd790369eda2149677347a3bc5a15226dbe0928d0ca - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 033a9d6c6cb059c17270280ac6121bc6cff59ff0b452c49cb65dce6f55e992a7 - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest d4fc5053bcf8daa493d15dfb6750582b5efc3690d8eb3260ed4645e012789c4b + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest baa489d341a4a2700fda70bf6886f5b9dab805820201010b108c47ee303eba41 + Match(End): cases 1 accepted 1 rejected 0 rejection None digest e01b98e21bd5c63e254e0bffd894935132c4d19139611e842ece4c4b57365538 + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 07248d0e233debd4957a759f5a206c0e19208e530dd0ae0b289433e0e4efdf89 + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 350c9e7214b4d19dda13cc2a40f5b0cdcba493056962d9973e2197039614545c + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 15a6a6f14af85c5f40b3395c445330d1cc20676dad8257390e5e5e92a500b094 + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 1ea6e03cf037327f78a691daa80c95514ed71fa4475d0bb7e840af5f13cabf87 + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest d1a34d662eeaf09b1094de8f6ffc636311b783883ff9218288ea81f66e685650 + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest c90d18111ee0768484b386281a560fad062ba6f8f11871256f4fc3dfb2830e76 + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest c80c79a90c901d855e127dc81690b9357fd87a290cf7a856d4618c1a54cfe7fb + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest a76debc09aff00b2f5343ea22224544c69ec8793774afc18a19783fdeb5cdf64 stream 16 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest dd140b0d516a82a58732fd1284bd4f26da3912938fd1296195717826ca5ce6fb - Match(End): cases 1 accepted 1 rejected 0 rejection None digest 79b37443fe6cbbb9f1e685ee0232170bd3cc3fcca8edbf104d9c304bdff15276 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 380f648b6b5a1cc408b08c18ec16e74342a0c565aeb4455485c6572a68178fdd - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest ba0d629849bd7c219c3ecb424e75a17049097500732e7887fd7b951eaaac4fbe + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 0f38e4a6ee59a0d81330375a4753651b84d4bde276f933e9c62b7bacc21d5f81 + Match(End): cases 1 accepted 1 rejected 0 rejection None digest 214a0977eb69bc66da2e0e8b2271fddfc90dc522672e4b874c7ae782a5520286 + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 22556a47a31956eb62f084419d74738ef09f8bdec3487bf53086ed20d06fca99 + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest cbb70092f731bbb2184d4a3f34a809f1450bc9df0a9a6956d25ea4c71d350358 Query(Continue): cases 32896 accepted 0 rejected 32896 rejection Some(LeafParentReplies) digest d678b609a984aec0e82f6704ec78d2c1312a319c00aef09315cb6d5a97afd3b3 Query(End): cases 32896 accepted 0 rejected 32896 rejection Some(LeafParentReplies) digest 20206e46e9569954fa7409adc1466295dd78517ee9b1f7a6dcb931af522b09f2 - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 9f508aa9bb69f2b3742ce4cd27f52fa10dc7526ce8a0f529d4ab3e922ca0c6c2 - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 29c688db20ab6a6a85b8a64ea17a0f0c9e3380c16c02a687588891c443cd6664 - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 7a250d61fe85b397ff21c6c0d2883c934615f0c59d3f7fa92090f2bbc974cb74 - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 69e0a19208d6edd30e31916235b05fcd9f3072a9f93668a49e7cd2d971621588 + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 1960aabcbf735cde670759145a15dfe46bd9f6593c32031ad4254a60ff292be0 + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest c7350f8f4be155503abf5f41d5d6ebb144b3c4068d4d41c66fb2d8aefb31c346 + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 7257d21c0f045fbdaae74ddd1f9c1ac52ac58d85f6b2efc24a92e2c2c7b48d61 + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest c19c0586156caa168dd0e5703a80d90674d0554d1712c535f49d55172049b0b4 Responder stream 00 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 964e37b3c509df37c0b8877fca3051bae3efbe83abc4924c862811e703cc4623 - Match(End): cases 1 accepted 1 rejected 0 rejection None digest e10f9ad1721b9db582acfaf071974eb6d3c339c7d83fd66f287e195c3e59e00b - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 8e41452eb051511766f0adbff2c4a2e082df0e9fb452497e549fef8928461e2e - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest b90c5effa96b16776cbf6e9949635b52682c2c65c7f797ff18573310cd77339c - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 7c974c0efd6be446db31c057cb8a5c8ab3be81b87a9c0dbeb9c9465088e2a743 - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest c1aa9df70be5616b950459c84c1da97879b7228dd5221b651b098cc4728d9ad8 - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 59b5fd7fcfce8862da5a311e322502cc8fdcd5b1ead61bd8e6b6e532bf774ede - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest e84ed624d4c85c71009ba4d06f28a9b3c251d88711e3ee37f9c5877b511222ad - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest e6c8657a09bbd68e3a4410183521d0dc564776c29489da7eef314c391632057b - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest c6137a2a9996157c93f59583278135d535fbd3d5439da4de7af076a1a699dc2d + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 979a4115ce0daaaf110b6885121b967b0a2d36b45807762760e5f62efd4a2a5a + Match(End): cases 1 accepted 1 rejected 0 rejection None digest 1afb54dcbc2b3eebc3055161cb60e8fecda1ca917acf4b688aae43582e9fb058 + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 2703d5de54045e1ca616761daea6e11f9b653d763ddfb8b8553c91fb0801c0d5 + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest f5d3416b14d3741a3267b55d576108c33cbd64e7423f6ff3b1cae726f1eeaffd + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 4705ca15cb20079b34532900513f8f19310054e1ad65af8844c31c81592e2c8c + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest acd6ab89a95a54cbc3a3cbdf91a950eab963eca3eb27bc02d1792fc3b5c09ea7 + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest ef42a1f4f55d67a16c4425c0b892fc4d8ee7ef28b2745d077cd58f0bb5814980 + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest e4409433cc158bfb054b2001e551041927ed7c45e10012eb5d3309c841a78ba9 + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 8f78ad754b008c62ef8f7e0346ece60c634bf8cc48e00d8d8ef118aa8faf4f5a + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 122aa7dc19fb16cfede3a7b7b70eff5651b16ae1f2aba2acd02b4820bfd67d95 stream 01 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 9bbd1097cc86719184aa34340aa3f2c06a63dafb37b86914d45f36563186235d - Match(End): cases 1 accepted 1 rejected 0 rejection None digest 45ee114c5b1f0ed4b48840ede520d9c80af340fd119de086f63b8a976a1ec742 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 0bb26c70fe8f5bf17d6645b50caba9dbea86edef3b92e14f585637a64ed0f548 - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 066871f0d0267b1b42edbb3835aed267fd755c6f7a819c1420def0671f62628d - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest af9a3ba555a3bcdfa2108b28f6539d8620aeb2eda089bfb7896820f98ba0a827 - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest f29eaef5dc2c87a94692e1687365ee51a341ad822204604da619148f4f742247 - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 319b28c9c064036c6f2b1e30f665b32d6061b2f03a826d934a56845924030837 - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 4dfd3a14bb679a3f3c72a3bac874d2ecb167299fcf03f1543fcdd49d76144db8 - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 7efb04b3cd8ae0a43ffaaf027a7d1ae31f033da458ccf85ed6339227bdfce8c2 - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest d210087cbc67d20f09c0fda1159b051ad38c629fe9783dfaf4e0cbbe985ecdcc + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 822e221183582cc39ec93269c2f05600f45f6257060a6d7ae3fa807ffdab3bfe + Match(End): cases 1 accepted 1 rejected 0 rejection None digest c7b891e63ff7694bf97ddae67d2d58f763785ab525dc164923540e714a136b94 + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest b514e4add70536373f31eb2ee2be3e2955c5c830f6e9c2d88368303d08c50c2f + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest dcd78220f73334462fc05c0092732081ce054f8390d6deae9478413ebc51c4b0 + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 492c659c7e07c17f674abb88954a176fcf35038e0c8c2c1552a895a21806986b + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 154a3788c85d4a50eba9328b8351ad401a987d7d6bc2986f937f2afcd880285f + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest f812816df7b5bcc7158910f89f3493fed4e09e28a39fa7ae6e6986e1b6e877a6 + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 6bb66af11167bfe6b41ff77a4eba9a8dc8b77a6079c0818e84c0d99c1f9b5295 + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 3f7595c05aa461b3adfeb2720b3ea00bba3bf16236e5dc8dea498ef648d64d11 + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 5a6f8915e99b97a83b5a0b30ae413f0ebfa61fdfa177f656be1a95ebed9385c6 stream 02 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 8f3624873d4b068a073bda3cf2e280a5ca195bd9066d3b2e0b9c6156a33a5e6a - Match(End): cases 1 accepted 1 rejected 0 rejection None digest 1f286c89dd40d9dd0a323fc7a4aa9222aeeff0d82b0cdc1e337859c7fa57249b - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 7ceb016d4177164a41f5315046fb7a104e7b402b8544076656fba53cdc7c5b8a - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest b232233fad61fdfe489b32005295b43142c3da73df863366f532cf4e04dc0960 - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 6c6a82a4201c2095453ffcc81b644af49180d33c49a958cbbd221bd259a931a5 - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 1d2091676cd8510ae281d0959ba783251c31b696d764c878213a67cdddb2fe00 - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 97b2682d961d4e0fbb9ca8093663428d6c5852e37baa1905f652fde30e31b0b3 - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest fc1d59d32a9266c1a77486e6d76a9bee51d1bb4cd943d380038230f42b9dcf4b - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 5aaec826e8f1aa4bb1dc7f60095baf57a5b3cb9adcb0a4cf688407b0e7b7347e - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 2221d8b7741c597c23a781ae3a4315325660af324d10d667036b1c22c70fd4ff + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 8da023b966a9819fd558c2276909fc87966cced0c324a13db00489c2a1d4db1e + Match(End): cases 1 accepted 1 rejected 0 rejection None digest 3a166443f96064928189f46632d85d880cf414706aff03099c3e5fb7ef706047 + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 0eeb109cb452c2417fe4c9d280f3e435bbe8f7a0ade330d494415fdce47724a8 + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 71d47af427367e34a0420af2c849bb06c38db6ca107c725a2f525038485387a4 + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 00ed3d0aae4f3ec8d63e223e67834db5dfe001390e6fd2db5bf76ad288fb18fb + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 58bd9c9778eff881c835a53fb270cc781bae4331ca767e54b9ddb6c4a629176d + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 217373b0d58da1136bcce08bdc1ed633f939c84ea1d4fa5bc56d69406cb6b26d + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 97688245e619012a49a5043b1f4f45983a0ccf57481e071867d37dd9921ec24d + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest aa8428f399e0244514b082aa56c9fda2510488a978a320028b219050c8e4c7bf + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 8d5ad0ff596aebc75fd7a0435dac0d8e7f521278f3489faccbe2b5bdb7de7be1 stream 03 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 66fad94ed67e1ffe5a3c03b47e195ecfd7fe08e96e933146ab068802c62711de - Match(End): cases 1 accepted 1 rejected 0 rejection None digest c85c9376cd1443e95f4a955c78ce5bc0886a3886744b6c641db8640fa471e6fd - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest ac1261c496af02319a1772ff71e9813a3b4658b27bab4d1610fba06196b9ef51 - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 157b7b3c36e30debeb2364b3e9c29267d936ae5b68ac616aee5fc533a24b52fb - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest c51172601ee20ac3a04e5a32720d521af1af3a90d29b71d6337acbafbe5fc115 - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest f427d3759ade3b024bfc7f57189200fbe0e220ecc832df29864cfccb0283a740 - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest e3525955c5573149fbb36710cf3cca454db1eef8fc2d7dfbfbadd08096b6b14a - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 9f920581eb1636ed6e52b37f50109b03b6010e431e7173a0af29bfda972f1f64 - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 8dc4e0a82ef1dfa3953132cd6cbe54650221b73a98162927793d488162a38fac - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest edc61faa00b781b3d047bab5809faf0b1a74bdb20f202712d9e5dbbf805525b8 + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 18362a39b132ae13a3ef04803937260cd185f761b2147d2506202f206c05a611 + Match(End): cases 1 accepted 1 rejected 0 rejection None digest 2796a215960684c415193f031b414fafd5054a0876fc885d45b2becd4b2333ac + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 93d46dba804ced90724fe3100b67406bb81cb40b17aa5a8e846de9f9723b6a76 + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest a372666b42bde5f3679a47b32d5846a9aa427dd3d3a3b11571c8c5c9bb74159d + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 181528afe22b0e7986dfeb8ff683e1b0c039b03804fb5da51e4a0d0128dde6db + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest b428487914e610ac8d2385e50a29bfe26a08a12bb71c812fedbffd824610103b + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest e3d0d3c889b924fc0d81f7c8cca6ee888baf3d0a9578c79eeedc46938a028d76 + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest a2d2469022d06234ced10ab8ee31795662aed518588c7af109a8f4e41764e968 + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest b9031379ac21c36c75ce35461c068591fa0acfc37dc59e0bdb5c198a51cc2436 + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 624110158fafc1bc489ba666d6f3cfcb1c42e88948baf16de09a942382a9ed00 stream 04 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 0aabcb6009a818cb51cd563f227fc5100ad87c2cc07e8f95ee952d3fd840c3eb - Match(End): cases 1 accepted 1 rejected 0 rejection None digest 43511d45dc52dd6d8f66c218e6bcb8272bdbd57a53a228670dc8e999cb99b8e9 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest b435ac27a6a88d353d269371500517496cf8f3cc386895cc8ac3059abd1151f5 - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest bafaa10bfebbfd58e6dd9555c38d68f074a88cc7714d5cda3a8d4932990cd87f - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest aa3e0335828851588f14358567427c9634d7463286a2264198060a3d9c188e11 - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest dfc58272001597c14dfa88433653257a8a3b12091c74fc18f40c6fe3c74a588d - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 9d263dbb054de831ed5346a3301e63c7e338b02a834d0db1bcd11d1eb77105ad - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 4d2250bf01b9cc3ef793712d67669ccc61ef5c42984841d578bfd2a848b5331b - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 964e4c82824016b610d97562de5eb0d355b4d17cf16acb220581550c2cb08201 - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 09f3d01b5a48fee382807fb80ccfae7e9a2aa167b28ed706e8cd6e536752e14f + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 163d3634fc1418cc68a6632a5bf272b4e4d13752e0ebd8add7928e88d8fe3c84 + Match(End): cases 1 accepted 1 rejected 0 rejection None digest 116b0f61934959f642b115f3a0695318b8097c7e1cfff1ace24c8a94f32fab0a + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest b418d391ea72a381e693c8c7501adaf436976974b307f000547de7f0a4eb056f + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 7b428726940fb374efc9da2c7f1ec5eb99a1eef0d4873b886e9e02e5718b0c24 + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 4bdecfce7e2f1db9dc54b3f203871bfc44be54a7b9a73523b5026e4554377a65 + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest b17a10c5fab248a0a35dc69aee60d3941af501d2b73377be8c9b4212777d917f + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 112ba83fa34002a0d82c9e67586f92a1ee7ed2c801738c666d212549609399cf + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest fbe0e0af3533eb75f617af3aa735816a7a88a3b4043f365dcc771605a1829282 + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 0745e6e4b9d144d37c7dfcb1fd8c3b20bbfb772ee9d7d9524896c836bb7a2c07 + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest c336003c19e3d38c544445fa47a491cc0c0bf38267c67a53205bd0cdefe69bff stream 05 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 775ed8cb20fd4d5405bb3d83b4c81a29738bc999c1341d2c4ec38a96e9c2a478 - Match(End): cases 1 accepted 1 rejected 0 rejection None digest 0d6f8784c353242b50acc993f6af93394b09564469f4976a8ecf3809a4b7c340 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 716e53ce1b990161233ac6562d3672e38cce0a91b108ebe915e2362216af4d79 - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 41fc3692f19ac323b3b2d7c8bd0307425676297f8e0bed6b221753c173461fd0 - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 75abb3accb666e6145d70c2d102fdd6ce5f7ee38cfb7d4a33696051837bedbfd - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest fe90890210e48d3fac30d24e2fb66ad2a6273e6d8b67cff578a161b3062680c0 - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest b528e65d2440e583aead617a94d194f56699e05e71235a9229f907dced0a9b79 - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest df3b29ef01094a59fa27126fbc554570a1b89e4be9cbd9f525279aefa812c774 - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest f42bf364254585a59063ca719e3e466fabdcf8bb4de407f05c44bf8c1b75b0f4 - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 37008b172b7470e27c3df9f915d753b7a1c501bdab556c600eddff42c51e5df1 + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 8b2bcb0aa72efc91a4308a50158458b40c34a5e6298420da3cc4436a1a7438c8 + Match(End): cases 1 accepted 1 rejected 0 rejection None digest fa929e4b8d272af567fcda22b7c09e8cca387a9b4e5eb6cf108d05224e568a54 + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest a8feda23c84986de29557cf75e55cca962a9f421a81bb2fd611d73f272b5e411 + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest e36e489f824ce1a98b5efc940e70fd3d736138760f2bba5f0b69127edbdf77b8 + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 57160278519ca5794d4b9d5d6cc5c4c83112973411c5eadbb4e306de0a578925 + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 1312217b358ae3a9a3ecc386ab11f025d074112cf2bce302422ce08836689d75 + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 4091eab1ac41b4c8a07bcb9daa72e9d0dbfb679978902f531b3ffcde71ee56b9 + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 4798a3f3ed5923953ed9c4bde3eb03ad5cdb6302c1259783695acab7afb17e79 + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 674d77ceb6d82027d98029a77aca2fc7b864800d20c83bffb0e1ffc67c303f6a + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest d2de9dc2f96e7d6b3d5fc7946b7038862714c55cdf872e100e18c7687a460b50 stream 06 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 19dc8b6508fcb25bd215ff98745b10a0226d0b7802317ff23693170f6b69e2bf - Match(End): cases 1 accepted 1 rejected 0 rejection None digest b3f847c2acab7165aed0ab12dbf8095ddedff4da4f27b75719e598bd7a1bc9d5 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 05319d1eba637b4941839be91b827a4e17bbb9ea997a9de62021e8c51d44f28d - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest e11362eed2d9395eae0b5e66d4adf8e7b26d5660c549658c61c500fc61921a97 - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 14a04b8700e66157eb28db77fc19075aec4fb93ac3b4cf7ac8f4a7dc6d195548 - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 9095d15c02ae8055927f39ada986d42a974f1621d7a97a48f22619dab7f51eb3 - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest a2d7ee70ef3c9562a15d7077586fd0d24b4d8f81db5a4fb9869782458de72fb2 - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest a4561e3b2a42f881801ad9ab5d79618254f5157ec1a50043c81d7a53f3245f72 - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 29f62ebb0b44a04c6c29c315771602bf97d0e948fdfc9e889a3ec27572876e1f - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 127c84b8f7a667189b8accc3ead7f9d759231b6d5e9f4d521bf8a27acffd3e20 + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest e4f7848df6ca1afcccab12651d9356f43ace95ed972a996669aec12de64eb8e2 + Match(End): cases 1 accepted 1 rejected 0 rejection None digest a86d5f8b56f9221c2d5f944ee03ae4f5d1cf058ba32b2db7353cc01b2510fe4d + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest a51b552a13d95e1c413876140bc217d5dfb95132fdc83134ff19be18d1329999 + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest ec94b9af7046d9cf87c300043e51fbdaeb46af865e97c0fab3cb09dea22f6cde + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 275db08be66aa5ef32a6747f2ec7bdcb9ef867f8f726ecb3e852dbe2cd9598cc + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 9697014f4cee53799c3890119160281f811c65287adb31221289bf3f184259ce + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 784f02431c5bdb551b5c2185f787bc5f6832212602ef363fd48910a3dcda312f + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 16c13cdfbb93ca755d7304932c5035a1575e7c82c67acc6cfdc377009109daaf + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 3a313910a93bb9c6fce179115ac038716b01ba7e924f61ea4032287c26da7c25 + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest fc52adcaab5665aac8ffc0c4b20decec58547c5cf358cf009917f3dd21c445f5 stream 07 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest dafa9fae71d3877a3a621fcd8bf9f75723bab8f02d0dab6a6bb5ea6d608c2786 - Match(End): cases 1 accepted 1 rejected 0 rejection None digest 75d5f73f826776473c9d2a2560c371f3e7d0092b123648b34240d4f82c4acc36 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 074aedf4174f5bc56a17f56d49dda4403c9bfdf0ab9f7bff581fd3757af72129 - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 46acb5c7c4557b55a31329576477e76c7a9f9a41fb45b8c5907ed7b9ec11a433 - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 66a2f0ee0a7799997880a907305e992f4ac33bf602d96761235310a2149c9bc8 - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 5d04b5006b60a70a1413b657e481c3ed6d0a75da18ecc410b72d19872a2b17eb - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 5e300b4cb071daa8574f20507a7da4472111351f1e740723f9b5bf8097b8e41b - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 0a562f5dd6f838c161f4f48188a8b8783acedd8f3938c965213ef59fb0e6d6e3 - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 0abc151c61d1bf0abdbf8de84f9a373da4dd54f63d5ddde0d3963e68e7a1bf8e - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest ef47eedfe6e578be84ed170e1f5cfb0978893dda7ff8ea95c5e56c76947e5064 + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest c09a0b577898409cda79efcfb32ef0892830769ed6ca0cf360350e8bc2f53704 + Match(End): cases 1 accepted 1 rejected 0 rejection None digest dfd0ae5b28c32c8e65b61656edc45398602de1a83762a2094e07e2aa4c3314bd + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 506b9458de2178528097b78795cb9ccf8099c865801df55cb8be36a28dfdb419 + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 472449d11e02b2cd60077313c13c45acea914d48ce9dcb247f09922494b86bc4 + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 98e3af9d722bedd4b6b2e79a380033b1133fe0653450d2c4cc6734c92f8d0e6d + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest ba112066ab293fb2cfda6a508bbb5962ad7e3de98a38f4e0bfb9c466dd00f6fd + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 4c0fac1a9e9adcb0a6e0702b55c704405e5dbccd11e44d28358c508439272a41 + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest b8a1477fb43bf17a7af0dd27866f013120ef5c4062811ac507d23886e399f945 + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest b9848cca526ae9ad112553ccbf357526d94e7eb024067cd758d3d74d88d25282 + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest a86c8046f0029b1ad34120d3f84167bbde755f794d9955f6af0b29a1aa6dfef7 stream 08 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest bd306ad277db2515cc49b306b36b9c4c499e6a192339b74c44fd61f598215e78 - Match(End): cases 1 accepted 1 rejected 0 rejection None digest 780efde07897d67399fa95cb91d9e08698bb571eb13c07f1dc7c808bea43ded7 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 0e3fbd1149bcf46ed868882fb0fff74fcdd959d671835f6909e71b1ed95845eb - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 17f68790f5b708d77429611d19cade3367733a261a50c4b41037fd88026cbaa3 - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 5851952e464d7e55c7f95efb554bc5613a5bdce4ccec542581674532a90ec418 - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest e286e082de98f67d8a5e12fa2ce5d46de10d524b70764cd9de29d59be3b54c2c - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest f0b52c1605a67276d58f1baeb1abb4b1ec603d3f5d037ce404b8052298b13828 - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 130042645e4d5f9eac83eaa1271585f1147b8dcce58bffd94aecc4cac6e96b96 - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 8bb8f6555c5bdf08019175230fe7e440d3ab688fa49909e75da4ac8c3e091809 - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest f96f9bf58b82edb0d80376acd21f4f8b5458aeed6f63c82f79da08f83eb73601 + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 76a538374c611ae0f915298a269e642bc343015d2c2200076d8fcbd61479dc99 + Match(End): cases 1 accepted 1 rejected 0 rejection None digest a04abe29794ef9764b323839ecf5f7b82025da1b1b981cdadd59096cb619e5fc + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest f2d8b2e7629631594e1e896a108847a2bb8f30b61dfb36c965db44cb089c7fcb + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest a1e8eab09925fabebaa69dae25c80fad8878c464666782d3d06997659ed01b20 + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest e78c3ea97c25284e115d9877aff71a71b456788785542e0e0999d8ffa5476340 + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 879dd07d45e47be9a3e5165a351810fa677600872c012104c228229d913d6a8f + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest b46dc4e98f83be2b7be8186e6acc499da2fd27016fc4a76fca24746d9fb036d3 + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 92a7e230ad7aec2ef7ebd968f4b66855e789b1cddb155783cc626dabd4a1956d + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 50361a2982a505da1553321c4bf5ef892c86047553a5709af75be5ba68acd891 + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 802d3cccb9510a00fff36ea14b439c1585e5a7b548f5492d159f032966dc0594 stream 09 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 6ff2d54cfbad7b12599d13898bcab4184ff9708ff7d135e6c86b86c51a6998bf - Match(End): cases 1 accepted 1 rejected 0 rejection None digest edceddeeeb01fa91218b35507effc6b6eab4ae1fa20d641d07724144ed9613b5 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 17edbc4a9820227254a08fbad0a5d88cc4aed289087687c1abea050b9aec6aa4 - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest f730acb629eef6f410a2464a59b555340285b5a9f4fa7c3c726353ff38c8e538 - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 7bfb131a76928337a39d00599979ddacea1cc88fbea1f251eabc7ce2130a16d1 - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 137950e7ae9ab307972b518682a1a6529819980301e0c6b61869477829624ce4 - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 138fd3f03e645cf6dbce8b55d67dfa3bee2d319611ccbd87fc5b323747ea818b - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest b19854ff51cedbf8a4e761891538cdd8a2466a8111f5a321f530c1c23c685b5b - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest b1b6b3a0adeaf7bdf761e9a23a79f238199e020dd452c21e9f5aab6a1da5d82b - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 27cf3d734488f83f038cb377078fe6cc24925393b66c6c31fe2960a6107a7415 + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest aa3ae45ca390d05aea43866b30f3c9ca3455b1f1b05a99c2547d06ae15874e3a + Match(End): cases 1 accepted 1 rejected 0 rejection None digest 8f85614bfdf9ce7fb9d42500f8c89ae650d80ceb9400e5c5931530be1035ce30 + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 17b32292a1292f52f825bb6dff8f66e938a85f8b123e1e2e423b16d8bea86094 + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest b00698287e44e9fbcbba00c223a58bdacaabaf4b11def029b4decefab932889a + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 09a6d4d83dcae40376a255a845c3d47468f095e621c50122512c1d7bdb8a1271 + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 99b3695a660ddec193b6ea2a212e44f3551115cc49d10f2dce461aed5bd92a02 + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 85bd41349fcd3affdd14f6d8f854e7c8824ddc7cb1d9eac780a4050e53d38543 + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest edb4ebfa31892e0874d8798829a6f5059853820d9988620268fbe41658f01627 + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest ff93c31134942756e47063b116c8f2081f1f6605c90214f6f46b7798774ba604 + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest ca69134ef23f6197177d7e761738644763e386ae540238e7ac2a6adc2b295e63 stream 10 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest ca44c3bce73a5610a872a1d0303b56a23cf24ca37777c13e6a24a87d1cfc24c7 - Match(End): cases 1 accepted 1 rejected 0 rejection None digest b54ad413a5a73d9c58159d8ba395a38e0f10aa9a228850241c4b0a86b3f7c060 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest d74014c3b3088e17e6d053c05d819d7cc2167122a4ae5f892635440d69384884 - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest e74e1793309ec1319e8638cf40ea30ecda33dac3ac34e039080e2c6b0fd9f4cc - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 1a0630fb34fbcf3cd72eb700271d55cb72d1396cf043d7594b4d011e06de3d65 - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 16d0104cd8cc53d64ee80959c406a0b544dcf97efc2a8d4883b8372bdb8107dc - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 09a2b373f6a05b7868f0b62badd9a7b70485c3887d07f3704e17287a495bdd0e - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest cf53b4f42b6df5f38a07ecb6dbae883816b89d0776d27fd781f8fa08105c0409 - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 63c674a610aa789bcab890eb7b995a513ea52f2cb4d35a3e773836850070c932 - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 299a727b9d68956daf0449e6984cb16c4b2164d84eb1458c06226902204b6609 + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 104bdf253fdd338b6dca9ab6c5e85eff63b2d89dcab038fa3b51037b50241dd8 + Match(End): cases 1 accepted 1 rejected 0 rejection None digest 1252adbe0adcb6d80d6c49e5f42328cffca047b1051f0db4156b495a7ab09c64 + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 159d7594519afacb1983e0e1342f0c1d9a1690df506f242f10f325ab913ae6a5 + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 737ac4e0e149b3e6035639cd463ee58b93665b608ded034b0bc6fce2fad7b22c + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest ca92e8a0028ceb29d923afb1756e1b3a2cd2d6612a1b94275505343fde2c9ba6 + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 38e0024a61e3ae9e82824f8cb39f7306170bf83b35ee25735bceb74c3a370f1c + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest d7f15e05b6b502dfc48e1b450586c1a59f45cc752db762871a0f25b60e80aff4 + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 1c86ba62acded2176304804fade5d64040d0bb166865de53ee1be8c782dd5ba3 + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 1980296bd259e73290087b9a647d9fb23fdd89126b703e5a0b787e409172c979 + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 9e61f33b566ce703dafbe5bb907defdd2b953e4fec670e1953541f43a70f4f08 stream 11 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest b686ece8f076664ffa0165733e0500ee836edf985acf0b484edbdfb613e6563b - Match(End): cases 1 accepted 1 rejected 0 rejection None digest 4b32a58c8dcba7682ce0b04152d5184e48f255bcbdde08d728ab1dc6ef5aa719 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest dd82e4cf4daf7c19991aa4c7bd1bb24e55eb7b73cfdaff3444b3b1bbf73a2694 - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 851a1229d333e606ec2aeb7fffe0f817662f5f010010af9b62ae437d2e60f4d2 - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest c930467f2aaeee953347042808ebb417d92cd89f56335f085d0e468f33d61ffa - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 20b37b9884de14193cca46bda29983412944b68c277fd6930dd5f60df04f782d - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 5611b1ec26af613d14ebb77b8a17f0efc88b9c5da0c29776e9692aa12ee2af76 - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest e504ec6c7b648cd9e1a489be7954ed0208ff6cfb264e366ba0a300da9c1820f7 - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest d8157eee870ca226593b505a153b951f657311ca5471860ee29e67aa548dd58b - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest a1351b92d9e69aaaee98b9ecf17a33f6e3a9e0270b149290f2d45cba7c6d6149 + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 672eae71bdd18bb1ed4805dc282787ce7a167b895f0134c9df4e29dad42d0e0e + Match(End): cases 1 accepted 1 rejected 0 rejection None digest f70ab3dfb26e0ff7b11a6fa548081c047fb8c4c7dbdecd07b2eb3997483883be + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 99838e8b827f8033b1aedcbd3350fb3bded51219357fbb6c574a8c39684034fe + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 77127117955c46c913e999393886d24787a4870cb61fe3055fb8f4f2713e40c1 + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 546ccc9d71ca5a4561828595de3f8454fe91877e56eb132f34f421b760ab8961 + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 411c31511a1c718afe1f86682afff7fb59ccd88b1bfdd01ef8fcfd0bc284694e + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 1d28a0b5f2e76df4b27e36706fd7077a5c7d1ff0ea500788e0cfde7051dfebf3 + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 67642881fc7ef3be06590df5430b90511638f1aac15aa74c924e5b92ed80d7a9 + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest a04832f9a32bb370e59ef7bcf4ab2e97b0b621b06af0c84b8f68f37358aad309 + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 609d5452aa7e0a88cb9b8a5534681c062b92167792e1fb4521937b6ce3f3e7be stream 12 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 8d2fb9fe08d4398ba2bdf41a1bfbd61f8a623a94a0997cc110425e2a32089fd4 - Match(End): cases 1 accepted 1 rejected 0 rejection None digest cadfaa6a713d548a63d259d1aeda96768bff2eae82fd1d399370bffc03137613 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest cecec5d2cb55bfcccd6f0d48be9dc21064ff5e0f63d7649e7f26df156d536f2f - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest a31da823b3c1bb429765cc210a258f220aaa8c92df9673eb382fe12821b111ac - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 254975c65f08456de3e7be51da5139736a55a1b5e7e9a6defbaa7a6dd95a0523 - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest c54fb8b09ef9fea57ae284009d05a41b8f993e06114b4ff918d8565b2bb43d3f - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 0d05f27f04bd9868452c564d84baea9c6ce14927eb4ce469ab9a6246fa26d926 - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 638ae22a8f314323aa7353af461430680e1fafa580f11ac1f73f21fc1e8d9d95 - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 5a3f1429c3d154082905a1e51ffd43a59cc80a48fb38e85abd61598fdca8dedc - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 0e99767e69250ac6fcbfb7b4d9c993c83ecf9077dc80bd989d10d8e16687c1a7 + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 6c298f19bca2e8078524cedae611cd2673c62ee97ad06e62243d4db52358f7cb + Match(End): cases 1 accepted 1 rejected 0 rejection None digest 65803d6519e8009a73c56b339a9f7ba46f1285b11c09f5ab4450354c39bf8490 + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest b92a07935202be27bf25b5c9bd51735c2060ebbd6c67c61fc119e8f8acaaf986 + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 8d1ef8402eb7a9aae43d4334f297341e8de2efa2662a53c309fac0053d04b365 + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 4b506718dbf3dbdc49cd1cc003390ae1363ec0d71a7825d979045f44068e9ca5 + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest e6aaff4e35949e4a7ff0a967d344799619b6e170295950c1bda9b267748ade96 + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 7f648e2f9a661f83059182d36099f8c071675e727ba7b9b8d7d0336188695173 + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 229a363b7bde06285d9df455c6eb65af90935aeb893613a98c6f2e485f57c3ae + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest bd9cbd15f7105c97ed69e1c3325d12268e1bc85725d3f944ec6de84fa8270aad + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 32af3bd5bb0b69e0ce179a770269a7418177fa563e8edc44df1c01f7cb2204dd stream 13 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 059088ab453f9a4ea6c1bda93d62172ec77f9e2e27dbf1aa355ebce4064ecfed - Match(End): cases 1 accepted 1 rejected 0 rejection None digest 87a00b397cf046dc00eb333155428d922229a4dbe8c97422902374c399cf0a40 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest f9ac3b457d07b5344335572c99e15dfd8b4ce1dbf74d20cdf45ff9d132f8ec62 - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 4fd9741405c8cc1354022519f755d08d511ea2455235de82b292d0bac0f2da9d - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 3c5d1011b48719d8d6b69ecbf983be0b3fb6757f0c4eea62740babdf03888f0a - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest e5abecd847278d559eb1e90add494e476aafb8b8a7b6da3adc46314c003c3d91 - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest bed90570bb0963ab5526ce6e55abda3be957893a5177c318819af3a7f46e29b0 - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 7d73dd645bbf5331ce6e68dcdde05bf0dd9057592b8d5d02c651e8855abbdd6b - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 785f77fdc47845b843501b702d0260af3ba977a343a8763a874321e7769160f7 - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest b1b179df568a7c34c8d7bf87203c161ef88e1ab38ca0d75e6557fd3527dc11a7 + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest fec35e1cf5737ab3dcd034e66551e32b381cb672291b64f6b0140d616190ce83 + Match(End): cases 1 accepted 1 rejected 0 rejection None digest 4a1861f34feb55ddb21f549739bd5666dd8fc27552f3a3173baaab2eb34df19d + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 7190a38fff28dca89d3aadb7d4f07ab67349fd3448877b469ef32bcf9af7c8ed + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 1406117ea9fa975564ae00ee94b0c99cc479fd72616f417d1b5901b525529cd4 + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 215bf8bb454b8bb072d8e980f85570b12f49a7e3bf889bee8773b54b1aff5184 + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest eca762246a893da6ade59d27aafc0da76e88cd7d83586ca687a0ee6fc689dcef + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest b2a369f93b2e2f1aa801ecedbf76c018cd5e9e54dffdfa368618b7e23ad6099a + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest e4d7c0752bde967a6a6c229128f2473713cac168302411d2068692ed867a4f8b + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 8febe5d88db3eb980d967075e77a4eacd709fadf23104dce8fbdaa4e771f9094 + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 68b3b4adc4d89d691374d0296f39d9667fd4b6206f1a203e20d19a37fa73e93d stream 14 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 412f432fbfc37aa305017091196a1cf08cdcfb58286c5d67a5879a7adfe41a98 - Match(End): cases 1 accepted 1 rejected 0 rejection None digest b926b2fa40df45f59a10eb48153efaba5c5aa388262e69137f571acab8678ad5 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 2d87757635342d0fd27aff56df18325535b7886318a1508f3dadd6a540a30e9f - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 5d537842e1e44c2fb2663485c69818515a5d724e84ab275eeeda1534a66b326f - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest b0adb97cf6e8fd75a3d6eca5d1ba6a81b4e9cfbce99793d9374bf35e6a53f756 - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 1a598804994f12a0bf7db52814ab606b664bb716c6077797d25ca28777067d00 - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest b8637f87ab12074b33dd6ef0cf82da1a1e576167190c225c72f9578963208c00 - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 28a6bede5bd63c5a42a5f30e4f7dbcc3628634af2962a8e8839ae4b9c29f604d - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 2f182802ac4c754db7b1e709ec1e2c62fed5a070e2c068faba47bb7964d3f4e2 - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest f5c41bf5e61ad5831f4929f863cfa6bfe71fa3b6ee9b27a21e7c0a7595e357b7 + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 16131963006b632de280a07deb46ea192c15d2dc8e25ddf274461261ae0ebca3 + Match(End): cases 1 accepted 1 rejected 0 rejection None digest 21a867cfc349c464b9c79771ff5ce115a7fb6c77c69eab4333c654d1617ec112 + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest cf8f33059852e53a2e2e01d7106399ece848983621f1dc666fb7d0c7ea6561e9 + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest f9e69834a2d2b1a86a0bc36b29a8cdd8d6af1c4d59e93c2fab12517ceb7ceb6f + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest b160156964892a2f2f7a872f1c2702fc6a8ed893ea3eade80f4a74931f4043f1 + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 22185f43a8a2836d412f0a7498ecc7dd9194720e2056c8b59bf95334de525cc9 + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest 3c85c62b9b1dab4c45bb9c9af63ca1de990e12d5402e2341e92d22703ed27095 + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 67070ac2787a5aec30323b7b1fd7e45c7d018878de75503d80e5e42ffb7f52a4 + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest d95cc1a17f764327b1a5269a0ade95b190f22f0d68b801023394934525457971 + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 71e233055c53dfe2ac77e4d9cf92dee19af492e82c3e32d7eb93dbd28d4eac8c stream 15 - Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest 857b425150218962c0a7548ad7867d0223631dbca974fdcc84cb94019355c7ad - Match(End): cases 1 accepted 1 rejected 0 rejection None digest 95287e0900ba61bf3efa683215b6b74a25af987141de2f338f74b9e9089c9694 - QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 98956d902b7f9ded95fe6659d503239a457ff8e0f9557ca5e48c5f3f73cafeb9 - QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 627c3e7aff204f684040fcf3fd123b5c9cfb945c9aae4167b12eab4723b953f0 - Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest b8bef3ff0962d6a257250d4bb4876d9f1d9aab401cd818b3e9a2c9475e1ec67e - Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 97fec0191ef2a75b3d6440739172679e85e8c853f502199048fa9c34fe479ad9 - Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest e2a690dbe15e9f6276788dee265fa948b186a5904698d55aa72c1ebf49bd7b9f - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest b10ff7406dbceaf57f9edfd790369eda2149677347a3bc5a15226dbe0928d0ca - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 033a9d6c6cb059c17270280ac6121bc6cff59ff0b452c49cb65dce6f55e992a7 - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest d4fc5053bcf8daa493d15dfb6750582b5efc3690d8eb3260ed4645e012789c4b + Match(Continue): cases 1 accepted 1 rejected 0 rejection None digest baa489d341a4a2700fda70bf6886f5b9dab805820201010b108c47ee303eba41 + Match(End): cases 1 accepted 1 rejected 0 rejection None digest e01b98e21bd5c63e254e0bffd894935132c4d19139611e842ece4c4b57365538 + QueryEmpty(Continue): cases 1 accepted 1 rejected 0 rejection None digest 07248d0e233debd4957a759f5a206c0e19208e530dd0ae0b289433e0e4efdf89 + QueryEmpty(End): cases 1 accepted 1 rejected 0 rejection None digest 350c9e7214b4d19dda13cc2a40f5b0cdcba493056962d9973e2197039614545c + Query(Continue): cases 32896 accepted 32896 rejected 0 rejection None digest 15a6a6f14af85c5f40b3395c445330d1cc20676dad8257390e5e5e92a500b094 + Query(End): cases 32896 accepted 32896 rejected 0 rejection None digest 1ea6e03cf037327f78a691daa80c95514ed71fa4475d0bb7e840af5f13cabf87 + Supply(Continue): cases 1 accepted 1 rejected 0 rejection None digest d1a34d662eeaf09b1094de8f6ffc636311b783883ff9218288ea81f66e685650 + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest c90d18111ee0768484b386281a560fad062ba6f8f11871256f4fc3dfb2830e76 + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest c80c79a90c901d855e127dc81690b9357fd87a290cf7a856d4618c1a54cfe7fb + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest a76debc09aff00b2f5343ea22224544c69ec8793774afc18a19783fdeb5cdf64 stream 16 Match(Continue): cases 1 accepted 0 rejected 1 rejection Some(TerminalLeafReplies) digest af35b4bbaa746c77cbfb5f3e4badd404be07dfbdb84fd40c73823d2cc7e1b054 Match(End): cases 1 accepted 0 rejected 1 rejection Some(TerminalLeafReplies) digest 64dcda0a643b88998ba505638306db5435d5cfb3c30c4b71b8633f4074288a13 @@ -375,6 +375,6 @@ Responder Query(Continue): cases 32896 accepted 0 rejected 32896 rejection Some(TerminalLeafReplies) digest d678b609a984aec0e82f6704ec78d2c1312a319c00aef09315cb6d5a97afd3b3 Query(End): cases 32896 accepted 0 rejected 32896 rejection Some(TerminalLeafReplies) digest 20206e46e9569954fa7409adc1466295dd78517ee9b1f7a6dcb931af522b09f2 Supply(Continue): cases 1 accepted 0 rejected 1 rejection Some(TerminalLeafReplies) digest 8a98948334763852e22cefc20322ea74bfef9f5b97b0c1ee2b11ccf8281288b3 - Supply(End): cases 1 accepted 1 rejected 0 rejection None digest 29c688db20ab6a6a85b8a64ea17a0f0c9e3380c16c02a687588891c443cd6664 - End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 7a250d61fe85b397ff21c6c0d2883c934615f0c59d3f7fa92090f2bbc974cb74 - End(Stream): cases 1 accepted 1 rejected 0 rejection None digest 69e0a19208d6edd30e31916235b05fcd9f3072a9f93668a49e7cd2d971621588 + Supply(End): cases 1 accepted 1 rejected 0 rejection None digest c7350f8f4be155503abf5f41d5d6ebb144b3c4068d4d41c66fb2d8aefb31c346 + End(Reply): cases 1 accepted 1 rejected 0 rejection None digest 7257d21c0f045fbdaae74ddd1f9c1ac52ac58d85f6b2efc24a92e2c2c7b48d61 + End(Stream): cases 1 accepted 1 rejected 0 rejection None digest c19c0586156caa168dd0e5703a80d90674d0554d1712c535f49d55172049b0b4 diff --git a/src/tree/mirror/streaming/remote/codec/snapshots/rumors__tree__mirror__streaming__remote__codec__tests__canonical_frame_atlas_snapshot.snap b/src/tree/mirror/streaming/remote/codec/snapshots/rumors__tree__mirror__streaming__remote__codec__tests__canonical_frame_atlas_snapshot.snap index 620c353a9..e92628340 100644 --- a/src/tree/mirror/streaming/remote/codec/snapshots/rumors__tree__mirror__streaming__remote__codec__tests__canonical_frame_atlas_snapshot.snap +++ b/src/tree/mirror/streaming/remote/codec/snapshots/rumors__tree__mirror__streaming__remote__codec__tests__canonical_frame_atlas_snapshot.snap @@ -10,363 +10,363 @@ Initiator QueryEmpty(End): rejected byte 33 class OpeningSupplies Query(Continue): rejected byte 44 class OpeningSupplies Query(End): rejected byte 55 class OpeningSupplies - Supply(Continue): accepted len 12 hex 66000000070000000341e0f6 - Supply(End): accepted len 12 hex 77000000070000000341e0f6 - End(Reply): accepted len 1 hex 88 - End(Stream): accepted len 1 hex 99 + Supply(Continue): accepted len 15 hex 821866d83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 821877d83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 811888 + End(Stream): accepted len 3 hex 811899 stream 01 - Match(Continue): accepted len 1 hex 01 - Match(End): accepted len 1 hex 12 - QueryEmpty(Continue): accepted len 1 hex 23 - QueryEmpty(End): accepted len 1 hex 34 - Query(Continue): accepted len 27 hex 450000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 560000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 67000000070000000341e0f6 - Supply(End): accepted len 12 hex 78000000070000000341e0f6 - End(Reply): accepted len 1 hex 89 - End(Stream): accepted len 1 hex 9a + Match(Continue): accepted len 2 hex 8101 + Match(End): accepted len 2 hex 8112 + QueryEmpty(Continue): accepted len 3 hex 811823 + QueryEmpty(End): accepted len 3 hex 811834 + Query(Continue): accepted len 31 hex 821845a1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 821856a1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 821867d83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 821878d83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 811889 + End(Stream): accepted len 3 hex 81189a stream 02 - Match(Continue): accepted len 1 hex 02 - Match(End): accepted len 1 hex 13 - QueryEmpty(Continue): accepted len 1 hex 24 - QueryEmpty(End): accepted len 1 hex 35 - Query(Continue): accepted len 27 hex 460000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 570000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 68000000070000000341e0f6 - Supply(End): accepted len 12 hex 79000000070000000341e0f6 - End(Reply): accepted len 1 hex 8a - End(Stream): accepted len 1 hex 9b + Match(Continue): accepted len 2 hex 8102 + Match(End): accepted len 2 hex 8113 + QueryEmpty(Continue): accepted len 3 hex 811824 + QueryEmpty(End): accepted len 3 hex 811835 + Query(Continue): accepted len 31 hex 821846a1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 821857a1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 821868d83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 821879d83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 81188a + End(Stream): accepted len 3 hex 81189b stream 03 - Match(Continue): accepted len 1 hex 03 - Match(End): accepted len 1 hex 14 - QueryEmpty(Continue): accepted len 1 hex 25 - QueryEmpty(End): accepted len 1 hex 36 - Query(Continue): accepted len 27 hex 470000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 580000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 69000000070000000341e0f6 - Supply(End): accepted len 12 hex 7a000000070000000341e0f6 - End(Reply): accepted len 1 hex 8b - End(Stream): accepted len 1 hex 9c + Match(Continue): accepted len 2 hex 8103 + Match(End): accepted len 2 hex 8114 + QueryEmpty(Continue): accepted len 3 hex 811825 + QueryEmpty(End): accepted len 3 hex 811836 + Query(Continue): accepted len 31 hex 821847a1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 821858a1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 821869d83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 82187ad83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 81188b + End(Stream): accepted len 3 hex 81189c stream 04 - Match(Continue): accepted len 1 hex 04 - Match(End): accepted len 1 hex 15 - QueryEmpty(Continue): accepted len 1 hex 26 - QueryEmpty(End): accepted len 1 hex 37 - Query(Continue): accepted len 27 hex 480000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 590000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 6a000000070000000341e0f6 - Supply(End): accepted len 12 hex 7b000000070000000341e0f6 - End(Reply): accepted len 1 hex 8c - End(Stream): accepted len 1 hex 9d + Match(Continue): accepted len 2 hex 8104 + Match(End): accepted len 2 hex 8115 + QueryEmpty(Continue): accepted len 3 hex 811826 + QueryEmpty(End): accepted len 3 hex 811837 + Query(Continue): accepted len 31 hex 821848a1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 821859a1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 82186ad83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 82187bd83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 81188c + End(Stream): accepted len 3 hex 81189d stream 05 - Match(Continue): accepted len 1 hex 05 - Match(End): accepted len 1 hex 16 - QueryEmpty(Continue): accepted len 1 hex 27 - QueryEmpty(End): accepted len 1 hex 38 - Query(Continue): accepted len 27 hex 490000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 5a0000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 6b000000070000000341e0f6 - Supply(End): accepted len 12 hex 7c000000070000000341e0f6 - End(Reply): accepted len 1 hex 8d - End(Stream): accepted len 1 hex 9e + Match(Continue): accepted len 2 hex 8105 + Match(End): accepted len 2 hex 8116 + QueryEmpty(Continue): accepted len 3 hex 811827 + QueryEmpty(End): accepted len 3 hex 811838 + Query(Continue): accepted len 31 hex 821849a1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 82185aa1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 82186bd83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 82187cd83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 81188d + End(Stream): accepted len 3 hex 81189e stream 06 - Match(Continue): accepted len 1 hex 06 - Match(End): accepted len 1 hex 17 - QueryEmpty(Continue): accepted len 1 hex 28 - QueryEmpty(End): accepted len 1 hex 39 - Query(Continue): accepted len 27 hex 4a0000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 5b0000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 6c000000070000000341e0f6 - Supply(End): accepted len 12 hex 7d000000070000000341e0f6 - End(Reply): accepted len 1 hex 8e - End(Stream): accepted len 1 hex 9f + Match(Continue): accepted len 2 hex 8106 + Match(End): accepted len 2 hex 8117 + QueryEmpty(Continue): accepted len 3 hex 811828 + QueryEmpty(End): accepted len 3 hex 811839 + Query(Continue): accepted len 31 hex 82184aa1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 82185ba1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 82186cd83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 82187dd83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 81188e + End(Stream): accepted len 3 hex 81189f stream 07 - Match(Continue): accepted len 1 hex 07 - Match(End): accepted len 1 hex 18 - QueryEmpty(Continue): accepted len 1 hex 29 - QueryEmpty(End): accepted len 1 hex 3a - Query(Continue): accepted len 27 hex 4b0000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 5c0000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 6d000000070000000341e0f6 - Supply(End): accepted len 12 hex 7e000000070000000341e0f6 - End(Reply): accepted len 1 hex 8f - End(Stream): accepted len 1 hex a0 + Match(Continue): accepted len 2 hex 8107 + Match(End): accepted len 3 hex 811818 + QueryEmpty(Continue): accepted len 3 hex 811829 + QueryEmpty(End): accepted len 3 hex 81183a + Query(Continue): accepted len 31 hex 82184ba1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 82185ca1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 82186dd83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 82187ed83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 81188f + End(Stream): accepted len 3 hex 8118a0 stream 08 - Match(Continue): accepted len 1 hex 08 - Match(End): accepted len 1 hex 19 - QueryEmpty(Continue): accepted len 1 hex 2a - QueryEmpty(End): accepted len 1 hex 3b - Query(Continue): accepted len 27 hex 4c0000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 5d0000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 6e000000070000000341e0f6 - Supply(End): accepted len 12 hex 7f000000070000000341e0f6 - End(Reply): accepted len 1 hex 90 - End(Stream): accepted len 1 hex a1 + Match(Continue): accepted len 2 hex 8108 + Match(End): accepted len 3 hex 811819 + QueryEmpty(Continue): accepted len 3 hex 81182a + QueryEmpty(End): accepted len 3 hex 81183b + Query(Continue): accepted len 31 hex 82184ca1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 82185da1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 82186ed83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 82187fd83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 811890 + End(Stream): accepted len 3 hex 8118a1 stream 09 - Match(Continue): accepted len 1 hex 09 - Match(End): accepted len 1 hex 1a - QueryEmpty(Continue): accepted len 1 hex 2b - QueryEmpty(End): accepted len 1 hex 3c - Query(Continue): accepted len 27 hex 4d0000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 5e0000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 6f000000070000000341e0f6 - Supply(End): accepted len 12 hex 80000000070000000341e0f6 - End(Reply): accepted len 1 hex 91 - End(Stream): accepted len 1 hex a2 + Match(Continue): accepted len 2 hex 8109 + Match(End): accepted len 3 hex 81181a + QueryEmpty(Continue): accepted len 3 hex 81182b + QueryEmpty(End): accepted len 3 hex 81183c + Query(Continue): accepted len 31 hex 82184da1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 82185ea1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 82186fd83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 821880d83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 811891 + End(Stream): accepted len 3 hex 8118a2 stream 10 - Match(Continue): accepted len 1 hex 0a - Match(End): accepted len 1 hex 1b - QueryEmpty(Continue): accepted len 1 hex 2c - QueryEmpty(End): accepted len 1 hex 3d - Query(Continue): accepted len 27 hex 4e0000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 5f0000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 70000000070000000341e0f6 - Supply(End): accepted len 12 hex 81000000070000000341e0f6 - End(Reply): accepted len 1 hex 92 - End(Stream): accepted len 1 hex a3 + Match(Continue): accepted len 2 hex 810a + Match(End): accepted len 3 hex 81181b + QueryEmpty(Continue): accepted len 3 hex 81182c + QueryEmpty(End): accepted len 3 hex 81183d + Query(Continue): accepted len 31 hex 82184ea1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 82185fa1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 821870d83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 821881d83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 811892 + End(Stream): accepted len 3 hex 8118a3 stream 11 - Match(Continue): accepted len 1 hex 0b - Match(End): accepted len 1 hex 1c - QueryEmpty(Continue): accepted len 1 hex 2d - QueryEmpty(End): accepted len 1 hex 3e - Query(Continue): accepted len 27 hex 4f0000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 600000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 71000000070000000341e0f6 - Supply(End): accepted len 12 hex 82000000070000000341e0f6 - End(Reply): accepted len 1 hex 93 - End(Stream): accepted len 1 hex a4 + Match(Continue): accepted len 2 hex 810b + Match(End): accepted len 3 hex 81181c + QueryEmpty(Continue): accepted len 3 hex 81182d + QueryEmpty(End): accepted len 3 hex 81183e + Query(Continue): accepted len 31 hex 82184fa1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 821860a1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 821871d83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 821882d83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 811893 + End(Stream): accepted len 3 hex 8118a4 stream 12 - Match(Continue): accepted len 1 hex 0c - Match(End): accepted len 1 hex 1d - QueryEmpty(Continue): accepted len 1 hex 2e - QueryEmpty(End): accepted len 1 hex 3f - Query(Continue): accepted len 27 hex 500000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 610000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 72000000070000000341e0f6 - Supply(End): accepted len 12 hex 83000000070000000341e0f6 - End(Reply): accepted len 1 hex 94 - End(Stream): accepted len 1 hex a5 + Match(Continue): accepted len 2 hex 810c + Match(End): accepted len 3 hex 81181d + QueryEmpty(Continue): accepted len 3 hex 81182e + QueryEmpty(End): accepted len 3 hex 81183f + Query(Continue): accepted len 31 hex 821850a1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 821861a1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 821872d83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 821883d83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 811894 + End(Stream): accepted len 3 hex 8118a5 stream 13 - Match(Continue): accepted len 1 hex 0d - Match(End): accepted len 1 hex 1e - QueryEmpty(Continue): accepted len 1 hex 2f - QueryEmpty(End): accepted len 1 hex 40 - Query(Continue): accepted len 27 hex 510000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 620000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 73000000070000000341e0f6 - Supply(End): accepted len 12 hex 84000000070000000341e0f6 - End(Reply): accepted len 1 hex 95 - End(Stream): accepted len 1 hex a6 + Match(Continue): accepted len 2 hex 810d + Match(End): accepted len 3 hex 81181e + QueryEmpty(Continue): accepted len 3 hex 81182f + QueryEmpty(End): accepted len 3 hex 811840 + Query(Continue): accepted len 31 hex 821851a1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 821862a1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 821873d83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 821884d83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 811895 + End(Stream): accepted len 3 hex 8118a6 stream 14 - Match(Continue): accepted len 1 hex 0e - Match(End): accepted len 1 hex 1f - QueryEmpty(Continue): accepted len 1 hex 30 - QueryEmpty(End): accepted len 1 hex 41 - Query(Continue): accepted len 27 hex 520000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 630000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 74000000070000000341e0f6 - Supply(End): accepted len 12 hex 85000000070000000341e0f6 - End(Reply): accepted len 1 hex 96 - End(Stream): accepted len 1 hex a7 + Match(Continue): accepted len 2 hex 810e + Match(End): accepted len 3 hex 81181f + QueryEmpty(Continue): accepted len 3 hex 811830 + QueryEmpty(End): accepted len 3 hex 811841 + Query(Continue): accepted len 31 hex 821852a1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 821863a1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 821874d83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 821885d83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 811896 + End(Stream): accepted len 3 hex 8118a7 stream 15 - Match(Continue): accepted len 1 hex 0f - Match(End): accepted len 1 hex 20 - QueryEmpty(Continue): accepted len 1 hex 31 - QueryEmpty(End): accepted len 1 hex 42 - Query(Continue): accepted len 27 hex 530000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 640000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 75000000070000000341e0f6 - Supply(End): accepted len 12 hex 86000000070000000341e0f6 - End(Reply): accepted len 1 hex 97 - End(Stream): accepted len 1 hex a8 + Match(Continue): accepted len 2 hex 810f + Match(End): accepted len 3 hex 811820 + QueryEmpty(Continue): accepted len 3 hex 811831 + QueryEmpty(End): accepted len 3 hex 811842 + Query(Continue): accepted len 31 hex 821853a1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 821864a1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 821875d83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 821886d83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 811897 + End(Stream): accepted len 3 hex 8118a8 stream 16 - Match(Continue): accepted len 1 hex 10 - Match(End): accepted len 1 hex 21 - QueryEmpty(Continue): accepted len 1 hex 32 - QueryEmpty(End): accepted len 1 hex 43 + Match(Continue): accepted len 2 hex 8110 + Match(End): accepted len 3 hex 811821 + QueryEmpty(Continue): accepted len 3 hex 811832 + QueryEmpty(End): accepted len 3 hex 811843 Query(Continue): rejected byte 54 class LeafParentReplies Query(End): rejected byte 65 class LeafParentReplies - Supply(Continue): accepted len 12 hex 76000000070000000341e0f6 - Supply(End): accepted len 12 hex 87000000070000000341e0f6 - End(Reply): accepted len 1 hex 98 - End(Stream): accepted len 1 hex a9 + Supply(Continue): accepted len 15 hex 821876d83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 821887d83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 811898 + End(Stream): accepted len 3 hex 8118a9 Responder stream 00 - Match(Continue): accepted len 1 hex 00 - Match(End): accepted len 1 hex 11 - QueryEmpty(Continue): accepted len 1 hex 22 - QueryEmpty(End): accepted len 1 hex 33 - Query(Continue): accepted len 27 hex 440000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 550000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 66000000070000000341e0f6 - Supply(End): accepted len 12 hex 77000000070000000341e0f6 - End(Reply): accepted len 1 hex 88 - End(Stream): accepted len 1 hex 99 + Match(Continue): accepted len 2 hex 8100 + Match(End): accepted len 2 hex 8111 + QueryEmpty(Continue): accepted len 3 hex 811822 + QueryEmpty(End): accepted len 3 hex 811833 + Query(Continue): accepted len 31 hex 821844a1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 821855a1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 821866d83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 821877d83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 811888 + End(Stream): accepted len 3 hex 811899 stream 01 - Match(Continue): accepted len 1 hex 01 - Match(End): accepted len 1 hex 12 - QueryEmpty(Continue): accepted len 1 hex 23 - QueryEmpty(End): accepted len 1 hex 34 - Query(Continue): accepted len 27 hex 450000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 560000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 67000000070000000341e0f6 - Supply(End): accepted len 12 hex 78000000070000000341e0f6 - End(Reply): accepted len 1 hex 89 - End(Stream): accepted len 1 hex 9a + Match(Continue): accepted len 2 hex 8101 + Match(End): accepted len 2 hex 8112 + QueryEmpty(Continue): accepted len 3 hex 811823 + QueryEmpty(End): accepted len 3 hex 811834 + Query(Continue): accepted len 31 hex 821845a1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 821856a1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 821867d83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 821878d83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 811889 + End(Stream): accepted len 3 hex 81189a stream 02 - Match(Continue): accepted len 1 hex 02 - Match(End): accepted len 1 hex 13 - QueryEmpty(Continue): accepted len 1 hex 24 - QueryEmpty(End): accepted len 1 hex 35 - Query(Continue): accepted len 27 hex 460000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 570000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 68000000070000000341e0f6 - Supply(End): accepted len 12 hex 79000000070000000341e0f6 - End(Reply): accepted len 1 hex 8a - End(Stream): accepted len 1 hex 9b + Match(Continue): accepted len 2 hex 8102 + Match(End): accepted len 2 hex 8113 + QueryEmpty(Continue): accepted len 3 hex 811824 + QueryEmpty(End): accepted len 3 hex 811835 + Query(Continue): accepted len 31 hex 821846a1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 821857a1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 821868d83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 821879d83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 81188a + End(Stream): accepted len 3 hex 81189b stream 03 - Match(Continue): accepted len 1 hex 03 - Match(End): accepted len 1 hex 14 - QueryEmpty(Continue): accepted len 1 hex 25 - QueryEmpty(End): accepted len 1 hex 36 - Query(Continue): accepted len 27 hex 470000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 580000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 69000000070000000341e0f6 - Supply(End): accepted len 12 hex 7a000000070000000341e0f6 - End(Reply): accepted len 1 hex 8b - End(Stream): accepted len 1 hex 9c + Match(Continue): accepted len 2 hex 8103 + Match(End): accepted len 2 hex 8114 + QueryEmpty(Continue): accepted len 3 hex 811825 + QueryEmpty(End): accepted len 3 hex 811836 + Query(Continue): accepted len 31 hex 821847a1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 821858a1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 821869d83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 82187ad83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 81188b + End(Stream): accepted len 3 hex 81189c stream 04 - Match(Continue): accepted len 1 hex 04 - Match(End): accepted len 1 hex 15 - QueryEmpty(Continue): accepted len 1 hex 26 - QueryEmpty(End): accepted len 1 hex 37 - Query(Continue): accepted len 27 hex 480000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 590000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 6a000000070000000341e0f6 - Supply(End): accepted len 12 hex 7b000000070000000341e0f6 - End(Reply): accepted len 1 hex 8c - End(Stream): accepted len 1 hex 9d + Match(Continue): accepted len 2 hex 8104 + Match(End): accepted len 2 hex 8115 + QueryEmpty(Continue): accepted len 3 hex 811826 + QueryEmpty(End): accepted len 3 hex 811837 + Query(Continue): accepted len 31 hex 821848a1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 821859a1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 82186ad83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 82187bd83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 81188c + End(Stream): accepted len 3 hex 81189d stream 05 - Match(Continue): accepted len 1 hex 05 - Match(End): accepted len 1 hex 16 - QueryEmpty(Continue): accepted len 1 hex 27 - QueryEmpty(End): accepted len 1 hex 38 - Query(Continue): accepted len 27 hex 490000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 5a0000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 6b000000070000000341e0f6 - Supply(End): accepted len 12 hex 7c000000070000000341e0f6 - End(Reply): accepted len 1 hex 8d - End(Stream): accepted len 1 hex 9e + Match(Continue): accepted len 2 hex 8105 + Match(End): accepted len 2 hex 8116 + QueryEmpty(Continue): accepted len 3 hex 811827 + QueryEmpty(End): accepted len 3 hex 811838 + Query(Continue): accepted len 31 hex 821849a1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 82185aa1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 82186bd83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 82187cd83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 81188d + End(Stream): accepted len 3 hex 81189e stream 06 - Match(Continue): accepted len 1 hex 06 - Match(End): accepted len 1 hex 17 - QueryEmpty(Continue): accepted len 1 hex 28 - QueryEmpty(End): accepted len 1 hex 39 - Query(Continue): accepted len 27 hex 4a0000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 5b0000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 6c000000070000000341e0f6 - Supply(End): accepted len 12 hex 7d000000070000000341e0f6 - End(Reply): accepted len 1 hex 8e - End(Stream): accepted len 1 hex 9f + Match(Continue): accepted len 2 hex 8106 + Match(End): accepted len 2 hex 8117 + QueryEmpty(Continue): accepted len 3 hex 811828 + QueryEmpty(End): accepted len 3 hex 811839 + Query(Continue): accepted len 31 hex 82184aa1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 82185ba1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 82186cd83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 82187dd83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 81188e + End(Stream): accepted len 3 hex 81189f stream 07 - Match(Continue): accepted len 1 hex 07 - Match(End): accepted len 1 hex 18 - QueryEmpty(Continue): accepted len 1 hex 29 - QueryEmpty(End): accepted len 1 hex 3a - Query(Continue): accepted len 27 hex 4b0000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 5c0000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 6d000000070000000341e0f6 - Supply(End): accepted len 12 hex 7e000000070000000341e0f6 - End(Reply): accepted len 1 hex 8f - End(Stream): accepted len 1 hex a0 + Match(Continue): accepted len 2 hex 8107 + Match(End): accepted len 3 hex 811818 + QueryEmpty(Continue): accepted len 3 hex 811829 + QueryEmpty(End): accepted len 3 hex 81183a + Query(Continue): accepted len 31 hex 82184ba1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 82185ca1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 82186dd83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 82187ed83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 81188f + End(Stream): accepted len 3 hex 8118a0 stream 08 - Match(Continue): accepted len 1 hex 08 - Match(End): accepted len 1 hex 19 - QueryEmpty(Continue): accepted len 1 hex 2a - QueryEmpty(End): accepted len 1 hex 3b - Query(Continue): accepted len 27 hex 4c0000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 5d0000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 6e000000070000000341e0f6 - Supply(End): accepted len 12 hex 7f000000070000000341e0f6 - End(Reply): accepted len 1 hex 90 - End(Stream): accepted len 1 hex a1 + Match(Continue): accepted len 2 hex 8108 + Match(End): accepted len 3 hex 811819 + QueryEmpty(Continue): accepted len 3 hex 81182a + QueryEmpty(End): accepted len 3 hex 81183b + Query(Continue): accepted len 31 hex 82184ca1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 82185da1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 82186ed83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 82187fd83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 811890 + End(Stream): accepted len 3 hex 8118a1 stream 09 - Match(Continue): accepted len 1 hex 09 - Match(End): accepted len 1 hex 1a - QueryEmpty(Continue): accepted len 1 hex 2b - QueryEmpty(End): accepted len 1 hex 3c - Query(Continue): accepted len 27 hex 4d0000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 5e0000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 6f000000070000000341e0f6 - Supply(End): accepted len 12 hex 80000000070000000341e0f6 - End(Reply): accepted len 1 hex 91 - End(Stream): accepted len 1 hex a2 + Match(Continue): accepted len 2 hex 8109 + Match(End): accepted len 3 hex 81181a + QueryEmpty(Continue): accepted len 3 hex 81182b + QueryEmpty(End): accepted len 3 hex 81183c + Query(Continue): accepted len 31 hex 82184da1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 82185ea1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 82186fd83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 821880d83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 811891 + End(Stream): accepted len 3 hex 8118a2 stream 10 - Match(Continue): accepted len 1 hex 0a - Match(End): accepted len 1 hex 1b - QueryEmpty(Continue): accepted len 1 hex 2c - QueryEmpty(End): accepted len 1 hex 3d - Query(Continue): accepted len 27 hex 4e0000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 5f0000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 70000000070000000341e0f6 - Supply(End): accepted len 12 hex 81000000070000000341e0f6 - End(Reply): accepted len 1 hex 92 - End(Stream): accepted len 1 hex a3 + Match(Continue): accepted len 2 hex 810a + Match(End): accepted len 3 hex 81181b + QueryEmpty(Continue): accepted len 3 hex 81182c + QueryEmpty(End): accepted len 3 hex 81183d + Query(Continue): accepted len 31 hex 82184ea1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 82185fa1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 821870d83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 821881d83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 811892 + End(Stream): accepted len 3 hex 8118a3 stream 11 - Match(Continue): accepted len 1 hex 0b - Match(End): accepted len 1 hex 1c - QueryEmpty(Continue): accepted len 1 hex 2d - QueryEmpty(End): accepted len 1 hex 3e - Query(Continue): accepted len 27 hex 4f0000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 600000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 71000000070000000341e0f6 - Supply(End): accepted len 12 hex 82000000070000000341e0f6 - End(Reply): accepted len 1 hex 93 - End(Stream): accepted len 1 hex a4 + Match(Continue): accepted len 2 hex 810b + Match(End): accepted len 3 hex 81181c + QueryEmpty(Continue): accepted len 3 hex 81182d + QueryEmpty(End): accepted len 3 hex 81183e + Query(Continue): accepted len 31 hex 82184fa1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 821860a1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 821871d83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 821882d83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 811893 + End(Stream): accepted len 3 hex 8118a4 stream 12 - Match(Continue): accepted len 1 hex 0c - Match(End): accepted len 1 hex 1d - QueryEmpty(Continue): accepted len 1 hex 2e - QueryEmpty(End): accepted len 1 hex 3f - Query(Continue): accepted len 27 hex 500000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 610000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 72000000070000000341e0f6 - Supply(End): accepted len 12 hex 83000000070000000341e0f6 - End(Reply): accepted len 1 hex 94 - End(Stream): accepted len 1 hex a5 + Match(Continue): accepted len 2 hex 810c + Match(End): accepted len 3 hex 81181d + QueryEmpty(Continue): accepted len 3 hex 81182e + QueryEmpty(End): accepted len 3 hex 81183f + Query(Continue): accepted len 31 hex 821850a1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 821861a1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 821872d83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 821883d83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 811894 + End(Stream): accepted len 3 hex 8118a5 stream 13 - Match(Continue): accepted len 1 hex 0d - Match(End): accepted len 1 hex 1e - QueryEmpty(Continue): accepted len 1 hex 2f - QueryEmpty(End): accepted len 1 hex 40 - Query(Continue): accepted len 27 hex 510000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 620000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 73000000070000000341e0f6 - Supply(End): accepted len 12 hex 84000000070000000341e0f6 - End(Reply): accepted len 1 hex 95 - End(Stream): accepted len 1 hex a6 + Match(Continue): accepted len 2 hex 810d + Match(End): accepted len 3 hex 81181e + QueryEmpty(Continue): accepted len 3 hex 81182f + QueryEmpty(End): accepted len 3 hex 811840 + Query(Continue): accepted len 31 hex 821851a1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 821862a1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 821873d83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 821884d83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 811895 + End(Stream): accepted len 3 hex 8118a6 stream 14 - Match(Continue): accepted len 1 hex 0e - Match(End): accepted len 1 hex 1f - QueryEmpty(Continue): accepted len 1 hex 30 - QueryEmpty(End): accepted len 1 hex 41 - Query(Continue): accepted len 27 hex 520000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 630000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 74000000070000000341e0f6 - Supply(End): accepted len 12 hex 85000000070000000341e0f6 - End(Reply): accepted len 1 hex 96 - End(Stream): accepted len 1 hex a7 + Match(Continue): accepted len 2 hex 810e + Match(End): accepted len 3 hex 81181f + QueryEmpty(Continue): accepted len 3 hex 811830 + QueryEmpty(End): accepted len 3 hex 811841 + Query(Continue): accepted len 31 hex 821852a1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 821863a1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 821874d83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 821885d83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 811896 + End(Stream): accepted len 3 hex 8118a7 stream 15 - Match(Continue): accepted len 1 hex 0f - Match(End): accepted len 1 hex 20 - QueryEmpty(Continue): accepted len 1 hex 31 - QueryEmpty(End): accepted len 1 hex 42 - Query(Continue): accepted len 27 hex 530000000000000000000000000000000000000000000000000000 - Query(End): accepted len 27 hex 640000000000000000000000000000000000000000000000000000 - Supply(Continue): accepted len 12 hex 75000000070000000341e0f6 - Supply(End): accepted len 12 hex 86000000070000000341e0f6 - End(Reply): accepted len 1 hex 97 - End(Stream): accepted len 1 hex a8 + Match(Continue): accepted len 2 hex 810f + Match(End): accepted len 3 hex 811820 + QueryEmpty(Continue): accepted len 3 hex 811831 + QueryEmpty(End): accepted len 3 hex 811842 + Query(Continue): accepted len 31 hex 821853a1005818000000000000000000000000000000000000000000000000 + Query(End): accepted len 31 hex 821864a1005818000000000000000000000000000000000000000000000000 + Supply(Continue): accepted len 15 hex 821875d83f49d83f46d9d25641e0f6 + Supply(End): accepted len 15 hex 821886d83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 811897 + End(Stream): accepted len 3 hex 8118a8 stream 16 Match(Continue): rejected byte 10 class TerminalLeafReplies Match(End): rejected byte 21 class TerminalLeafReplies @@ -375,6 +375,6 @@ Responder Query(Continue): rejected byte 54 class TerminalLeafReplies Query(End): rejected byte 65 class TerminalLeafReplies Supply(Continue): rejected byte 76 class TerminalLeafReplies - Supply(End): accepted len 12 hex 87000000070000000341e0f6 - End(Reply): accepted len 1 hex 98 - End(Stream): accepted len 1 hex a9 + Supply(End): accepted len 15 hex 821887d83f49d83f46d9d25641e0f6 + End(Reply): accepted len 3 hex 811898 + End(Stream): accepted len 3 hex 8118a9 diff --git a/src/tree/mirror/streaming/remote/codec/tests.rs b/src/tree/mirror/streaming/remote/codec/tests.rs index cfda12463..0551193d3 100644 --- a/src/tree/mirror/streaming/remote/codec/tests.rs +++ b/src/tree/mirror/streaming/remote/codec/tests.rs @@ -256,7 +256,15 @@ fn canonical_frame_atlas_snapshot() { Ok(wire) => { let mut encoded = Vec::new(); encode(speaker, &(stream, frame.clone()), &mut encoded).unwrap(); - assert_eq!(encoded.first(), Some(&wire.to_byte())); + // The frame head carries the dense code as a uint + // item right behind the array head. + let mut expected_signal = Vec::new(); + crate::tree::mirror::cbor::write_head( + &mut expected_signal, + crate::tree::mirror::cbor::MAJOR_UINT, + u64::from(wire.to_byte()), + ); + assert_eq!(&encoded[1..1 + expected_signal.len()], expected_signal); assert_eq!( decode_exact(speaker, RunBudget::default(), &encoded).unwrap(), (stream, frame) @@ -267,8 +275,19 @@ fn canonical_frame_atlas_snapshot() { atlas.push('\n'); } Err(invalid) => { - let error = decode_exact(speaker, RunBudget::default(), &[invalid.byte()]) - .unwrap_err(); + let mut rejected = Vec::new(); + crate::tree::mirror::cbor::write_head( + &mut rejected, + crate::tree::mirror::cbor::MAJOR_ARRAY, + 1, + ); + crate::tree::mirror::cbor::write_head( + &mut rejected, + crate::tree::mirror::cbor::MAJOR_UINT, + u64::from(invalid.byte()), + ); + let error = + decode_exact(speaker, RunBudget::default(), &rejected).unwrap_err(); assert_eq!(error.origin, Origin::stream(speaker, stream)); assert!(matches!( error.kind, diff --git a/src/tree/mirror/streaming/remote/codec/tests/error_atlas.rs b/src/tree/mirror/streaming/remote/codec/tests/error_atlas.rs index eaf44cc30..b044c0f46 100644 --- a/src/tree/mirror/streaming/remote/codec/tests/error_atlas.rs +++ b/src/tree/mirror/streaming/remote/codec/tests/error_atlas.rs @@ -1,4 +1,15 @@ -//! Stable witnesses for every codec error reachable without resource exhaustion. +//! Stable witnesses for every frame-stream codec error reachable without +//! resource exhaustion. +//! +//! The scope is the frame stream's own taxonomies — the encode, decode, +//! and record-iteration errors the `describe_*` matches below inventory. +//! The codec's handshake-layer surface is witnessed where it lives: +//! `GreetingError` in greeting/tests.rs, beside the greeting reader; and +//! `ListingIssue`, which the frame decoder collapses into this taxonomy +//! (witnessed here as `QueryOutOfOrder` and `Malformed(part=QueryChildren)`), +//! carries its typed surface through the greeting, witnessed in the same +//! suite. Both hold exemption entries below so a witness landing here is +//! flagged for promotion. //! //! Coverage is enforced from both ends: every `describe_*` match below is //! wildcard-free, so a new error variant fails compilation until it is @@ -11,6 +22,7 @@ //! which a deleted variant satisfies trivially — so pruning a variant must //! prune its `EXEMPT_MARKERS` entry by hand. +use crate::message::{PayloadCodec, PayloadDepthLimit}; use std::{ error::Error, fmt::Write as _, @@ -25,9 +37,10 @@ use super::super::{ DecodeError, DecodeErrorKind, DecodeLeafError, DecodeSignalError, EncodeError, EncodeErrorKind, Flow, Frame, FrameWrite, LeafRunError, Reaction, RunBudget, Speaker, Stream, WireFrame, decode, decode_exact, encode, - frame::{LeafRun, QUERY_CHILD_LEN}, + frame::LeafRun, signal::{Signal, WireSignal}, }; +use crate::tree::mirror::cbor::{self, MAJOR_BSTR, MAJOR_TAG, TAG_CBOR_SEQUENCE}; use crate::{Version, message::Message, tree::typed::Hash}; use serde::Serialize; @@ -48,31 +61,54 @@ const WITNESS_MARKERS: &[&str] = &[ "kind: Truncated(missing=", "kind: QueryOutOfOrder(previous=", "kind: InvalidRun::Empty", - "kind: InvalidRun::TruncatedHeader(", + "kind: InvalidRun::Head(", + "kind: InvalidRun::NotARecord(", "kind: InvalidRun::TruncatedRecord(", "kind: OverbatchedRun(declared=", "kind: TrailingBytes(count=", + "kind: FrameShape(", + "kind: FrameArity(", + "kind: Malformed(part=", // DecodeLeafError (describe_leaf_kind). "kind: Record::Version(io=", "kind: Record::Message(io=", - // FramePart: every frame component must fail somewhere. These ride the - // encode Write witnesses; FramePart has no exhaustive match here, so a - // new component's marker must be added by hand alongside its witnesses. + // FramePart: every frame component must fail somewhere, whichever + // side witnesses it — the encode Write witnesses carry most parts, + // while Signal renders only from decode-side witnesses (the encoder + // never fails at the signal separately from the frame head). FramePart + // has no exhaustive match here, so a new component's marker must be + // added by hand alongside its witnesses. + "part=FrameHead", "part=Signal", - "part=QueryCount", "part=QueryChildren", "part=SupplyLength", "part=SupplyRun", ]; -/// Variants deliberately absent from the atlas, each with the reason it is -/// unreachable without resource exhaustion. -const EXEMPT_MARKERS: &[(&str, &str)] = &[( - "kind: SupplyTooLarge(", - "requires a run body past the u32 frame ceiling: a >4 GiB in-memory run \ - is resource exhaustion by construction; the ceiling itself is pinned at \ - its exact boundary in frame/tests.rs", -)]; +/// Variants deliberately absent from the atlas, each with the reason — +/// unreachable without resource exhaustion, or witnessed in another +/// layer's own suite. +const EXEMPT_MARKERS: &[(&str, &str)] = &[ + ( + "kind: SupplyTooLarge(", + "requires a run body past the wire's run byte cap: a >4 GiB in-memory \ + run is resource exhaustion by construction; the cap itself is pinned \ + at its exact boundary in frame/tests.rs", + ), + ( + "kind: Greeting", + "GreetingError is the handshake layer's surface, not a frame-stream \ + error: its variants are witnessed in greeting/tests.rs, beside the \ + greeting reader", + ), + ( + "kind: Listing", + "ListingIssue never surfaces from the frame decoders: they collapse \ + it into QueryOutOfOrder and Malformed(part=QueryChildren), both \ + witnessed here; its typed surface is the greeting's \ + (GreetingError::Listing), witnessed in greeting/tests.rs", + ), +]; /// Interior stream used where both speakers admit every signal state. const INTERIOR_STREAM: u8 = 8; @@ -88,7 +124,9 @@ fn one_record_run(version: Version, value: run } -/// Every feasible typed failure pins its origin, fields, and source chain. +/// Every feasible typed failure pins its fields and source chain, and its +/// origin where one exists (the record-level witnesses carry none; +/// `record_errors` says why). #[test] fn codec_error_atlas_snapshot() { insta::assert_snapshot!(build_atlas()); @@ -103,7 +141,8 @@ fn atlas_covers_every_error_variant() { assert!( atlas.contains(marker), "no atlas witness renders {marker:?}: add a witness for the \ - variant or an explicit exemption in EXEMPT_MARKERS", + variant, or move its marker out of WITNESS_MARKERS into a \ + reasoned EXEMPT_MARKERS entry", ); } for (marker, reason) in EXEMPT_MARKERS { @@ -142,12 +181,15 @@ fn encode_errors(atlas: &mut String) { ); for speaker in [Speaker::Initiator, Speaker::Responder] { + // Offsets in whole delivered bytes: the interior-stream query and + // supply frames open with a three-byte frame head (one array byte, + // a two-byte signal head), and the small supply run's own heads + // take three more. for (label, frame, offset) in [ - ("write/signal", &query, 0), - ("write/query-count", &query, 1), - ("write/query-children", &query, 2), - ("write/supply-length", &supply, 1), - ("write/supply-run", &supply, 5), + ("write/frame-head", &query, 0), + ("write/query-children", &query, 3), + ("write/supply-length", &supply, 3), + ("write/supply-run", &supply, 6), ] { let error = encode(speaker, frame, &mut FailAfterWriter::new(offset)).unwrap_err(); record_encode(atlas, &format!("{speaker:?}/{label}"), &error); @@ -188,24 +230,32 @@ fn decode_errors(atlas: &mut String) { ); for speaker in [Speaker::Initiator, Speaker::Responder] { + // The three-byte frame head and the small supply run's heads + // locate every read failure below. let error = decode( speaker, RunBudget::default(), &mut FailAfterReader::new(matched.clone(), 0), ) .unwrap_err(); + record_decode(atlas, &format!("{speaker:?}/read/frame-head"), &error); + + let error = decode( + speaker, + RunBudget::default(), + &mut FailAfterReader::new(matched.clone(), 1), + ) + .unwrap_err(); record_decode(atlas, &format!("{speaker:?}/read/signal"), &error); - for (label, offset) in [("query-count", 1), ("query-children", 2)] { - let error = decode( - speaker, - RunBudget::default(), - &mut FailAfterReader::new(query.clone(), offset), - ) - .unwrap_err(); - record_decode(atlas, &format!("{speaker:?}/read/{label}"), &error); - } - for (label, offset) in [("supply-length", 1), ("supply-run", 5)] { + let error = decode( + speaker, + RunBudget::default(), + &mut FailAfterReader::new(query.clone(), 3), + ) + .unwrap_err(); + record_decode(atlas, &format!("{speaker:?}/read/query-children"), &error); + for (label, offset) in [("supply-length", 3), ("supply-run", 6)] { let error = decode( speaker, RunBudget::default(), @@ -216,26 +266,75 @@ fn decode_errors(atlas: &mut String) { } for (label, bytes) in [ - ("signal", &[][..]), - ("query-count", &query[..1]), - ("query-children", &query[..2]), - ("supply-length", &supply[..1]), - ("supply-run", &supply[..5]), + ("frame-head", &[][..]), + ("signal", &matched[..1]), + ("query-children", &query[..3]), + ("supply-length", &supply[..4]), + ("supply-run", &supply[..6]), ] { let error = decode_exact(speaker, RunBudget::default(), bytes).unwrap_err(); record_decode(atlas, &format!("{speaker:?}/truncated/{label}"), &error); } - let error = - decode_exact(speaker, RunBudget::default(), &[FIRST_RESERVED_SIGNAL]).unwrap_err(); + let mut reserved = Vec::new(); + cbor::write_head(&mut reserved, cbor::MAJOR_ARRAY, 1); + cbor::write_head( + &mut reserved, + cbor::MAJOR_UINT, + u64::from(FIRST_RESERVED_SIGNAL), + ); + let error = decode_exact(speaker, RunBudget::default(), &reserved).unwrap_err(); record_decode(atlas, &format!("{speaker:?}/reserved-signal"), &error); - let mut unordered = query.clone(); - unordered[2] = 2; - unordered[2 + QUERY_CHILD_LEN] = 1; + // The frame item's own shape violations: a non-array item, an + // arity contradicting the signal, and non-canonical heads. + let error = decode_exact(speaker, RunBudget::default(), &[0x00]).unwrap_err(); + record_decode(atlas, &format!("{speaker:?}/frame/not-an-array"), &error); + + let mut mismatched = Vec::new(); + cbor::write_head(&mut mismatched, cbor::MAJOR_ARRAY, 2); + mismatched.extend_from_slice(&matched[1..]); + let error = decode_exact(speaker, RunBudget::default(), &mismatched).unwrap_err(); + record_decode(atlas, &format!("{speaker:?}/frame/arity"), &error); + + // The matched frame's signal is a one-byte head (a small code), + // so its code byte is the head itself; respell it widened. + let widened = [0x81, 0x19, 0x00, matched[1]]; + let error = decode_exact(speaker, RunBudget::default(), &widened).unwrap_err(); + record_decode(atlas, &format!("{speaker:?}/frame/widened-signal"), &error); + + let unordered = encoded( + speaker, + ( + stream, + Frame::Reaction( + Reaction::Query(vec![(2, Hash::default()), (1, Hash::default())]), + Flow::Continue, + ), + ), + ); let error = decode_exact(speaker, RunBudget::default(), &unordered).unwrap_err(); record_decode(atlas, &format!("{speaker:?}/query-out-of-order"), &error); + // A listing whose first key is a well-formed head of the wrong + // kind (a byte string where a radix belongs) reaches the listing + // gate and collapses into this taxonomy as + // Malformed(part=QueryChildren). + let listing_signal = WireSignal::new(speaker, stream, Signal::Query(Flow::Continue)) + .unwrap() + .to_byte(); + let mut defective_listing = Vec::new(); + cbor::write_head(&mut defective_listing, cbor::MAJOR_ARRAY, 2); + cbor::write_head( + &mut defective_listing, + cbor::MAJOR_UINT, + u64::from(listing_signal), + ); + cbor::write_head(&mut defective_listing, cbor::MAJOR_MAP, 1); + cbor::write_head(&mut defective_listing, MAJOR_BSTR, 0); + let error = decode_exact(speaker, RunBudget::default(), &defective_listing).unwrap_err(); + record_decode(atlas, &format!("{speaker:?}/query/listing-key"), &error); + let error = decode_exact( speaker, RunBudget::default(), @@ -250,9 +349,19 @@ fn decode_errors(atlas: &mut String) { &raw_supply(stream, Flow::Continue, &[0, 0]), ) .unwrap_err(); - record_decode(atlas, &format!("{speaker:?}/run/truncated-header"), &error); + record_decode(atlas, &format!("{speaker:?}/run/not-a-record"), &error); - let mut overrun = 2_u32.to_be_bytes().to_vec(); + let error = decode_exact( + speaker, + RunBudget::default(), + &raw_supply(stream, Flow::Continue, &[0xd8, 0x3f, 0x58, 0x01, 0x00]), + ) + .unwrap_err(); + record_decode(atlas, &format!("{speaker:?}/run/widened-head"), &error); + + let mut overrun = Vec::new(); + cbor::write_head(&mut overrun, MAJOR_TAG, TAG_CBOR_SEQUENCE); + cbor::write_head(&mut overrun, MAJOR_BSTR, 2); overrun.push(0); let error = decode_exact( speaker, @@ -289,7 +398,10 @@ fn decode_errors(atlas: &mut String) { for (label, speaker, stream, frame) in placement_witnesses() { let signal = frame_signal(&frame); let invalid = WireSignal::new(speaker, stream, signal).unwrap_err(); - let error = decode_exact(speaker, RunBudget::default(), &[invalid.byte()]).unwrap_err(); + let mut bytes = Vec::new(); + cbor::write_head(&mut bytes, cbor::MAJOR_ARRAY, 1); + cbor::write_head(&mut bytes, cbor::MAJOR_UINT, u64::from(invalid.byte())); + let error = decode_exact(speaker, RunBudget::default(), &bytes).unwrap_err(); record_decode(atlas, &format!("{label}/decode"), &error); } } @@ -302,13 +414,15 @@ fn decode_errors(atlas: &mut String) { fn record_errors(atlas: &mut String) { writeln!(atlas, "RECORD").unwrap(); - // A zero-length record is structurally valid; its empty body fails - // at the version decoder. + // An empty-content record is structurally valid; its missing + // version-atom tag fails at the version decoder. let run = LeafRun::from_encoded(framed_record(&[])).unwrap(); record_leaf(atlas, "record/version", &next_record_error(&run)); - // A record ending after its version fails at the message decoder. + // A record ending after its tagged version fails at the message + // decoder. let mut version = Vec::new(); + cbor::write_head(&mut version, MAJOR_TAG, crate::tags::VERSION_TAG); ciborium::ser::into_writer(&Version::new(), &mut version).unwrap(); let run = LeafRun::from_encoded(framed_record(&version)).unwrap(); record_leaf(atlas, "record/message", &next_record_error(&run)); @@ -322,16 +436,19 @@ fn record_errors(atlas: &mut String) { record_leaf(atlas, "record/trailing", &next_record_error(&run)); } -/// Frame one record body with its length header, as a run body. +/// Frame one record content behind its embedded-sequence heads, as a run +/// body. fn framed_record(record: &[u8]) -> Vec { - let mut body = (record.len() as u32).to_be_bytes().to_vec(); + let mut body = Vec::new(); + cbor::write_head(&mut body, MAJOR_TAG, TAG_CBOR_SEQUENCE); + cbor::write_head(&mut body, MAJOR_BSTR, record.len() as u64); body.extend_from_slice(record); body } /// The first record's decode failure from a structurally valid run. fn next_record_error(run: &LeafRun) -> DecodeLeafError { - run.records(Message::deserializer::()) + run.records(PayloadCodec::new::(PayloadDepthLimit::default())) .next() .expect("the run holds one record") .unwrap_err() @@ -393,8 +510,11 @@ fn raw_supply(stream: Stream, flow: Flow, body: &[u8]) -> Vec { let signal = WireSignal::new(Speaker::Initiator, stream, Signal::Supply(flow)) .unwrap() .to_byte(); - let mut encoded = vec![signal]; - encoded.extend_from_slice(&(body.len() as u32).to_be_bytes()); + let mut encoded = Vec::new(); + cbor::write_head(&mut encoded, cbor::MAJOR_ARRAY, 2); + cbor::write_head(&mut encoded, cbor::MAJOR_UINT, u64::from(signal)); + cbor::write_head(&mut encoded, MAJOR_TAG, TAG_CBOR_SEQUENCE); + cbor::write_head(&mut encoded, MAJOR_BSTR, body.len() as u64); encoded.extend_from_slice(body); encoded } @@ -482,8 +602,22 @@ fn describe_decode_kind(out: &mut String, kind: &DecodeErrorKind) { DecodeErrorKind::InvalidRun(LeafRunError::Empty) => { write!(out, "InvalidRun::Empty").unwrap() } - DecodeErrorKind::InvalidRun(LeafRunError::TruncatedHeader { remaining }) => { - write!(out, "InvalidRun::TruncatedHeader(remaining={remaining})").unwrap() + DecodeErrorKind::InvalidRun(LeafRunError::Head { remaining, source }) => write!( + out, + "InvalidRun::Head(remaining={remaining}, source={source})" + ) + .unwrap(), + DecodeErrorKind::InvalidRun(LeafRunError::NotARecord { remaining, detail }) => write!( + out, + "InvalidRun::NotARecord(remaining={remaining}, {detail})" + ) + .unwrap(), + DecodeErrorKind::FrameShape { detail } => write!(out, "FrameShape({detail})").unwrap(), + DecodeErrorKind::FrameArity { expected, found } => { + write!(out, "FrameArity(expected={expected}, found={found})").unwrap() + } + DecodeErrorKind::Malformed { part, detail } => { + write!(out, "Malformed(part={part:?}, {detail})").unwrap() } DecodeErrorKind::InvalidRun(LeafRunError::TruncatedRecord { len, remaining }) => write!( out, diff --git a/src/tree/mirror/streaming/remote/codec/tests/snapshots/rumors__tree__mirror__streaming__remote__codec__tests__error_atlas__codec_error_atlas_snapshot.snap b/src/tree/mirror/streaming/remote/codec/tests/snapshots/rumors__tree__mirror__streaming__remote__codec__tests__error_atlas__codec_error_atlas_snapshot.snap index 299c92125..1cb5b7070 100644 --- a/src/tree/mirror/streaming/remote/codec/tests/snapshots/rumors__tree__mirror__streaming__remote__codec__tests__error_atlas__codec_error_atlas_snapshot.snap +++ b/src/tree/mirror/streaming/remote/codec/tests/snapshots/rumors__tree__mirror__streaming__remote__codec__tests__error_atlas__codec_error_atlas_snapshot.snap @@ -3,17 +3,11 @@ source: src/tree/mirror/streaming/remote/codec/tests/error_atlas.rs expression: build_atlas() --- ENCODE - Initiator/write/signal - display: Initiator stream 8: could not write the frame's signal byte + Initiator/write/frame-head + display: Initiator stream 8: could not write the frame's frame head origin: Initiator stream 8 - kind: Write(part=Signal, io=Other) - source[0]: could not write the frame's signal byte - source[1]: Io(Other) - Initiator/write/query-count - display: Initiator stream 8: could not write the frame's query count - origin: Initiator stream 8 - kind: Write(part=QueryCount, io=Other) - source[0]: could not write the frame's query count + kind: Write(part=FrameHead, io=Other) + source[0]: could not write the frame's frame head source[1]: Io(Other) Initiator/write/query-children display: Initiator stream 8: could not write the frame's query child listing @@ -22,10 +16,10 @@ ENCODE source[0]: could not write the frame's query child listing source[1]: Io(Other) Initiator/write/supply-length - display: Initiator stream 8: could not write the frame's supply run length + display: Initiator stream 8: could not write the frame's supply run head origin: Initiator stream 8 kind: Write(part=SupplyLength, io=Other) - source[0]: could not write the frame's supply run length + source[0]: could not write the frame's supply run head source[1]: Io(Other) Initiator/write/supply-run display: Initiator stream 8: could not write the frame's supply run @@ -39,17 +33,11 @@ ENCODE kind: Flush(io=Other) source[0]: could not flush the completed frame source[1]: Io(Other) - Responder/write/signal - display: Responder stream 8: could not write the frame's signal byte - origin: Responder stream 8 - kind: Write(part=Signal, io=Other) - source[0]: could not write the frame's signal byte - source[1]: Io(Other) - Responder/write/query-count - display: Responder stream 8: could not write the frame's query count + Responder/write/frame-head + display: Responder stream 8: could not write the frame's frame head origin: Responder stream 8 - kind: Write(part=QueryCount, io=Other) - source[0]: could not write the frame's query count + kind: Write(part=FrameHead, io=Other) + source[0]: could not write the frame's frame head source[1]: Io(Other) Responder/write/query-children display: Responder stream 8: could not write the frame's query child listing @@ -58,10 +46,10 @@ ENCODE source[0]: could not write the frame's query child listing source[1]: Io(Other) Responder/write/supply-length - display: Responder stream 8: could not write the frame's supply run length + display: Responder stream 8: could not write the frame's supply run head origin: Responder stream 8 kind: Write(part=SupplyLength, io=Other) - source[0]: could not write the frame's supply run length + source[0]: could not write the frame's supply run head source[1]: Io(Other) Responder/write/supply-run display: Responder stream 8: could not write the frame's supply run @@ -76,17 +64,17 @@ ENCODE source[0]: could not flush the completed frame source[1]: Io(Other) DECODE + Initiator/read/frame-head + display: Initiator direction: could not read the frame's frame head + origin: Initiator direction + kind: Read(part=FrameHead, io=Other) + source[0]: could not read the frame's frame head + source[1]: Io(Other) Initiator/read/signal - display: Initiator direction: could not read the frame's signal byte + display: Initiator direction: could not read the frame's signal origin: Initiator direction kind: Read(part=Signal, io=Other) - source[0]: could not read the frame's signal byte - source[1]: Io(Other) - Initiator/read/query-count - display: Initiator stream 8: could not read the frame's query count - origin: Initiator stream 8 - kind: Read(part=QueryCount, io=Other) - source[0]: could not read the frame's query count + source[0]: could not read the frame's signal source[1]: Io(Other) Initiator/read/query-children display: Initiator stream 8: could not read the frame's query child listing @@ -95,10 +83,10 @@ DECODE source[0]: could not read the frame's query child listing source[1]: Io(Other) Initiator/read/supply-length - display: Initiator stream 8: could not read the frame's supply run length + display: Initiator stream 8: could not read the frame's supply run head origin: Initiator stream 8 kind: Read(part=SupplyLength, io=Other) - source[0]: could not read the frame's supply run length + source[0]: could not read the frame's supply run head source[1]: Io(Other) Initiator/read/supply-run display: Initiator stream 8: could not read the frame's supply run @@ -106,17 +94,17 @@ DECODE kind: Read(part=SupplyRun, io=Other) source[0]: could not read the frame's supply run source[1]: Io(Other) + Initiator/truncated/frame-head + display: Initiator direction: frame ended before its frame head + origin: Initiator direction + kind: Truncated(missing=FrameHead, io=UnexpectedEof) + source[0]: frame ended before its frame head + source[1]: Io(UnexpectedEof) Initiator/truncated/signal - display: Initiator direction: frame ended before its signal byte + display: Initiator direction: frame ended before its signal origin: Initiator direction kind: Truncated(missing=Signal, io=UnexpectedEof) - source[0]: frame ended before its signal byte - source[1]: Io(UnexpectedEof) - Initiator/truncated/query-count - display: Initiator stream 8: frame ended before its query count - origin: Initiator stream 8 - kind: Truncated(missing=QueryCount, io=UnexpectedEof) - source[0]: frame ended before its query count + source[0]: frame ended before its signal source[1]: Io(UnexpectedEof) Initiator/truncated/query-children display: Initiator stream 8: frame ended before its query child listing @@ -125,10 +113,10 @@ DECODE source[0]: frame ended before its query child listing source[1]: Io(UnexpectedEof) Initiator/truncated/supply-length - display: Initiator stream 8: frame ended before its supply run length + display: Initiator stream 8: frame ended before its supply run head origin: Initiator stream 8 kind: Truncated(missing=SupplyLength, io=UnexpectedEof) - source[0]: frame ended before its supply run length + source[0]: frame ended before its supply run head source[1]: Io(UnexpectedEof) Initiator/truncated/supply-run display: Initiator stream 8: frame ended before its supply run @@ -137,52 +125,78 @@ DECODE source[0]: frame ended before its supply run source[1]: Io(UnexpectedEof) Initiator/reserved-signal - display: Initiator stream 0: signal byte 0xaa encodes an invalid semantic state + display: Initiator stream 0: signal code 0xaa encodes an invalid semantic state origin: Initiator stream 0 kind: InvalidSignal::Reserved(byte=aa, state=10) - source[0]: signal byte 0xaa encodes an invalid semantic state + source[0]: signal code 0xaa encodes an invalid semantic state source[1]: semantic signal state 10 is outside the valid range + Initiator/frame/not-an-array + display: Initiator direction: frame is not a CBOR reaction array: frame item is not an array + origin: Initiator direction + kind: FrameShape(frame item is not an array) + source[0]: frame is not a CBOR reaction array: frame item is not an array + Initiator/frame/arity + display: Initiator stream 8: frame array carries 2 item(s) where its signal takes 1 + origin: Initiator stream 8 + kind: FrameArity(expected=1, found=2) + source[0]: frame array carries 2 item(s) where its signal takes 1 + Initiator/frame/widened-signal + display: Initiator direction: frame's signal is malformed: head not in shortest form + origin: Initiator direction + kind: Malformed(part=Signal, head not in shortest form) + source[0]: frame's signal is malformed: head not in shortest form Initiator/query-out-of-order display: Initiator stream 8: query child radix 1 does not follow 2 in ascending order origin: Initiator stream 8 kind: QueryOutOfOrder(previous=2, radix=1) source[0]: query child radix 1 does not follow 2 in ascending order + Initiator/query/listing-key + display: Initiator stream 8: frame's query child listing is malformed: listing key is not a radix + origin: Initiator stream 8 + kind: Malformed(part=QueryChildren, listing key is not a radix) + source[0]: frame's query child listing is malformed: listing key is not a radix Initiator/run/empty display: Initiator stream 8: a supply run carries no leaf records origin: Initiator stream 8 kind: InvalidRun::Empty source[0]: a supply run carries no leaf records - Initiator/run/truncated-header - display: Initiator stream 8: a leaf record header overruns the 2 bytes left in its run + Initiator/run/not-a-record + display: Initiator stream 8: a 2-byte run tail is not a leaf record: record does not open with the embedded-sequence tag + origin: Initiator stream 8 + kind: InvalidRun::NotARecord(remaining=2, record does not open with the embedded-sequence tag) + source[0]: a 2-byte run tail is not a leaf record: record does not open with the embedded-sequence tag + Initiator/run/widened-head + display: Initiator stream 8: a leaf record's heads are invalid in the 5 bytes left in its run: CBOR head is not in shortest form origin: Initiator stream 8 - kind: InvalidRun::TruncatedHeader(remaining=2) - source[0]: a leaf record header overruns the 2 bytes left in its run + kind: InvalidRun::Head(remaining=5, source=CBOR head is not in shortest form) + source[0]: a leaf record's heads are invalid in the 5 bytes left in its run: CBOR head is not in shortest form + source[1]: CBOR head is not in shortest form Initiator/run/truncated-record display: Initiator stream 8: a leaf record of 2 bytes overruns the 1 bytes left in its run origin: Initiator stream 8 kind: InvalidRun::TruncatedRecord(len=2, remaining=1) source[0]: a leaf record of 2 bytes overruns the 1 bytes left in its run Initiator/run/overbatched - display: Initiator stream 8: supply frame occupies 19 wire bytes, batching records past the 0-byte run budget + display: Initiator stream 8: supply frame charges 28 wire bytes, batching records past the 0-byte run budget origin: Initiator stream 8 - kind: OverbatchedRun(declared=19, budget=0) - source[0]: supply frame occupies 19 wire bytes, batching records past the 0-byte run budget + kind: OverbatchedRun(declared=28, budget=0) + source[0]: supply frame charges 28 wire bytes, batching records past the 0-byte run budget Initiator/frame/trailing display: Initiator stream 8: 1 trailing bytes follow the frame origin: Initiator stream 8 kind: TrailingBytes(count=1) source[0]: 1 trailing bytes follow the frame + Responder/read/frame-head + display: Responder direction: could not read the frame's frame head + origin: Responder direction + kind: Read(part=FrameHead, io=Other) + source[0]: could not read the frame's frame head + source[1]: Io(Other) Responder/read/signal - display: Responder direction: could not read the frame's signal byte + display: Responder direction: could not read the frame's signal origin: Responder direction kind: Read(part=Signal, io=Other) - source[0]: could not read the frame's signal byte - source[1]: Io(Other) - Responder/read/query-count - display: Responder stream 8: could not read the frame's query count - origin: Responder stream 8 - kind: Read(part=QueryCount, io=Other) - source[0]: could not read the frame's query count + source[0]: could not read the frame's signal source[1]: Io(Other) Responder/read/query-children display: Responder stream 8: could not read the frame's query child listing @@ -191,10 +205,10 @@ DECODE source[0]: could not read the frame's query child listing source[1]: Io(Other) Responder/read/supply-length - display: Responder stream 8: could not read the frame's supply run length + display: Responder stream 8: could not read the frame's supply run head origin: Responder stream 8 kind: Read(part=SupplyLength, io=Other) - source[0]: could not read the frame's supply run length + source[0]: could not read the frame's supply run head source[1]: Io(Other) Responder/read/supply-run display: Responder stream 8: could not read the frame's supply run @@ -202,17 +216,17 @@ DECODE kind: Read(part=SupplyRun, io=Other) source[0]: could not read the frame's supply run source[1]: Io(Other) + Responder/truncated/frame-head + display: Responder direction: frame ended before its frame head + origin: Responder direction + kind: Truncated(missing=FrameHead, io=UnexpectedEof) + source[0]: frame ended before its frame head + source[1]: Io(UnexpectedEof) Responder/truncated/signal - display: Responder direction: frame ended before its signal byte + display: Responder direction: frame ended before its signal origin: Responder direction kind: Truncated(missing=Signal, io=UnexpectedEof) - source[0]: frame ended before its signal byte - source[1]: Io(UnexpectedEof) - Responder/truncated/query-count - display: Responder stream 8: frame ended before its query count - origin: Responder stream 8 - kind: Truncated(missing=QueryCount, io=UnexpectedEof) - source[0]: frame ended before its query count + source[0]: frame ended before its signal source[1]: Io(UnexpectedEof) Responder/truncated/query-children display: Responder stream 8: frame ended before its query child listing @@ -221,10 +235,10 @@ DECODE source[0]: frame ended before its query child listing source[1]: Io(UnexpectedEof) Responder/truncated/supply-length - display: Responder stream 8: frame ended before its supply run length + display: Responder stream 8: frame ended before its supply run head origin: Responder stream 8 kind: Truncated(missing=SupplyLength, io=UnexpectedEof) - source[0]: frame ended before its supply run length + source[0]: frame ended before its supply run head source[1]: Io(UnexpectedEof) Responder/truncated/supply-run display: Responder stream 8: frame ended before its supply run @@ -233,56 +247,82 @@ DECODE source[0]: frame ended before its supply run source[1]: Io(UnexpectedEof) Responder/reserved-signal - display: Responder stream 0: signal byte 0xaa encodes an invalid semantic state + display: Responder stream 0: signal code 0xaa encodes an invalid semantic state origin: Responder stream 0 kind: InvalidSignal::Reserved(byte=aa, state=10) - source[0]: signal byte 0xaa encodes an invalid semantic state + source[0]: signal code 0xaa encodes an invalid semantic state source[1]: semantic signal state 10 is outside the valid range + Responder/frame/not-an-array + display: Responder direction: frame is not a CBOR reaction array: frame item is not an array + origin: Responder direction + kind: FrameShape(frame item is not an array) + source[0]: frame is not a CBOR reaction array: frame item is not an array + Responder/frame/arity + display: Responder stream 8: frame array carries 2 item(s) where its signal takes 1 + origin: Responder stream 8 + kind: FrameArity(expected=1, found=2) + source[0]: frame array carries 2 item(s) where its signal takes 1 + Responder/frame/widened-signal + display: Responder direction: frame's signal is malformed: head not in shortest form + origin: Responder direction + kind: Malformed(part=Signal, head not in shortest form) + source[0]: frame's signal is malformed: head not in shortest form Responder/query-out-of-order display: Responder stream 8: query child radix 1 does not follow 2 in ascending order origin: Responder stream 8 kind: QueryOutOfOrder(previous=2, radix=1) source[0]: query child radix 1 does not follow 2 in ascending order + Responder/query/listing-key + display: Responder stream 8: frame's query child listing is malformed: listing key is not a radix + origin: Responder stream 8 + kind: Malformed(part=QueryChildren, listing key is not a radix) + source[0]: frame's query child listing is malformed: listing key is not a radix Responder/run/empty display: Responder stream 8: a supply run carries no leaf records origin: Responder stream 8 kind: InvalidRun::Empty source[0]: a supply run carries no leaf records - Responder/run/truncated-header - display: Responder stream 8: a leaf record header overruns the 2 bytes left in its run + Responder/run/not-a-record + display: Responder stream 8: a 2-byte run tail is not a leaf record: record does not open with the embedded-sequence tag + origin: Responder stream 8 + kind: InvalidRun::NotARecord(remaining=2, record does not open with the embedded-sequence tag) + source[0]: a 2-byte run tail is not a leaf record: record does not open with the embedded-sequence tag + Responder/run/widened-head + display: Responder stream 8: a leaf record's heads are invalid in the 5 bytes left in its run: CBOR head is not in shortest form origin: Responder stream 8 - kind: InvalidRun::TruncatedHeader(remaining=2) - source[0]: a leaf record header overruns the 2 bytes left in its run + kind: InvalidRun::Head(remaining=5, source=CBOR head is not in shortest form) + source[0]: a leaf record's heads are invalid in the 5 bytes left in its run: CBOR head is not in shortest form + source[1]: CBOR head is not in shortest form Responder/run/truncated-record display: Responder stream 8: a leaf record of 2 bytes overruns the 1 bytes left in its run origin: Responder stream 8 kind: InvalidRun::TruncatedRecord(len=2, remaining=1) source[0]: a leaf record of 2 bytes overruns the 1 bytes left in its run Responder/run/overbatched - display: Responder stream 8: supply frame occupies 19 wire bytes, batching records past the 0-byte run budget + display: Responder stream 8: supply frame charges 28 wire bytes, batching records past the 0-byte run budget origin: Responder stream 8 - kind: OverbatchedRun(declared=19, budget=0) - source[0]: supply frame occupies 19 wire bytes, batching records past the 0-byte run budget + kind: OverbatchedRun(declared=28, budget=0) + source[0]: supply frame charges 28 wire bytes, batching records past the 0-byte run budget Responder/frame/trailing display: Responder stream 8: 1 trailing bytes follow the frame origin: Responder stream 8 kind: TrailingBytes(count=1) source[0]: 1 trailing bytes follow the frame placement/opening-supplies/decode - display: Initiator stream 0: signal byte 0x00 is invalid for the initiator's opening supplies + display: Initiator stream 0: signal code 0x00 is invalid for the initiator's opening supplies origin: Initiator stream 0 kind: InvalidSignal::Placement(byte=00, class=OpeningSupplies) - source[0]: signal byte 0x00 is invalid for the initiator's opening supplies + source[0]: signal code 0x00 is invalid for the initiator's opening supplies placement/leaf-parent/decode - display: Initiator stream 16: signal byte 0x54 is invalid for the initiator's leaf-parent replies + display: Initiator stream 16: signal code 0x54 is invalid for the initiator's leaf-parent replies origin: Initiator stream 16 kind: InvalidSignal::Placement(byte=54, class=LeafParentReplies) - source[0]: signal byte 0x54 is invalid for the initiator's leaf-parent replies + source[0]: signal code 0x54 is invalid for the initiator's leaf-parent replies placement/terminal-leaf/decode - display: Responder stream 16: signal byte 0x10 is invalid for the responder's terminal leaf replies + display: Responder stream 16: signal code 0x10 is invalid for the responder's terminal leaf replies origin: Responder stream 16 kind: InvalidSignal::Placement(byte=10, class=TerminalLeafReplies) - source[0]: signal byte 0x10 is invalid for the responder's terminal leaf replies + source[0]: signal code 0x10 is invalid for the responder's terminal leaf replies RECORD record/version display: supplied Version could not be decoded diff --git a/src/tree/mirror/streaming/remote/error.rs b/src/tree/mirror/streaming/remote/error.rs index 7ea061952..2826b4ef0 100644 --- a/src/tree/mirror/streaming/remote/error.rs +++ b/src/tree/mirror/streaming/remote/error.rs @@ -5,12 +5,14 @@ //! re-exported here so a caller can match a failure down to its precise cause //! without depending on the private implementation modules. -pub use super::adapter::{DecodeError, EncodeError, OpeningError, ScopeError}; +pub use super::adapter::{ + DecodeError as ReplyDecodeError, EncodeError as ReplyEncodeError, OpeningError, ScopeError, +}; pub use super::codec::{ DecodeError as CodecDecodeError, DecodeErrorKind as CodecDecodeErrorKind, DecodeLeafError, DecodeSignalError, EncodeError as CodecEncodeError, EncodeErrorKind as CodecEncodeErrorKind, - FramePart, InvalidSignalPlacement, InvalidWireSignal, LeafRunError, Origin, QueryOrderError, - Speaker, Stream, StreamClass, + FramePart, GreetingError, HeadError, InvalidSignalPlacement, InvalidWireSignal, LeafRunError, + ListingIssue, Origin, QueryOrderError, Speaker, Stream, StreamClass, }; pub use super::proxy::Error as RemoteError; pub use super::streams::{AcceptError, ReplyFrameError, SendError, StreamError}; diff --git a/src/tree/mirror/streaming/remote/proxy/error.rs b/src/tree/mirror/streaming/remote/proxy/error.rs index 31c7df77d..1c17c0668 100644 --- a/src/tree/mirror/streaming/remote/proxy/error.rs +++ b/src/tree/mirror/streaming/remote/proxy/error.rs @@ -1,5 +1,6 @@ //! Failures surfaced by the remote protocol participant. +use crate::message::PayloadDepthLimit; use crate::tree::mirror::streaming::remote::{adapter, codec, streams}; /// A protocol or adapter failure while proxying one remote counterparty. @@ -18,15 +19,26 @@ pub enum Error { /// Reading one of the peer's greeting frames failed. #[error("failed to read streaming handshake")] HandshakeRead(#[source] std::io::Error), - /// A greeting body was not a canonical causal version or listing. + /// The peer's greeting arrived but is not canonical rumors CBOR. #[error("failed to decode streaming handshake")] - HandshakeDecode(#[source] std::io::Error), + HandshakeDecode(#[source] codec::GreetingError), /// Writing and flushing the local greeting frames failed. #[error("failed to write streaming handshake")] HandshakeWrite(#[source] std::io::Error), /// The peer's greeting listing violated canonical ascending radix order. #[error("peer greeting carried a non-canonical root-fan listing")] HandshakeListing(#[source] codec::QueryOrderError), + /// The peer's configured payload depth limit differs from ours. + /// + /// Detected symmetrically, after the greetings and before anything + /// else, so a mixed fleet is caught even on a converged session. + #[error("peer's payload depth limit ({remote}) differs from ours ({local})")] + PayloadDepthMismatch { + /// This side's configured limit. + local: PayloadDepthLimit, + /// The limit the peer's greeting declared. + remote: PayloadDepthLimit, + }, /// The locally-produced distinguished opening could not be encoded. #[error("local opening reply is invalid")] OpeningEncode(#[source] adapter::OpeningError), diff --git a/src/tree/mirror/streaming/remote/proxy/start.rs b/src/tree/mirror/streaming/remote/proxy/start.rs index bf5b4f2b3..b7ee4f36c 100644 --- a/src/tree/mirror/streaming/remote/proxy/start.rs +++ b/src/tree/mirror/streaming/remote/proxy/start.rs @@ -1,35 +1,30 @@ //! The wire participant's protocol handshake states. -use crate::message::PayloadDeserializer; -use std::io; +use crate::message::{PayloadCodec, PayloadDepthLimit}; use tokio::io::{AsyncRead, AsyncWrite}; use crate::{ - Version, link::{Acceptor, Connector, Link}, + observe::{CaptureRead, Role, SessionHandle}, tree::{ - mirror::{ - framing, - streaming::{ - Backend, Leaf, - message::{Greeting, initiates}, - protocol::{self, Accept, CompleteConnect, Connect}, - remote::{ - codec::{RunBudget, Speaker, validate_children}, - proxy::{ - Connected, Error, - work::{Physical, Work}, - }, - streams::{AcceptDriver, claims, error_route}, + mirror::streaming::{ + Backend, Leaf, + message::{Greeting, initiates}, + protocol::{self, Accept, CompleteConnect, Connect}, + remote::{ + codec::{RunBudget, Speaker, greeting as greeting_codec}, + proxy::{ + Connected, Error, + work::{Physical, Work}, }, - stats::Recorder, - window::{Window, WindowConfig}, + streams::{AcceptDriver, claims, error_route}, }, + stats::Recorder, + window::{Window, WindowConfig}, }, typed::{ Hash, - hash::MERKLE_HASH_LEN, height::{Root, Z}, }, }, @@ -55,10 +50,12 @@ where /// The session's stats recorder: every stream this session binds /// counts its codec bytes through it. stats: Recorder, - /// The peer's payload deserializer: the typed ingress every supplied - /// leaf record decodes through (see - /// [`Message::deserializer`](crate::message::Message::deserializer)). - deserializer: PayloadDeserializer, + /// The peer's payload codec: the typed ingress every supplied + /// leaf record decodes through (see [`PayloadCodec`]). + codec: PayloadCodec, + /// The session's observation handle: every wire item this session + /// moves is delivered through it (inert unless a handler attached). + observe: SessionHandle, } impl Handshaking @@ -67,16 +64,17 @@ where { /// Bind one session's link carrier before exchanging causal versions. /// - /// `deserializer` is the peer's payload deserializer: every leaf + /// `codec` is the peer's payload codec: every leaf /// record this session decodes builds its payload through it. - pub fn start(backend: B, link: Link, deserializer: PayloadDeserializer) -> Self { + pub fn start(backend: B, link: Link, codec: PayloadCodec) -> Self { Self { backend, link, versions: Start, window: WindowConfig::default(), stats: Recorder::default(), - deserializer, + codec, + observe: SessionHandle::default(), } } @@ -96,6 +94,15 @@ where self.stats = stats; self } + + /// Share the session's observation handle, so the greeting exchange + /// and every stream this session binds deliver their wire items. + /// + /// Without this call the session runs with the inert default. + pub fn observe(mut self, observe: SessionHandle) -> Self { + self.observe = observe; + self + } } /// Handshake state before this participant has sent its version. @@ -132,7 +139,7 @@ where /// Receive the remote greeting before asking the local server to answer it. async fn connect(mut self) -> Result<(Greeting, Self::Next), Self::Error> { - let remote = receive::(&mut self.link.control_read).await?; + let remote = receive::(&mut self.link.control_read, &self.observe).await?; let greeting = remote.clone(); let next = Handshaking { backend: self.backend, @@ -140,7 +147,8 @@ where versions: Connecting { remote }, window: self.window, stats: self.stats, - deserializer: self.deserializer, + codec: self.codec, + observe: self.observe, }; Ok((greeting, next)) } @@ -157,8 +165,15 @@ where type Next = Connected; /// Send the local server's greeting, then open only if versions differ. - async fn complete_connect(mut self, theirs: Greeting) -> Result { - send::(&theirs, &mut self.link.control_write).await?; + async fn complete_connect(mut self, mut theirs: Greeting) -> Result { + // The wire value of the local limit is the codec's: the one + // configuration every parse of this session already runs under. + theirs.payload_depth_limit = self.codec.limit().get(); + send::(&theirs, &mut self.link.control_write, &self.observe).await?; + // Payload depth limits must be equal — checked after both + // greetings are in hand and before the equal-versions resolution, + // so a mixed configuration is caught even on a converged session. + payload_depth_limits_match::(&self.codec, &self.versions.remote)?; let window = self.window.resolve( theirs.set_len, self.versions.remote.set_len, @@ -175,7 +190,8 @@ where self.versions.remote, self.link, self.stats, - self.deserializer, + self.codec, + self.observe, )) } } @@ -191,10 +207,20 @@ where type Next = Connected; /// Exchange greetings concurrently, then open only if versions differ. - async fn accept(mut self, request: Greeting) -> Result<(Greeting, Self::Next), Self::Error> { - let send = send::(&request, &mut self.link.control_write); - let receive = receive::(&mut self.link.control_read); + async fn accept( + mut self, + mut request: Greeting, + ) -> Result<(Greeting, Self::Next), Self::Error> { + // The wire value of the local limit is the codec's: the one + // configuration every parse of this session already runs under. + request.payload_depth_limit = self.codec.limit().get(); + let send = send::(&request, &mut self.link.control_write, &self.observe); + let receive = receive::(&mut self.link.control_read, &self.observe); let (_, remote) = futures_util::future::try_join(send, receive).await?; + // Payload depth limits must be equal — checked after both + // greetings are in hand and before the equal-versions resolution, + // so a mixed configuration is caught even on a converged session. + payload_depth_limits_match::(&self.codec, &remote)?; let greeting = remote.clone(); let window = self.window.resolve( request.set_len, @@ -212,12 +238,35 @@ where remote, self.link, self.stats, - self.deserializer, + self.codec, + self.observe, ); Ok((greeting, next)) } } +/// Require the peer's declared payload depth limit to equal ours. +/// +/// The limit is a property of the shared set — every replica must be +/// able to hold and forward all content — so it is exchanged for +/// equality, never negotiated: negotiating down is unsound (a peer may +/// already hold messages deeper than a negotiated bound, which it would +/// then not be allowed to gossip), so any negotiation scheme merely +/// relocates the failure to mid-session, conditional on which leaves +/// differ. Both sides detect the mismatch symmetrically, like a network +/// mismatch. +fn payload_depth_limits_match(codec: &PayloadCodec, remote: &Greeting) -> Result<(), Error> { + let local = codec.limit(); + let declared = PayloadDepthLimit::new(remote.payload_depth_limit); + if declared != local { + return Err(Error::PayloadDepthMismatch { + local, + remote: declared, + }); + } + Ok(()) +} + /// Compute the session's supply-run budget. /// /// The budget is the smaller of the two greetings' targets, so each @@ -228,85 +277,58 @@ fn run_budget(ours: &Greeting, theirs: &Greeting) -> RunBudget { RunBudget::from_bytes(usize::try_from(bytes).unwrap_or(usize::MAX)) } -/// Send one greeting: the size-prefixed causal-version frame, then the -/// root-fan listing frame. +/// Send one greeting: a single self-delimiting control-stream item, +/// flushed in one hop. /// -/// The first frame's body is `set_len (8 B LE) ‖ max_version_bytes -/// (8 B LE) ‖ target_message_size (8 B LE) ‖ version`. Both frames flush -/// on the same hop; the listing frame is the wire carriage of the -/// opening question's content (see [`Greeting`] for the always-carry -/// trade). -async fn send(greeting: &Greeting, write: &mut W) -> Result<(), Error> +/// The spelling lives in +/// [`codec::greeting`](crate::tree::mirror::streaming::remote::codec::greeting). +/// The listing rides inside the item — the wire carriage of the opening +/// question's content (see [`Greeting`] for the always-carry trade). +async fn send( + greeting: &Greeting, + write: &mut W, + observe: &SessionHandle, +) -> Result<(), Error> where W: AsyncWrite + Unpin, { - let mut write = framing::FrameWrite::new(write); - let mut first = - Vec::with_capacity(framing::GREETING_SIZE_WORDS_LEN + greeting.version.as_bytes().len()); - first.extend_from_slice(&greeting.set_len.to_le_bytes()); - first.extend_from_slice(&greeting.max_version_bytes.to_le_bytes()); - first.extend_from_slice(&greeting.target_message_size.to_le_bytes()); - first.extend_from_slice(greeting.version.as_bytes()); - write.frame(&first).await.map_err(Error::HandshakeWrite)?; - // The listing frame is raw fixed-width records — radix byte, then the - // Merkle hash — with the frame length carrying the count, exactly the - // codec's query-listing shape. - let mut listing = Vec::with_capacity(greeting.listing.len() * (1 + MERKLE_HASH_LEN)); - for (radix, hash) in &greeting.listing { - listing.push(*radix); - listing.extend_from_slice(hash.as_bytes()); - } - write.frame(&listing).await.map_err(Error::HandshakeWrite) + use tokio::io::AsyncWriteExt as _; + let item = greeting_codec::encode_greeting(greeting); + write + .write_all(&item) + .await + .map_err(Error::HandshakeWrite)?; + write.flush().await.map_err(Error::HandshakeWrite)?; + observe.control_sent(&item); + Ok(()) } -/// Receive and canonically decode one greeting: the size-prefixed -/// causal-version frame, then the root-fan listing frame. +/// Receive and canonically decode one greeting item. /// -/// The listing is peer-controlled, so its canonical strictly-ascending radix -/// order is enforced here — the same rule the frame codec applies to a wire -/// query — before any scope is built from it. -async fn receive(read: &mut R) -> Result> +/// The greeting is peer-controlled, so its whole spelling is enforced +/// on ingress — deterministic heads, the exact key roster, and the +/// listing's canonical strictly-ascending radix order, the same rule the +/// frame codec applies to a wire query — before any scope is built +/// from it. +async fn receive(read: &mut R, observe: &SessionHandle) -> Result> where R: AsyncRead + Unpin, { - let mut read = framing::FrameRead::new(read); - let bytes = read.frame().await.map_err(Error::HandshakeRead)?; - let short = || { - Error::HandshakeDecode(io::Error::new( - io::ErrorKind::InvalidData, - "greeting version frame is shorter than its size prefixes", - )) + let route = |e| match e { + greeting_codec::ReadGreetingError::Io(io) => Error::HandshakeRead(io), + greeting_codec::ReadGreetingError::Decode(defect) => Error::HandshakeDecode(defect), + greeting_codec::ReadGreetingError::Listing(order) => Error::HandshakeListing(order), }; - let (set_len, max_version_bytes, target_message_size) = - framing::greeting_words(&bytes).ok_or_else(short)?; - let version = Version::decode(&bytes[framing::GREETING_SIZE_WORDS_LEN..]) - .map_err(|e| Error::HandshakeDecode(io::Error::new(io::ErrorKind::InvalidData, e)))?; - let bytes = read.frame().await.map_err(Error::HandshakeRead)?; - // The frame length carries the record count; a remainder is a - // malformed listing, not a short read. - if !bytes.len().is_multiple_of(1 + MERKLE_HASH_LEN) { - return Err(Error::HandshakeDecode(io::Error::new( - io::ErrorKind::InvalidData, - "listing frame is not a whole number of radix-hash records", - ))); + if observe.attached() { + let mut capture = CaptureRead::new(read); + let greeting = greeting_codec::read_greeting(&mut capture) + .await + .map_err(route)?; + observe.control_received(capture.bytes()); + Ok(greeting) + } else { + greeting_codec::read_greeting(read).await.map_err(route) } - let listing: Vec<(u8, Hash)> = bytes - .chunks_exact(1 + MERKLE_HASH_LEN) - .map(|record| { - let (&radix, hash) = record.split_first().expect("a record has a radix byte"); - let mut bytes = [0u8; MERKLE_HASH_LEN]; - bytes.copy_from_slice(hash); - (radix, Hash(bytes)) - }) - .collect(); - validate_children(&listing).map_err(Error::HandshakeListing)?; - Ok(Greeting { - version, - set_len, - max_version_bytes, - target_message_size, - listing, - }) } /// Return untouched control halves on equality, otherwise open the session. @@ -323,7 +345,8 @@ fn connected( remote: Greeting, link: Link, stats: Recorder, - deserializer: PayloadDeserializer, + codec: PayloadCodec, + observe: SessionHandle, ) -> Connected where B: Backend: Leaf>, @@ -345,6 +368,12 @@ where } else { Speaker::Responder }; + // The election is decided exactly here; observers learn it before + // any data stream can open. + observe.elected(match local { + Speaker::Initiator => Role::Initiator, + Speaker::Responder => Role::Responder, + }); open( backend, window, @@ -355,7 +384,8 @@ where remote.listing, link, stats, - deserializer, + codec, + observe, ) } @@ -379,7 +409,8 @@ fn open( peer_listing: Vec<(u8, Hash)>, link: Link, stats: Recorder, - deserializer: PayloadDeserializer, + codec: PayloadCodec, + observe: SessionHandle, ) -> Connected where B: Backend: Leaf>, @@ -412,9 +443,11 @@ where accept, errors, }, - deserializer, + codec, ); - Connected::new(remote, epoch, connector, claims, route, budget, stats, work) + Connected::new( + remote, epoch, connector, claims, route, budget, stats, observe, work, + ) } #[cfg(test)] diff --git a/src/tree/mirror/streaming/remote/proxy/start/tests.rs b/src/tree/mirror/streaming/remote/proxy/start/tests.rs index 8e3e0211e..0373d3e62 100644 --- a/src/tree/mirror/streaming/remote/proxy/start/tests.rs +++ b/src/tree/mirror/streaming/remote/proxy/start/tests.rs @@ -1,16 +1,16 @@ //! Ingress validation of the control-stream greeting decoder. //! -//! The greeting's frames are peer-controlled bytes arriving on the control -//! stream — first the causal-version frame, then the root-fan listing frame, -//! whose structural validation lives in [`receive`]: the same canonical-order -//! rule the frame codec applies to a wire query, applied at the greeting -//! ingress. +//! The greeting is one peer-controlled item arriving on the control +//! stream — the embedded-item tag wrapping a byte string of the greeting +//! map — whose structural validation lives in [`receive`]: deterministic +//! heads, the exact key roster, and the same canonical-order rule the +//! frame codec applies to a wire query, applied at the greeting ingress. //! //! The scripted-fault harness wraps only data streams, so this ingress is //! exercised here directly: crafted control-stream bytes must surface the //! typed greeting errors ([`Error::HandshakeRead`] for truncation and //! length lies, [`Error::HandshakeListing`] for canonical-order violations, -//! [`Error::HandshakeDecode`] for malformed bodies), never a panic, and a +//! [`Error::HandshakeDecode`] for malformed items), never a panic, and a //! canonical greeting must decode intact. use std::convert::Infallible; @@ -20,87 +20,79 @@ use proptest::prelude::*; use super::{Error, Greeting, receive}; use crate::Version; +use crate::observe::SessionHandle; use crate::tree::arb::nth_party; +use crate::tree::mirror::cbor::{self, HeadError, MAJOR_BSTR, TAG_EMBEDDED_ITEM}; use crate::tree::mirror::streaming::remote::codec::QueryOrderError; +use crate::tree::mirror::streaming::remote::codec::greeting::{GreetingError, encode_greeting}; use crate::tree::typed::Hash; -/// Length-delimit one frame body exactly as [`super::send`] does. -fn frame(body: &[u8]) -> Vec { - let len = u32::try_from(body.len()).expect("test frame bodies fit in u32"); - let mut bytes = len.to_be_bytes().to_vec(); - bytes.extend_from_slice(body); +/// Wrap raw content exactly as the greeting item does: the embedded-item +/// tag, then a byte string of the content. +fn raw_item(content: &[u8]) -> Vec { + let mut bytes = Vec::new(); + cbor::write_tag(&mut bytes, TAG_EMBEDDED_ITEM); + cbor::write_head(&mut bytes, MAJOR_BSTR, content.len() as u64); + bytes.extend_from_slice(content); bytes } -/// A version frame's body: the sender's set-size, version-size-bound, -/// and message-size-target prefixes, then the version. -fn version_body(version: &Version) -> Vec { - let mut body = 0_u64.to_le_bytes().to_vec(); - body.extend_from_slice(&0_u64.to_le_bytes()); - body.extend_from_slice(&0_u64.to_le_bytes()); - body.extend_from_slice(version.as_bytes()); - body -} - -/// A full greeting: the identity-version frame, then `listing_body` framed. -fn greeting(listing_body: &[u8]) -> Vec { - let mut bytes = frame(&version_body(&Version::new())); - bytes.extend_from_slice(&frame(listing_body)); - bytes +/// A greeting whose sizes are zero and whose listing is caller-selected; +/// the encoder trusts its caller, so a non-canonical listing synthesizes +/// wire violations directly. +fn greeting(listing: Vec<(u8, Hash)>) -> Vec { + encode_greeting(&Greeting { + version: Version::new(), + set_len: 0, + max_version_bytes: 0, + payload_depth_limit: 0, + target_message_size: 0, + listing, + }) } /// Decode crafted greeting bytes through the production ingress. async fn receive_greeting(bytes: &[u8]) -> Result> { - receive(&mut &bytes[..]).await + receive(&mut &bytes[..], &SessionHandle::default()).await } -/// A nonempty causal version, so truncating its encoding leaves bytes to cut. -fn ticked_version() -> Version { - let party = nth_party(0); - let mut version = Version::new(); - version.tick(&party); - version -} - -/// Encode a root-fan listing as its wire form: raw radix-hash records, -/// the frame length carrying the count. -fn encode_listing(children: &[(u8, Hash)]) -> Vec { - let mut body = Vec::new(); - for (radix, hash) in children { - body.push(*radix); - body.extend_from_slice(hash.as_bytes()); - } - body +/// The map content behind a greeting item's heads. +fn content_of(item: &[u8]) -> Vec { + let mut input = item; + cbor::read_head(&mut input).expect("the item's tag head"); + cbor::read_head(&mut input).expect("the item's string head"); + input.to_vec() } -/// A greeting cut inside the version frame's length header fails as a typed -/// read error. +/// A greeting cut inside the item's heads fails as a typed read error. /// -/// The four header bytes are the first peer-controlled bytes of the -/// greeting; a peer that closes mid-header must surface +/// The tag and byte-string heads are the first peer-controlled bytes of +/// the greeting; a peer that closes mid-head must surface /// [`Error::HandshakeRead`] with `UnexpectedEof` — never a hang waiting on /// bytes that cannot arrive. #[pollster::test] async fn truncated_version_header_is_a_typed_read_error() { - let result = receive_greeting(&[0, 0]).await.map(|_| ()); + let result = receive_greeting(&[0xd8]).await.map(|_| ()); match result { Err(Error::HandshakeRead(error)) => { assert_eq!(error.kind(), std::io::ErrorKind::UnexpectedEof) } - other => panic!("expected the truncated header's typed rejection, got {other:?}"), + other => panic!("expected the truncated head's typed rejection, got {other:?}"), } } -/// A version frame declaring more bytes than the stream carries fails as a +/// A greeting item declaring more bytes than the stream carries fails as a /// typed read error. /// -/// An over-declared length header makes the frame's exact read run off the -/// end of the peer's bytes; the lie must surface [`Error::HandshakeRead`] -/// with `UnexpectedEof`, never a partially filled frame handed to the -/// decoder. +/// An over-declared byte-string head makes the item's exact read run off +/// the end of the peer's bytes; the lie must surface +/// [`Error::HandshakeRead`] with `UnexpectedEof`, never a partially filled +/// item handed to the decoder. #[pollster::test] async fn over_declared_version_frame_is_a_typed_read_error() { - let mut bytes = 8_u32.to_be_bytes().to_vec(); + let mut bytes = Vec::new(); + cbor::write_tag(&mut bytes, TAG_EMBEDDED_ITEM); + cbor::write_head(&mut bytes, MAJOR_BSTR, 8); bytes.extend_from_slice(&[1, 2, 3]); let result = receive_greeting(&bytes).await.map(|_| ()); @@ -108,99 +100,99 @@ async fn over_declared_version_frame_is_a_typed_read_error() { Err(Error::HandshakeRead(error)) => { assert_eq!(error.kind(), std::io::ErrorKind::UnexpectedEof) } - other => panic!("expected the over-declared frame's typed rejection, got {other:?}"), + other => panic!("expected the over-declared item's typed rejection, got {other:?}"), } } -/// A zero-length version frame fails as a typed decode error. +/// An empty greeting item fails as a typed decode error. /// -/// A frame whose declared length is zero carries neither the set-size -/// prefix nor a version; the empty body must surface -/// [`Error::HandshakeDecode`] — the under-declared degenerate case, -/// distinct from the transport-level truncations above. +/// An item whose byte string is empty carries no map at all; it must +/// surface [`Error::HandshakeDecode`] with the head defect (the map's +/// content ends before its opening head) — the under-declared degenerate +/// case, distinct from the transport-level truncations above. #[pollster::test] async fn empty_version_frame_is_a_typed_decode_error() { - let result = receive_greeting(&frame(&[])).await.map(|_| ()); + let result = receive_greeting(&raw_item(&[])).await.map(|_| ()); assert!( - matches!(result, Err(Error::HandshakeDecode(_))), - "expected the empty version body's typed rejection, got {result:?}", + matches!( + result, + Err(Error::HandshakeDecode(GreetingError::Head( + HeadError::Truncated + ))), + ), + "expected the empty item's typed rejection, got {result:?}", ); } -/// A version body truncated inside an honestly sized frame fails as a typed -/// decode error. +/// A control stream opening with anything but the embedded-item tag fails +/// as a typed decode error. /// -/// The frame is well-formed — its header matches its body — but the body is -/// a strict prefix of a canonical version encoding, so the decoder runs out -/// of bits: [`Error::HandshakeDecode`], never a panic and never a shorter -/// version silently accepted. +/// The tag is the greeting's identity on the wire: a bare map (however +/// well-formed inside) is not the greeting's one spelling. #[pollster::test] -async fn truncated_version_body_is_a_typed_decode_error() { - let mut body = version_body(&ticked_version()); - body.truncate(body.len() - 1); +async fn untagged_greeting_is_a_typed_decode_error() { + let item = greeting(Vec::new()); + let content = content_of(&item); - let result = receive_greeting(&frame(&body)).await.map(|_| ()); + let result = receive_greeting(&content).await.map(|_| ()); assert!( - matches!(result, Err(Error::HandshakeDecode(_))), - "expected the truncated version body's typed rejection, got {result:?}", + matches!(result, Err(Error::HandshakeDecode(GreetingError::Shape(_))),), + "expected the untagged item's typed rejection, got {result:?}", ); } -/// A version frame with bytes after the version fails as a typed decode -/// error. +/// A greeting item with bytes after its map fails as a typed decode error. /// -/// The version encoding is prefix-free and the greeting decode is -/// canonical: the frame must contain exactly one version, so trailing bytes -/// surface [`Error::HandshakeDecode`] rather than being silently dropped -/// (which would let two encodings name one greeting). +/// The greeting decode is canonical: the item must contain exactly one +/// map, so trailing bytes surface [`Error::HandshakeDecode`] rather than +/// being silently dropped (which would let two encodings name one +/// greeting). #[pollster::test] async fn trailing_version_bytes_are_rejected() { - let mut body = version_body(&ticked_version()); - body.push(0xFF); + let item = greeting(Vec::new()); + let mut content = content_of(&item); + content.push(0xFF); - let result = receive_greeting(&frame(&body)).await.map(|_| ()); + let result = receive_greeting(&raw_item(&content)).await.map(|_| ()); assert!( - matches!(result, Err(Error::HandshakeDecode(_))), + matches!(result, Err(Error::HandshakeDecode(GreetingError::Shape(_))),), "expected the trailing bytes' typed rejection, got {result:?}", ); } -/// A greeting that ends after the version frame fails as a typed read error. +/// A greeting whose stream ends inside the item's content fails as a +/// typed read error. /// -/// The listing frame is not optional: a peer that sends its version and -/// closes must surface [`Error::HandshakeRead`] with `UnexpectedEof` on the -/// missing listing, never a greeting with a defaulted listing. +/// The byte-string head promised more content than arrived: the exact +/// read runs off the stream's end, a transport-level truncation. #[pollster::test] async fn missing_listing_frame_is_a_typed_read_error() { - let bytes = frame(&version_body(&Version::new())); + let item = greeting(Vec::new()); + let bytes = &item[..item.len() - 1]; - let result = receive_greeting(&bytes).await.map(|_| ()); + let result = receive_greeting(bytes).await.map(|_| ()); match result { Err(Error::HandshakeRead(error)) => { assert_eq!(error.kind(), std::io::ErrorKind::UnexpectedEof) } - other => panic!("expected the missing listing's typed rejection, got {other:?}"), + other => panic!("expected the cut content's typed rejection, got {other:?}"), } } proptest! { - /// Arbitrary greeting bodies decode to a greeting or a typed error, - /// never a panic. + /// Arbitrary greeting item contents decode to a greeting or a typed + /// error, never a panic. /// - /// Both frames are honestly sized around arbitrary bodies, so the fuzz - /// lands on the body decoders (the version's bit codec, the listing's - /// record shape and order check) rather than on the allocator via a lied - /// length header — the header lies are pinned deterministically above. - /// Every outcome must be `Ok` or one of the three typed greeting - /// errors. + /// The item is honestly sized around arbitrary content, so the fuzz + /// lands on the map decoder (heads, key roster, version atom, listing + /// shape and order) rather than on the allocator via a lied length — + /// the head lies are pinned deterministically above. Every outcome + /// must be `Ok` or one of the three typed greeting errors. #[test] fn arbitrary_greeting_bodies_never_panic( - version_body in vec(any::(), 0..64), - listing_body in vec(any::(), 0..64), + content in vec(any::(), 0..96), ) { - let mut bytes = frame(&version_body); - bytes.extend_from_slice(&frame(&listing_body)); - + let bytes = raw_item(&content); let result = pollster::block_on(receive_greeting(&bytes)).map(|_| ()); prop_assert!(matches!( result, @@ -219,10 +211,9 @@ proptest! { /// with the exact violating pair, before any scope is built from it. #[pollster::test] async fn unordered_listing_is_rejected() { - let listing = vec![(2_u8, Hash::default()), (1_u8, Hash::default())]; - let body = encode_listing(&listing); + let item = greeting(vec![(2_u8, Hash::default()), (1_u8, Hash::default())]); - let result = receive_greeting(&greeting(&body)).await.map(|_| ()); + let result = receive_greeting(&item).await.map(|_| ()); assert!( matches!( result, @@ -242,10 +233,9 @@ async fn unordered_listing_is_rejected() { /// with both offending radixes reported. #[pollster::test] async fn duplicate_listing_radix_is_rejected() { - let listing = vec![(3_u8, Hash::default()), (3_u8, Hash::default())]; - let body = encode_listing(&listing); + let item = greeting(vec![(3_u8, Hash::default()), (3_u8, Hash::default())]); - let result = receive_greeting(&greeting(&body)).await.map(|_| ()); + let result = receive_greeting(&item).await.map(|_| ()); assert!( matches!( result, @@ -258,42 +248,26 @@ async fn duplicate_listing_radix_is_rejected() { ); } -/// A listing frame whose record body is truncated fails as a typed decode -/// error. +/// A greeting map whose content is cut short fails as a typed decode error. /// -/// A frame declaring more listing entries than its body carries must surface -/// [`Error::HandshakeDecode`] — a typed greeting failure, never a panic and -/// never a partial listing. +/// The greeting's byte string ends inside the map's final entry; the cut +/// must surface [`Error::HandshakeDecode`] with the head defect — a typed +/// greeting failure, never a panic and never a partially parsed greeting. #[pollster::test] -async fn truncated_listing_body_is_rejected() { - let listing = vec![(0_u8, Hash::default()), (1_u8, Hash::default())]; - let mut body = encode_listing(&listing); - body.truncate(body.len() - 1); +async fn truncated_map_content_is_rejected() { + let item = greeting(vec![(0_u8, Hash::default()), (1_u8, Hash::default())]); + let mut content = content_of(&item); + content.truncate(content.len() - 1); - let result = receive_greeting(&greeting(&body)).await.map(|_| ()); + let result = receive_greeting(&raw_item(&content)).await.map(|_| ()); assert!( - matches!(result, Err(Error::HandshakeDecode(_))), - "expected the truncated body's typed rejection, got {result:?}", - ); -} - -/// A listing frame with bytes after the listing fails as a typed decode -/// error. -/// -/// The greeting decode is canonical: the frame must be a whole number of -/// radix-hash records, so trailing garbage surfaces -/// [`Error::HandshakeDecode`] rather than being silently ignored (which -/// would let two encodings name one greeting). -#[pollster::test] -async fn trailing_listing_bytes_are_rejected() { - let listing: Vec<(u8, Hash)> = Vec::new(); - let mut body = encode_listing(&listing); - body.push(0xFF); - - let result = receive_greeting(&greeting(&body)).await.map(|_| ()); - assert!( - matches!(result, Err(Error::HandshakeDecode(_))), - "expected the trailing bytes' typed rejection, got {result:?}", + matches!( + result, + Err(Error::HandshakeDecode(GreetingError::Head( + HeadError::Truncated + ))), + ), + "expected the truncated map's typed rejection, got {result:?}", ); } @@ -301,16 +275,27 @@ async fn trailing_listing_bytes_are_rejected() { /// /// The empty listing is a legal greeting — an empty tree's root fan — and /// the validation path must pass it through: the decoded handshake carries -/// the sent version and the empty listing, exercising the success arm of the +/// the sent fields and the empty listing, exercising the success arm of the /// same ingress the rejection tests pin. #[pollster::test] async fn empty_listing_greeting_decodes() { - let listing: Vec<(u8, Hash)> = Vec::new(); - let body = encode_listing(&listing); + let mut version = Version::new(); + version.tick(&nth_party(0)); + let item = encode_greeting(&Greeting { + version: version.clone(), + set_len: 7, + max_version_bytes: 512, + payload_depth_limit: 256, + target_message_size: 1 << 16, + listing: Vec::new(), + }); - let handshake = receive_greeting(&greeting(&body)) + let handshake = receive_greeting(&item) .await .expect("a canonical empty-listing greeting decodes"); - assert_eq!(handshake.version, Version::new()); + assert_eq!(handshake.version, version); + assert_eq!(handshake.set_len, 7); + assert_eq!(handshake.max_version_bytes, 512); + assert_eq!(handshake.target_message_size, 1 << 16); assert!(handshake.listing.is_empty()); } diff --git a/src/tree/mirror/streaming/remote/proxy/state.rs b/src/tree/mirror/streaming/remote/proxy/state.rs index 7d51a6601..c79223d72 100644 --- a/src/tree/mirror/streaming/remote/proxy/state.rs +++ b/src/tree/mirror/streaming/remote/proxy/state.rs @@ -10,6 +10,7 @@ use std::marker::PhantomData; use crate::link::{Acceptor, Connector}; +use crate::observe::SessionHandle; use crate::tree::{ mirror::streaming::{ Backend, Leaf, @@ -44,6 +45,9 @@ where /// The session's stats recorder, handed to every stream this session /// binds so the codec seam's byte counts accumulate in one place. stats: Recorder, + /// The session's observation handle, handed to every stream this + /// session binds so each can create its own observer when it opens. + observe: SessionHandle, work: Work, } @@ -63,6 +67,7 @@ where self.budget, self.route.clone(), self.stats.clone(), + self.observe.clone(), ) } @@ -75,6 +80,7 @@ where local, stream_at::(local), self.stats.clone(), + self.observe.clone(), ) } } @@ -119,6 +125,7 @@ where route: ErrorRoute, budget: RunBudget, stats: Recorder, + observe: SessionHandle, work: Work, ) -> Self { Self { @@ -130,6 +137,7 @@ where route, budget, stats, + observe, work, })), } diff --git a/src/tree/mirror/streaming/remote/proxy/tests.rs b/src/tree/mirror/streaming/remote/proxy/tests.rs index ec4dc5f0d..6aab4a867 100644 --- a/src/tree/mirror/streaming/remote/proxy/tests.rs +++ b/src/tree/mirror/streaming/remote/proxy/tests.rs @@ -1,5 +1,7 @@ //! End-to-end sessions between materialized peers and protocol-start proxies. +use crate::message::{PayloadCodec, PayloadDepthLimit}; +use serde::Serialize; use serde::de::DeserializeOwned; use std::convert::Infallible; use std::sync::Arc; @@ -10,6 +12,7 @@ use proptest::collection::vec; use proptest::prelude::*; use crate::link::memory_with_capacity; +use crate::observe::SessionHandle; use crate::testing::{ IoPlan, IoReportHandle, IoSide, Quiescence, reorder_accepts, run_to_quiescence, wrap_link, }; @@ -56,10 +59,18 @@ async fn reconcile(a: TreeRoot, b: TreeRoot) -> (TreeRoot, TreeRoot) { let b = Handshaking::start(Local, Root::::from(b)).window(WindowConfig::FLOOR); let (a_link, b_link) = memory_with_capacity(TRANSPORT_CAPACITY); - let remote_b = RemoteHandshaking::start(Local, a_link, Message::deserializer::<()>()) - .window(WindowConfig::FLOOR); - let remote_a = RemoteHandshaking::start(Local, b_link, Message::deserializer::<()>()) - .window(WindowConfig::FLOOR); + let remote_b = RemoteHandshaking::start( + Local, + a_link, + PayloadCodec::new::<()>(PayloadDepthLimit::default()), + ) + .window(WindowConfig::FLOOR); + let remote_a = RemoteHandshaking::start( + Local, + b_link, + PayloadCodec::new::<()>(PayloadDepthLimit::default()), + ) + .window(WindowConfig::FLOOR); let (a, b) = join!(Box::pin(mirror(a, remote_b)), Box::pin(mirror(remote_a, b))); let (a, _control) = a.expect("endpoint A should reconcile through its proxy"); @@ -75,15 +86,23 @@ async fn reconcile_symmetric_accepts( transport_capacity: usize, ) -> (TreeRoot, TreeRoot) where - T: DeserializeOwned + Send + Sync + 'static, + T: Serialize + DeserializeOwned + Eq + Send + Sync + 'static, { let a = Handshaking::start(Local, Root::::from(a)).window(WindowConfig::FLOOR); let b = Handshaking::start(Local, Root::::from(b)).window(WindowConfig::FLOOR); let (a_link, b_link) = memory_with_capacity(transport_capacity); - let remote_b = RemoteHandshaking::start(Local, a_link, Message::deserializer::()) - .window(WindowConfig::FLOOR); - let remote_a = RemoteHandshaking::start(Local, b_link, Message::deserializer::()) - .window(WindowConfig::FLOOR); + let remote_b = RemoteHandshaking::start( + Local, + a_link, + PayloadCodec::new::(PayloadDepthLimit::default()), + ) + .window(WindowConfig::FLOOR); + let remote_a = RemoteHandshaking::start( + Local, + b_link, + PayloadCodec::new::(PayloadDepthLimit::default()), + ) + .window(WindowConfig::FLOOR); let (a, b) = join!(Box::pin(mirror(a, remote_b)), Box::pin(mirror(b, remote_a)),); let (a, _control) = a.expect("endpoint A should reconcile through its proxy"); @@ -108,17 +127,25 @@ async fn reconcile_symmetric_accepts_reordered( reordered: Arc, ) -> (TreeRoot, TreeRoot) where - T: DeserializeOwned + Send + Sync + 'static, + T: Serialize + DeserializeOwned + Eq + Send + Sync + 'static, { let a = Handshaking::start(Local, Root::::from(a)).window(WindowConfig::FLOOR); let b = Handshaking::start(Local, Root::::from(b)).window(WindowConfig::FLOOR); let (a_link, b_link) = memory_with_capacity(transport_capacity); let a_link = reorder_accepts(a_link, REORDER_BATCH, reordered.clone()); let b_link = reorder_accepts(b_link, REORDER_BATCH, reordered); - let remote_b = RemoteHandshaking::start(Local, a_link, Message::deserializer::()) - .window(WindowConfig::FLOOR); - let remote_a = RemoteHandshaking::start(Local, b_link, Message::deserializer::()) - .window(WindowConfig::FLOOR); + let remote_b = RemoteHandshaking::start( + Local, + a_link, + PayloadCodec::new::(PayloadDepthLimit::default()), + ) + .window(WindowConfig::FLOOR); + let remote_a = RemoteHandshaking::start( + Local, + b_link, + PayloadCodec::new::(PayloadDepthLimit::default()), + ) + .window(WindowConfig::FLOOR); let (a, b) = join!(Box::pin(mirror(a, remote_b)), Box::pin(mirror(b, remote_a)),); let (a, _control) = a.expect("endpoint A should reconcile through its proxy"); @@ -130,14 +157,15 @@ where /// transport halves, proving that neither phase consumes the other's bytes. async fn reconcile_after_preamble(a: TreeRoot, b: TreeRoot) -> (TreeRoot, TreeRoot) where - T: DeserializeOwned + Send + Sync + 'static, + T: Serialize + DeserializeOwned + Eq + Send + Sync + 'static, { let a = Handshaking::start(Local, Root::::from(a)).window(WindowConfig::FLOOR); let b = Handshaking::start(Local, Root::::from(b)).window(WindowConfig::FLOOR); let (mut a_link, mut b_link) = memory_with_capacity(64 * 1024); let network = crate::Network::from_bytes([1; 16]); - let mut a_staged = handshake::Staged::new(); - let mut b_staged = handshake::Staged::new(); + let mut a_staged = handshake::Staged::new(crate::Protocol::V2); + let mut b_staged = handshake::Staged::new(crate::Protocol::V2); + let observe = SessionHandle::default(); let (seen_a, seen_b) = join!( handshake::preamble( crate::Protocol::V2, @@ -146,6 +174,7 @@ where &mut a_staged, &mut a_link.control_read, &mut a_link.control_write, + &observe ), handshake::preamble( crate::Protocol::V2, @@ -154,15 +183,24 @@ where &mut b_staged, &mut b_link.control_read, &mut b_link.control_write, + &observe ), ); seen_a.expect("A preamble"); seen_b.expect("B preamble"); - let remote_b = RemoteHandshaking::start(Local, a_link, Message::deserializer::()) - .window(WindowConfig::FLOOR); - let remote_a = RemoteHandshaking::start(Local, b_link, Message::deserializer::()) - .window(WindowConfig::FLOOR); + let remote_b = RemoteHandshaking::start( + Local, + a_link, + PayloadCodec::new::(PayloadDepthLimit::default()), + ) + .window(WindowConfig::FLOOR); + let remote_a = RemoteHandshaking::start( + Local, + b_link, + PayloadCodec::new::(PayloadDepthLimit::default()), + ) + .window(WindowConfig::FLOOR); let (a, b) = join!(Box::pin(mirror(a, remote_b)), Box::pin(mirror(b, remote_a)),); let (a, _control) = a.expect("endpoint A should reconcile through its proxy"); let (b, _control) = b.expect("endpoint B should reconcile through its proxy"); @@ -244,10 +282,18 @@ async fn reconcile_with_stacked_failures( } else { failing }; - let remote_b = RemoteHandshaking::start(left_backend, a_link, Message::deserializer::<()>()) - .window(WindowConfig::FLOOR); - let remote_a = RemoteHandshaking::start(right_backend, b_link, Message::deserializer::<()>()) - .window(WindowConfig::FLOOR); + let remote_b = RemoteHandshaking::start( + left_backend, + a_link, + PayloadCodec::new::<()>(PayloadDepthLimit::default()), + ) + .window(WindowConfig::FLOOR); + let remote_a = RemoteHandshaking::start( + right_backend, + b_link, + PayloadCodec::new::<()>(PayloadDepthLimit::default()), + ) + .window(WindowConfig::FLOOR); let (left, right) = join!(Box::pin(mirror(a, remote_b)), Box::pin(mirror(remote_a, b))); ( @@ -258,11 +304,11 @@ async fn reconcile_with_stacked_failures( /// Extract the injected backend operation from a proxy conversion failure. fn injected_operation(error: &ProxyFailure) -> Option { - use crate::tree::mirror::streaming::remote::{DecodeError, EncodeError}; + use crate::tree::mirror::streaming::remote::{ReplyDecodeError, ReplyEncodeError}; match error { - RemoteError::Encode(EncodeError::Backend(Failure::Injected(operation))) - | RemoteError::Decode(DecodeError::Backend(Failure::Injected(operation))) => { + RemoteError::Encode(ReplyEncodeError::Backend(Failure::Injected(operation))) + | RemoteError::Decode(ReplyDecodeError::Backend(Failure::Injected(operation))) => { Some(*operation) } _ => None, diff --git a/src/tree/mirror/streaming/remote/proxy/tests/containment.rs b/src/tree/mirror/streaming/remote/proxy/tests/containment.rs index 3cce0da4c..340a8c36c 100644 --- a/src/tree/mirror/streaming/remote/proxy/tests/containment.rs +++ b/src/tree/mirror/streaming/remote/proxy/tests/containment.rs @@ -1,6 +1,6 @@ //! Version-containment enforcement over the full wire stack. -use crate::message::Message; +use crate::message::{PayloadCodec, PayloadDepthLimit}; use futures::join; use crate::link::memory_with_capacity; @@ -33,10 +33,18 @@ async fn reconcile_results( let b = Handshaking::start(Local, Root::::from(b)).window(WindowConfig::FLOOR); let (a_link, b_link) = memory_with_capacity(TRANSPORT_CAPACITY); - let remote_b = RemoteHandshaking::start(Local, a_link, Message::deserializer::<()>()) - .window(WindowConfig::FLOOR); - let remote_a = RemoteHandshaking::start(Local, b_link, Message::deserializer::<()>()) - .window(WindowConfig::FLOOR); + let remote_b = RemoteHandshaking::start( + Local, + a_link, + PayloadCodec::new::<()>(PayloadDepthLimit::default()), + ) + .window(WindowConfig::FLOOR); + let remote_a = RemoteHandshaking::start( + Local, + b_link, + PayloadCodec::new::<()>(PayloadDepthLimit::default()), + ) + .window(WindowConfig::FLOOR); let (a, b) = join!(Box::pin(mirror(a, remote_b)), Box::pin(mirror(remote_a, b))); ( diff --git a/src/tree/mirror/streaming/remote/proxy/tests/declarations.rs b/src/tree/mirror/streaming/remote/proxy/tests/declarations.rs index 2b8e3fec5..08944218c 100644 --- a/src/tree/mirror/streaming/remote/proxy/tests/declarations.rs +++ b/src/tree/mirror/streaming/remote/proxy/tests/declarations.rs @@ -19,7 +19,7 @@ use crate::tree::{ Error as MirrorError, streaming::{ remote::{ - CodecDecodeError, CodecDecodeErrorKind, DecodeError, Error as RemoteError, + CodecDecodeError, CodecDecodeErrorKind, Error as RemoteError, ReplyDecodeError, StreamError, }, window::FAN, @@ -198,7 +198,7 @@ fn understated_version_bytes_fail_the_session() { }; assert!(matches!( receiver_error, - RemoteError::Decode(DecodeError::OversizedVersion { declared: 0, .. }) + RemoteError::Decode(ReplyDecodeError::OversizedVersion { declared: 0, .. }) )); assert!(left.is_err()); assert!(right.is_err()); @@ -262,7 +262,7 @@ fn understated_set_len_fails_the_session() { assert!( matches!( receiver_error, - RemoteError::Decode(DecodeError::OverdrawnSupply { declared: 0 }) + RemoteError::Decode(ReplyDecodeError::OverdrawnSupply { declared: 0 }) ), "mistyped set_len violation: {receiver_error:?}", ); @@ -334,7 +334,7 @@ fn set_len_overrun_within_one_reply_fails_at_ingress() { assert!( matches!( receiver_error, - RemoteError::Decode(DecodeError::OverdrawnSupply { declared: 1 }) + RemoteError::Decode(ReplyDecodeError::OverdrawnSupply { declared: 1 }) ), "mistyped within-one-reply set_len violation: {receiver_error:?}", ); diff --git a/src/tree/mirror/streaming/remote/proxy/tests/harness.rs b/src/tree/mirror/streaming/remote/proxy/tests/harness.rs index b77a3a611..9327cbb7f 100644 --- a/src/tree/mirror/streaming/remote/proxy/tests/harness.rs +++ b/src/tree/mirror/streaming/remote/proxy/tests/harness.rs @@ -1,6 +1,6 @@ //! Reusable two-proxy session harness for transport-adversity properties. -use crate::message::Message; +use crate::message::{PayloadCodec, PayloadDepthLimit}; use std::{ convert::Infallible, io, @@ -17,7 +17,7 @@ use tokio::io::ReadBuf; use crate::link::{Acceptor, Connector, Done, Link, MemoryLink, memory_with_capacity}; use crate::testing::{IoPlan, IoReportHandle, IoSide, wrap_link}; -use crate::tree::mirror::framing::{GREETING_WORD_LEN, LENGTH_HEADER_LEN}; +use crate::tree::mirror::cbor; use crate::tree::mirror::streaming::window::WindowConfig; use crate::tree::{ Root as TreeRoot, @@ -41,10 +41,6 @@ const QUERY_STATES: RangeInclusive = 4..=5; /// Dense states below this boundary carry reactions rather than bare ends. const REACTION_STATE_COUNT: u8 = 8; -// The label's width is defined canonically beside the sender that writes -// it; the harness scripts frames with the same constant. -use super::super::super::streams::LABEL_LEN; - /// Failure returned by the materialized-left/proxy-right driver. pub type LeftError = MirrorError, RemoteError>; @@ -121,15 +117,16 @@ impl Script { /// /// Every flush below the [`StreamSender`] carries exactly one frame, so the /// flush boundary is the frame boundary. The stream's first flush carries -/// the two-byte label ahead of its frame; mutations offset past it and leave -/// it intact. +/// the label items ahead of its frame; mutations parse past them and leave +/// them intact. /// /// [`StreamSender`]: crate::tree::mirror::streaming::remote::streams::StreamSender pub struct ScriptedWrite { inner: W, script: Option