From ecebe0ae30ea22228f1929f5901f5b88bba47273 Mon Sep 17 00:00:00 2001 From: Mark Mennell Date: Mon, 10 Aug 2026 21:27:19 +1000 Subject: [PATCH 1/9] Sync SPEC.md from fmsg-spec v0.5.0 consistency fixes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Notable for fmsgd: the spec now states what the implementation already does — stored-message verification includes messages the host sent, exactly one header-response code, add to may not overlap to, add-to copies omit topic and set has-pid, unsupported first byte gets code 2. Co-Authored-By: Claude Fable 5 --- SPEC.md | 24 +++++++++++++----------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/SPEC.md b/SPEC.md index 7ebcf1a..284f4e4 100644 --- a/SPEC.md +++ b/SPEC.md @@ -24,7 +24,7 @@ All fields are read sequentially. `[ ]` = conditionally present. | 4 | from | address | Sender address. | | 5 | to | uint8 count + addresses | ≥ 1 distinct (case-insensitive) addresses. | | 6 | [add to from] | address | Present iff flag bit 1 set. Must be in _from_ or _to_. | -| 7 | [add to] | uint8 count + addresses | Present iff flag bit 1 set. ≥ 1 distinct addresses. | +| 7 | [add to] | uint8 count + addresses | Present iff flag bit 1 set. ≥ 1 distinct addresses, none also in _to_ (case-insensitive). | | 8 | time | float64 | POSIX epoch, stamped by sending host. | | 9 | [topic] | uint8 length + UTF-8 | Present iff pid is NOT present. Length may be 0. | | 10 | type | uint8 + [US-ASCII string] | If flag bit 2 (common type) set: uint8 is a Common Media Type ID (see §4). Otherwise: uint8 is length of subsequent ASCII Media Type string. | @@ -137,7 +137,7 @@ Single-value codes (sent as first/only byte): | 7 | too old | Timestamp too far in past. | | 8 | future time | Timestamp too far in future. | | 9 | time travel | Timestamp before parent's timestamp. | -| 10 | duplicate | Already received for all recipients. | +| 10 | duplicate | Already received for all recipients, or this exact add-to batch already recorded. | | 11 | accept add to | Add-to accepted; parent already stored; no _add to_ recipients on this host (including notification-only participant domains). Stop. | | 64 | continue | Header accepted; send data. | | 65 | skip data | Add-to accepted; parent already stored; _add to_ recipients on this host. Skip data, per-recipient codes follow. | @@ -180,7 +180,7 @@ Host A delivers iff _from_ or _add to from_ belongs to Host A's domain. When _has add to_ is NOT set: perform the steps below for each unique recipient domain. -When _has add to_ IS set: perform the steps below for each unique participant domain — the domains of _from_ and of every address in _to_ and _add to_. _from_'s domain is omitted when _from_ is the _add to from_ (the adder is the original sender, whose host is Host A). Domains having no address in this message's _to_ or _add to_ are **notification-only**: the exchange completes at the single response code in step 5 (code 11 on success, or code 6 when the domain's host does not hold the parent) and never reaches step 6. +When _has add to_ IS set: perform the steps below for each unique participant domain — the domains of _from_ and of every address in _to_ and _add to_. _from_'s domain is omitted when _from_ is the _add to from_ (the adder is the original sender, whose host is Host A). Domains having no address in this message's _to_ or _add to_ are **notification-only**: the exchange completes at the single response code in step 5 (code 11 on success, or code 6 when the domain's host does not hold the parent) and never reaches step 6. (A domain hosting recipients in _to_ but none in _add to_ likewise completes at code 11 on success.) 1. Resolve recipient domain IPs via ``fmsg.``. Connect to first responsive IP (Connection 1). Retry with backoff if unreachable. 2. Register the message header hash and Host B's IP in an outgoing record (for matching challenges). @@ -204,7 +204,7 @@ When _has add to_ IS set: perform the steps below for each unique participant do 2. Parse remaining header. If unparseable → TERMINATE. 3. Validate (all must pass, else respond code 1 invalid and close): - _to_ has ≥ 1 distinct address. - - If _has add to_: _add to from_ exists and is in _from_ or _to_; _add to_ has ≥ 1 distinct address. + - If _has add to_: _add to from_ exists and is in _from_ or _to_; _add to_ has ≥ 1 distinct address, none also in _to_ (case-insensitive). - If _has add to_ not set: ≥ 1 recipient in _to_ belongs to Host B's domain. If _has add to_ set: ≥ 1 participant (_from_, _to_, _add to from_ or _add to_) belongs to Host B's domain. - Common type IDs (message and attachment) are mapped. - _expanded size_ fields are present iff the corresponding zlib-deflate flag is set. @@ -228,6 +228,8 @@ When _has add to_ IS set: perform the steps below for each unique participant do ### 10.4 Receiving — ACCEPT Response, Data Download and Per-Recipient Response +Steps 1–3 determine exactly one response code for the message header: the first rule that matches decides the code sent and the remaining rules are not evaluated. + 1. If _add to_ set and parent verified stored in step 7: - If Host B has already recorded this exact add-to batch (§11) → respond code 10 (duplicate), close. - If any _add to_ recipient belongs to Host B's domain → respond 65 (skip data). @@ -254,9 +256,9 @@ The challenge is optional (Receiving Host's discretion). It runs on a separate C **Sending Host (Host A) handles:** 1. Read first byte on incoming connection: - - 1–127 → incoming message, handle normally. - - 129–255 → CHALLENGE, continue. - - Other → TERMINATE connection. + - 1–127 and supported → incoming message, handle normally. + - 129–255 and (256 − value) supported → CHALLENGE, continue. + - Otherwise → respond code 2 (unsupported version), close. 2. Read 32-byte header hash. Match against outgoing record by header hash AND challenger's IP. No match → TERMINATE. 3. Send CHALLENGE RESPONSE: 32-byte SHA-256 of entire message. @@ -267,7 +269,7 @@ Host A MUST maintain a record of outgoing messages keyed by message header hash, ## 11. Verifying Message Stored A message is verified as stored iff: -- A SHA-256 digest matches a previously accepted message (code 200 or 11). +- A SHA-256 digest matches a message either previously accepted by the host (code 200 to ≥ 1 recipient, or code 11), or sent by the host (its _from_ / _add to from_ belongs to the host's domain and the host transmitted it or holds it for sending). - That message currently exists and is retrievable. A host MUST retain each stored message in full and exactly as transmitted — including the complete _to_ and _add to_ recipient lists, not only recipients on its own domain — so the message hash can always be faithfully recomputed and participant checks (§10.3 step 7) evaluate against the true participant set. @@ -279,10 +281,10 @@ Each add-to batch produces a distinct hash. Only the exact batch that had an acc ## 12. Adding Recipients An add-to message is a duplicate of the original message with these differences: -- Flag bit 1 (_has add to_) set. -- _pid_ = hash of the message being added to. +- Flag bits 0 (_has pid_) and 1 (_has add to_) set. +- _pid_ = hash of the message being added to (replacing any _pid_ the original had). - _add to from_ = participant initiating the add (must be in original _from_ or _to_). -- _add to_ = new recipient addresses. +- _add to_ = new recipient addresses (none already in _to_). - _time_ = new timestamp. - _topic_ is NOT present (pid is set). From 4e492de05c0993e0f6a080fb8b909ec934427a23 Mon Sep 17 00:00:00 2001 From: Mark Mennell Date: Tue, 11 Aug 2026 07:04:44 +1000 Subject: [PATCH 2/9] Sync SPEC.md: add-to batch branch semantics and hash identity Co-Authored-By: Claude Fable 5 --- SPEC.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/SPEC.md b/SPEC.md index 284f4e4..9354ab1 100644 --- a/SPEC.md +++ b/SPEC.md @@ -137,7 +137,7 @@ Single-value codes (sent as first/only byte): | 7 | too old | Timestamp too far in past. | | 8 | future time | Timestamp too far in future. | | 9 | time travel | Timestamp before parent's timestamp. | -| 10 | duplicate | Already received for all recipients, or this exact add-to batch already recorded. | +| 10 | duplicate | Already received for all recipients, or an add-to batch with this message hash already recorded. | | 11 | accept add to | Add-to accepted; parent already stored; no _add to_ recipients on this host (including notification-only participant domains). Stop. | | 64 | continue | Header accepted; send data. | | 65 | skip data | Add-to accepted; parent already stored; _add to_ recipients on this host. Skip data, per-recipient codes follow. | @@ -276,7 +276,7 @@ A host MUST retain each stored message in full and exactly as transmitted — in For accept-add-to (code 11) messages, the hash is computed by combining the add-to message header with the original message's data and attachment data. -Each add-to batch produces a distinct hash. Only the exact batch that had an accepted response (200 or 11) matches. +Each add-to batch produces a distinct hash — batch identity IS the batch message hash, which covers _time_: the same _add to_ addresses re-issued at a new _time_ are a distinct batch. Only the exact batch that had an accepted response (200 or 11) matches. ## 12. Adding Recipients @@ -290,7 +290,11 @@ An add-to message is a duplicate of the original message with these differences: An add-to message MUST be sent to every participant domain per §10.2, so all participants of the message being added to — including the original sender, when not themselves the _add to from_ — learn of the added recipients, not only the domains hosting the new recipients. This is required because a subsequent reply may reference this add-to message via _pid_, and a host can only accept a reply whose parent it holds. -Add-to batches do not chain: recipients are always added to the original message; an add-to message's _pid_ MUST NOT reference another add-to message. A message therefore has 0 or more add-to batches, each independently referencing it. +Add-to batches do not chain: recipients are always added to the original message; an add-to message's _pid_ MUST NOT reference another add-to message. A message therefore has 0 or more add-to batches, each a sibling branch under the original — the thread evolves as a tree. + +Added recipients are participants of their add-to batch message only, not of the original: their replies MUST reference the batch message via _pid_ (referencing the original would fail the participant check, §10.3 step 7) and extend the batch's branch. + +A batch is identified by its message hash, which covers _time_ (§11): re-issuing the same _add to_ addresses at a new _time_ is a new, distinct batch — a new sibling branch — not a duplicate. ## 13. Security Requirements From 9b4a5fa607ed9b0c054dd6e2273d2c61038cc12f Mon Sep 17 00:00:00 2001 From: Mark Mennell Date: Tue, 11 Aug 2026 09:33:02 +1000 Subject: [PATCH 3/9] Sync SPEC.md: batch hash verification by reconstruction Co-Authored-By: Claude Fable 5 --- SPEC.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/SPEC.md b/SPEC.md index 9354ab1..14733d4 100644 --- a/SPEC.md +++ b/SPEC.md @@ -269,12 +269,12 @@ Host A MUST maintain a record of outgoing messages keyed by message header hash, ## 11. Verifying Message Stored A message is verified as stored iff: -- A SHA-256 digest matches a message either previously accepted by the host (code 200 to ≥ 1 recipient, or code 11), or sent by the host (its _from_ / _add to from_ belongs to the host's domain and the host transmitted it or holds it for sending). +- A SHA-256 digest matches a message either previously accepted by the host (code 200 to ≥ 1 recipient, or code 11), or sent by the host (its _from_ / _add to from_ belongs to the host's domain and the host transmitted it or holds it for sending — including each add-to batch it sent, since replies may reference a batch by its hash). - That message currently exists and is retrievable. A host MUST retain each stored message in full and exactly as transmitted — including the complete _to_ and _add to_ recipient lists, not only recipients on its own domain — so the message hash can always be faithfully recomputed and participant checks (§10.3 step 7) evaluate against the true participant set. -For accept-add-to (code 11) messages, the hash is computed by combining the add-to message header with the original message's data and attachment data. +An add-to message's data never crosses the wire when the host already holds the original (codes 11 and 65), and its sender likewise already holds it. Its hash is therefore computed by RECONSTRUCTION: the add-to header exactly as transmitted combined with the original message's data and attachment data. Every host holding a batch — accepted via 11 or 65, or sent by it — MUST be able to reconstruct it so replies referencing the batch hash can be verified; hosts on the 65 path MUST record the batch fields (_add to from_, _add to_, _time_) exactly as transmitted, as the 11 path already requires. Each add-to batch produces a distinct hash — batch identity IS the batch message hash, which covers _time_: the same _add to_ addresses re-issued at a new _time_ are a distinct batch. Only the exact batch that had an accepted response (200 or 11) matches. From 968013726bd5ec84750992c63a9505827f2bc92a Mon Sep 17 00:00:00 2001 From: Mark Mennell Date: Tue, 11 Aug 2026 09:45:56 +1000 Subject: [PATCH 4/9] Sync SPEC.md: terminate on unsupported challenge version Co-Authored-By: Claude Fable 5 --- SPEC.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/SPEC.md b/SPEC.md index 14733d4..cef7c0f 100644 --- a/SPEC.md +++ b/SPEC.md @@ -200,7 +200,7 @@ When _has add to_ IS set: perform the steps below for each unique participant do 1. Read first byte on Connection 1: - 1–127 and supported → message version, continue. - 129–255 and (256 − value) supported → incoming CHALLENGE, handle per §10.5. - - Otherwise → respond code 2 (unsupported version), close. + - Otherwise, unsupported: value ≤ 128 → respond code 2 (unsupported version), close (the peer reads a response code first). Value > 128 → TERMINATE without responding (the challenger's next read is exactly a 32-byte hash; a code byte would be indistinguishable from it). 2. Parse remaining header. If unparseable → TERMINATE. 3. Validate (all must pass, else respond code 1 invalid and close): - _to_ has ≥ 1 distinct address. @@ -258,7 +258,7 @@ The challenge is optional (Receiving Host's discretion). It runs on a separate C 1. Read first byte on incoming connection: - 1–127 and supported → incoming message, handle normally. - 129–255 and (256 − value) supported → CHALLENGE, continue. - - Otherwise → respond code 2 (unsupported version), close. + - Otherwise, unsupported: value ≤ 128 → respond code 2 (unsupported version), close. Value > 128 → TERMINATE without responding (the challenger's next read is exactly a 32-byte hash; a code byte would be indistinguishable from it). 2. Read 32-byte header hash. Match against outgoing record by header hash AND challenger's IP. No match → TERMINATE. 3. Send CHALLENGE RESPONSE: 32-byte SHA-256 of entire message. From a0ea8a675d77477b8b3c114b89947f59be1987e1 Mon Sep 17 00:00:00 2001 From: Mark Mennell Date: Tue, 11 Aug 2026 09:56:27 +1000 Subject: [PATCH 5/9] Sync SPEC.md: name the one notification-only case explicitly Co-Authored-By: Claude Fable 5 --- SPEC.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/SPEC.md b/SPEC.md index cef7c0f..4cf5dfa 100644 --- a/SPEC.md +++ b/SPEC.md @@ -180,7 +180,7 @@ Host A delivers iff _from_ or _add to from_ belongs to Host A's domain. When _has add to_ is NOT set: perform the steps below for each unique recipient domain. -When _has add to_ IS set: perform the steps below for each unique participant domain — the domains of _from_ and of every address in _to_ and _add to_. _from_'s domain is omitted when _from_ is the _add to from_ (the adder is the original sender, whose host is Host A). Domains having no address in this message's _to_ or _add to_ are **notification-only**: the exchange completes at the single response code in step 5 (code 11 on success, or code 6 when the domain's host does not hold the parent) and never reaches step 6. (A domain hosting recipients in _to_ but none in _add to_ likewise completes at code 11 on success.) +When _has add to_ IS set: perform the steps below for each unique participant domain — the domains of _from_ and of every address in _to_ and _add to_. _from_'s domain is omitted when _from_ is the _add to from_ (the adder is the original sender, whose host is Host A). Domains having no address in this message's _to_ or _add to_ are **notification-only**: the exchange completes at the single response code in step 5 (code 11 on success, or code 6 when the domain's host does not hold the parent) and never reaches step 6. (A domain hosting recipients in _to_ but none in _add to_ likewise completes at code 11 on success.) Since _add to from_ is always in _from_ or _to_, the only possible notification-only domain is _from_'s — it arises exactly when a recipient, not the original sender, adds recipients; the notification is what lets _from_'s host hold the batch and accept replies referencing it via _pid_. 1. Resolve recipient domain IPs via ``fmsg.``. Connect to first responsive IP (Connection 1). Retry with backoff if unreachable. 2. Register the message header hash and Host B's IP in an outgoing record (for matching challenges). From 0c4548bc9c0605f3d5c32c103d17bcf28931c8be Mon Sep 17 00:00:00 2001 From: Mark Mennell Date: Tue, 11 Aug 2026 10:14:50 +1000 Subject: [PATCH 6/9] Sync SPEC.md: simplify code 11 wording Co-Authored-By: Claude Fable 5 --- SPEC.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/SPEC.md b/SPEC.md index 4cf5dfa..7d3667a 100644 --- a/SPEC.md +++ b/SPEC.md @@ -138,7 +138,7 @@ Single-value codes (sent as first/only byte): | 8 | future time | Timestamp too far in future. | | 9 | time travel | Timestamp before parent's timestamp. | | 10 | duplicate | Already received for all recipients, or an add-to batch with this message hash already recorded. | -| 11 | accept add to | Add-to accepted; parent already stored; no _add to_ recipients on this host (including notification-only participant domains). Stop. | +| 11 | accept add to | Already holds the rest of the message and hosts no _add to_ recipient — nothing more to send; batch recorded. Stop. | | 64 | continue | Header accepted; send data. | | 65 | skip data | Add-to accepted; parent already stored; _add to_ recipients on this host. Skip data, per-recipient codes follow. | @@ -180,7 +180,7 @@ Host A delivers iff _from_ or _add to from_ belongs to Host A's domain. When _has add to_ is NOT set: perform the steps below for each unique recipient domain. -When _has add to_ IS set: perform the steps below for each unique participant domain — the domains of _from_ and of every address in _to_ and _add to_. _from_'s domain is omitted when _from_ is the _add to from_ (the adder is the original sender, whose host is Host A). Domains having no address in this message's _to_ or _add to_ are **notification-only**: the exchange completes at the single response code in step 5 (code 11 on success, or code 6 when the domain's host does not hold the parent) and never reaches step 6. (A domain hosting recipients in _to_ but none in _add to_ likewise completes at code 11 on success.) Since _add to from_ is always in _from_ or _to_, the only possible notification-only domain is _from_'s — it arises exactly when a recipient, not the original sender, adds recipients; the notification is what lets _from_'s host hold the batch and accept replies referencing it via _pid_. +When _has add to_ IS set: perform the steps below for each unique participant domain — the domains of _from_ and of every address in _to_ and _add to_. _from_'s domain is omitted when _from_ is the _add to from_ (the adder is the original sender, whose host is Host A). Domains having no address in this message's _to_ or _add to_ are **notification-only**: the exchange completes at the single response code in step 5 (code 11 on success, or code 6 when the domain's host does not hold the parent) and never reaches step 6. (A domain hosting recipients in _to_ but none in _add to_ likewise completes at code 11 on success. The only possible notification-only domain is _from_'s — when a recipient, not the sender, adds recipients.) 1. Resolve recipient domain IPs via ``fmsg.``. Connect to first responsive IP (Connection 1). Retry with backoff if unreachable. 2. Register the message header hash and Host B's IP in an outgoing record (for matching challenges). From 05215dacd4c9525dc92ef34a133955f1dad00d95 Mon Sep 17 00:00:00 2001 From: Mark Mennell Date: Tue, 11 Aug 2026 10:19:25 +1000 Subject: [PATCH 7/9] Sync SPEC.md: code 10 is simply duplicate Co-Authored-By: Claude Fable 5 --- SPEC.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/SPEC.md b/SPEC.md index 7d3667a..2027b9f 100644 --- a/SPEC.md +++ b/SPEC.md @@ -137,7 +137,7 @@ Single-value codes (sent as first/only byte): | 7 | too old | Timestamp too far in past. | | 8 | future time | Timestamp too far in future. | | 9 | time travel | Timestamp before parent's timestamp. | -| 10 | duplicate | Already received for all recipients, or an add-to batch with this message hash already recorded. | +| 10 | duplicate | Message already received by this host (for every recipient it hosts, if any). | | 11 | accept add to | Already holds the rest of the message and hosts no _add to_ recipient — nothing more to send; batch recorded. Stop. | | 64 | continue | Header accepted; send data. | | 65 | skip data | Add-to accepted; parent already stored; _add to_ recipients on this host. Skip data, per-recipient codes follow. | From a090143d8e74fae56a8047fed5d82f1fa0274af5 Mon Sep 17 00:00:00 2001 From: Mark Mennell Date: Tue, 11 Aug 2026 10:21:36 +1000 Subject: [PATCH 8/9] Sync SPEC.md: code 10 row without qualifiers Co-Authored-By: Claude Fable 5 --- SPEC.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/SPEC.md b/SPEC.md index 2027b9f..47ccba6 100644 --- a/SPEC.md +++ b/SPEC.md @@ -137,7 +137,7 @@ Single-value codes (sent as first/only byte): | 7 | too old | Timestamp too far in past. | | 8 | future time | Timestamp too far in future. | | 9 | time travel | Timestamp before parent's timestamp. | -| 10 | duplicate | Message already received by this host (for every recipient it hosts, if any). | +| 10 | duplicate | Message already received by this host. | | 11 | accept add to | Already holds the rest of the message and hosts no _add to_ recipient — nothing more to send; batch recorded. Stop. | | 64 | continue | Header accepted; send data. | | 65 | skip data | Add-to accepted; parent already stored; _add to_ recipients on this host. Skip data, per-recipient codes follow. | From 70ccb9f303c617c7347cc9fe389218f26522ad8c Mon Sep 17 00:00:00 2001 From: Mark Mennell Date: Tue, 11 Aug 2026 10:23:43 +1000 Subject: [PATCH 9/9] Sync SPEC.md: terse code 11 table row Co-Authored-By: Claude Fable 5 --- SPEC.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/SPEC.md b/SPEC.md index 47ccba6..54fcede 100644 --- a/SPEC.md +++ b/SPEC.md @@ -138,7 +138,7 @@ Single-value codes (sent as first/only byte): | 8 | future time | Timestamp too far in future. | | 9 | time travel | Timestamp before parent's timestamp. | | 10 | duplicate | Message already received by this host. | -| 11 | accept add to | Already holds the rest of the message and hosts no _add to_ recipient — nothing more to send; batch recorded. Stop. | +| 11 | accept add to | Additional recipients received, discontinue. | | 64 | continue | Header accepted; send data. | | 65 | skip data | Add-to accepted; parent already stored; _add to_ recipients on this host. Skip data, per-recipient codes follow. |