Skip to content

feat: reconciliation-on-connect handshake — Phases 1–2 (server + protocol, synthetic-client proof)#523

Merged
danshapiro merged 8 commits into
mainfrom
feat/sync-handshake-phase1-2
Jul 24, 2026
Merged

feat: reconciliation-on-connect handshake — Phases 1–2 (server + protocol, synthetic-client proof)#523
danshapiro merged 8 commits into
mainfrom
feat/sync-handshake-phase1-2

Conversation

@danshapiro

Copy link
Copy Markdown
Owner

Reconciliation-on-connect handshake — Phases 1–2 (server + protocol, synthetic-client proof)

Implements docs/plans/2026-07-22-reconciliation-handshake-design.md Phase 1 and Phase 2 ONLY. Client adoption (Phase 3) and the §12 fresh-agent extension are explicitly out of scope.

Ship-alone safety (frozen client untouched — proven, not asserted)

Three independent gates keep the feature dark: hello capability → ready advertisement → result-only-in-response-to-request.

  • handshake_advertises_pane_reconcile_only_when_negotiated (unit) + hello_without_capability_gets_unchanged_ready_and_with_it_gets_advertised (wire): a non-negotiating hello produces a ready with no capabilities key — clean-boot handshake bytes unchanged.
  • frozen_client_create_path_is_unchanged_no_dedupe (wire): a non-negotiating connection's terminal.create keeps today's blind-spawn behavior byte-for-byte.
  • non_negotiating_connection_gets_no_reconcile_response (wire): the request is accept-and-strip ignored off-capability.
  • Full existing PW-RUST e2e + coordinated npm test suite green against this server build (§9.2 scenario 6, e2e-level inertness).

What landed (8 commits)

  1. protocolpane.reconcile.request/.result frames, paneReconcileV1 on hello + ready.capabilities, RECONCILE_TOO_LARGE/RECONCILE_UNAVAILABLE; inventory 27→28 / 52→53; protocolVersion stays 7.
  2. terminalcreateRequestId stamped atomically with the registry insert (field of the inserted row; concurrent insert/reader race test); newest_live_by_create_request_id / newest_by_create_request_id (newest generation first, live vs exited-inclusive); is_live/probe; always-on ws.reconcile.duplicate_pty backstop detector; public register_headless seam for cross-crate tests.
  3. terminal — §7.5 respawn-generation cap (respawn_exhausted), healthy-generation reset.
  4. wsSessionExistenceProbe (Present/Absent/Unknown with the §5.1 defined semantics; unknown provider → Absent never Unknown) + pure derive_verdicts (§5.2/§5.3) with decision-table unit tests for rows 1, 2, 2b, 3, 4, 4b, 5, 6, 8, 9, 10 plus §9.1 tests 3/4a/9/11/12/14/15 and the Assumption-1 Phase-1 acceptance check (REST-created resume resolves identity via the registry-side resume_session_id across the crate boundary); capability negotiation; handle_pane_reconcile (cap → TOO_LARGE, catch_unwind → UNAVAILABLE, both carrying reconcileId).
  5. ws — §5.4 single-flight create-dedupe (capability-gated adopt branch) + the wire-level §9.1 test file.
  6. server — index-backed probe over the shared SessionIndex (peek()/is_fresh() sync accessors added to freshell-sessions); §9.1 test 13 real-index staleness (session written after a cold read resolves Present on re-query, no latched stale Absent); monotone ever_observed set gating dead_session vs fresh(identity_never_observed); main.rs wiring (NoIndexProbe fallback when no provider home).
  7. ws/terminal — keyed-create reservation (begin/end_keyed_create + RAII guard) closing the check-then-spawn window between two truly concurrent creates for one key (the spawn takes ms; the row is only observable at insert).
  8. e2etest/e2e-browser/specs/reconcile-handshake-rust.spec.ts (§9.2, raw Node WebSocket synthetic client on the HARNESS-01 owned RustServer): restart → shell fresh / resumed CLI respawn with correct sessionRef → complete respawn → attach (doubles as WSL-restart equivalence + refreshed-browser analogue); observed-then-deleted session → explicit dead_session, disk untouched; two concurrent reconciling connections → exactly one live PTY per key (the council-blocker regression). All 3 green in 25.8s.

Verification evidence

  • cargo test --workspace: green (61 suite results, 0 failures).
  • cargo clippy --workspace --all-targets: no new warnings — every remaining warning verified present at origin/main (e.g. the TabsPushResponse large-variant warning exists at ec9970c, terminal.rs:1638).
  • cargo fmt applied to all touched crates.
  • Coordinated FRESHELL_TEST_SUMMARY="track1 handshake pre-PR" npm test: full-suite success exit=0 at HEAD 5de61ec, dirty:0.
  • Playwright: npx playwright test ... reconcile-handshake-rust → 3 passed.

Open decision (recorded, not resolved)

  • §8.0 retry mechanism (change #4a) is USER DECISION PENDING. This PR ships the design's documented placeholder: retry{reason:'index_warming', retryAfterMs:2000}. Adopting Sam's Option B later collapses row 5 into the existing RECONCILE_UNAVAILABLE path with no other table change.

Deferred (out of this track's scope)

  • Phase 3 client adoption (§10) and Phase 4 convergence — including the §5.5 client-side createRequestId contract fixes (persistMiddleware.ts re-mint on hydrate; REST ingress minting none — REST creates pass None for the stamp today, documented at the call site).
  • Incident-4 sidebar-grey quick win (assessment Option A): it is a client-side change (src/store/selectors/sidebarSelectors.ts open-state join) — another track owns client alerting/selector files, so per the wave brief it is noted here instead of implemented.
  • §12 fresh-agent verdicts (explicitly fenced out of v1).
  • §9.2 scenario 2 as a separate spec (double-restart with a kill between request and result) — the interrupted-round convergence shape is pinned at wire level (interrupted_create_converges_to_attach_on_the_next_reconcile); a dedicated e2e double-restart spec would be additive.

Environment notes for the orchestrator

  • The shared disk hit 100% (0 bytes free) mid-run. Freed regenerable caches only: npm cache clean --force, uv cache clean, go clean -cache, and cargo clean of the stale, merged .worktrees/rust-tauri-port/target (34 GB, untouched since 2026-07-18). No live worktrees or processes were touched.

codex and others added 8 commits July 23, 2026 23:43
…concile error codes

Additive wire surface for the reconciliation handshake (design doc
2026-07-22-reconciliation-handshake-design.md §4):

- ClientMessage::PaneReconcileRequest (pane.reconcile.request), parse-tolerant
  per-pane claims so malformed entries get 'invalid' verdicts, never a frame
  parse failure
- ServerMessage::PaneReconcileResult (pane.reconcile.result) with per-pane
  verdict enum attach|respawn|fresh|dead_session|retry|invalid
- HelloCapabilities.paneReconcileV1 + Ready.capabilities advertisement
  (Option + skip_serializing_if: clean-boot handshake bytes unchanged)
- ErrorCode::{ReconcileTooLarge,ReconcileUnavailable}
- ws-message-inventory.json 27→28 client / 52→53 server types

protocolVersion stays 7.

🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
… + newest-generation accessors

Reconciliation handshake design §5.1: the pane's stable creation key is now a
field of the inserted TerminalShared row, set under the same registry lock
that inserts it — no observer can ever see a row without its key (§9.1 test 5,
proven by a concurrent insert/reader race test).

- TerminalRegistry::create gains create_request_id (WS path stamps
  terminal.create.requestId; REST ingress passes None per §5.5 precondition 2)
- newest_live_by_create_request_id / newest_by_create_request_id: newest
  generation first, live-only vs exited-inclusive (§5.2 derivation inputs)
- is_live / probe: run-status + identity-probe row per terminal (crate-boundary
  identity path for REST-created resumes, design assumption 1)
- §5.4 backstop detector: ≥2 live PTYs on one key emits
  ws.reconcile.duplicate_pty warn — the duplicate-writer shape made loud
- register_headless: public headless seam so cross-crate (freshell-ws) wire
  tests can seed live/exited generations deterministically

🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
A respawn ↔ instant-exit loop (corrupt JSONL, CLI dying on resume) would make
the reconcile derivation re-issue 'respawn' forever. The registry now counts
consecutive generations that exited inside a liveness window (default 30s,
cap 3); respawn_exhausted(key) lets the derivation return
dead_session(reason='respawn_exhausted') instead — restoring the 'at most one
respawn' convergence bound as a guarantee. A generation that survives the
window resets the counter; user-initiated kills never count.

🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
…egotiation, WS handler

The server half of the reconciliation handshake (design §4/§5/§8):

- existence.rs: SessionExistenceProbe trait with defined semantics — Present/
  Absent require a known consulted provider; unknown provider → Absent (never
  Unknown); Unknown reserved for a cold index on a known provider. NoIndexProbe
  fallback for boots with no provider home.
- reconcile.rs: pure, read-only derive_verdicts over registry × identity ×
  disk (§5.2/§5.3) — decision-table unit tests cover rows 1, 2, 2b, 3, 4, 4b,
  5, 6, 8, 9, 10 plus §9.1-named tests: cardinality/opacity with hostile
  paneKeys, pure-read idempotency, exited-only-generations never attach,
  duplicate createRequestId flagged (§5.5), respawn_exhausted → dead_session
  (§7.5), REST-created-resume crate-boundary identity (assumption 1's
  Phase-1 acceptance check), trust-boundary corrected:true.
- ready capability advertisement gated on hello.capabilities.paneReconcileV1
  (§4.2): non-negotiating handshake stays byte-identical (frozen-client
  inertness at source).
- handle_pane_reconcile: 200-pane cap → RECONCILE_TOO_LARGE, catch_unwind →
  RECONCILE_UNAVAILABLE (both carrying reconcileId), result only in response
  to a request on a negotiated connection.
- retry verdict ships the documented §8.0 placeholder (index_warming +
  retryAfterMs) — the retry-mechanism decision is recorded OPEN for the user.

🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
The council's sole blocker-class finding (§0/§5.4): two reconciling browser
tabs that both receive 'respawn' for one createRequestId after a restart both
fire terminal.create — absent a guard that is two live PTYs on one key, i.e.
two JSONL writers on one session file. On paneReconcileV1 connections ONLY,
handle_create now consults newest_live_by_create_request_id first and ADOPTS
the existing live terminal (emits terminal.created naming the existing
terminalId, spawns nothing). The frozen client never negotiates the
capability, so its create flow is byte-for-byte unchanged — proven by the
frozen_client_create_path_is_unchanged_no_dedupe wire test.

Wire tests (tests/pane_reconcile.rs, real axum + tokio-tungstenite on
ephemeral ports): §9.1.1 negotiation + unchanged ready, §9.1.3 cardinality/
opacity with hostile paneKeys, §9.1.4 idempotency incl. the Incident-2
create-interrupted-before-created regression converging to attach, §9.1.7
RECONCILE_TOO_LARGE with reconcileId, §9.1.8 trust boundary corrected:true,
§9.1.10 adopt-vs-legacy both halves, and non-negotiating-connection inertness.

IdentityProbeRow gains cwd so the adopt frame echoes the existing terminal's
working directory.

🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
…sionIndex

The real disk-truth probe behind the reconcile derivation (design §5.1):

- SessionIndex gains sync, non-blocking peek() + is_fresh() — the published
  (fresh or stale) snapshot without ever waiting on a sweep
- IndexExistenceProbe: unknown provider → Absent never Unknown; cold index →
  honest Unknown + detached background refresh; stale snapshot answers now
  but kicks a refresh so re-queries converge — §9.1 test 13 proves a session
  written to disk AFTER a cold read resolves Present on re-query (no latched
  stale Absent), against a REAL SessionIndex over a temp claude home
- ever_observed is a monotone per-boot observed-set fed from every snapshot
  read — it survives the session later disappearing from disk, gating
  dead_session vs fresh(identity_never_observed)
- main.rs wires the probe from the same shared index the History surfaces
  read (session_index construction moved above WsState, no behavior change);
  NoIndexProbe fallback when no provider home resolves

🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
…wn window

The §5.4 adopt branch alone is check-then-spawn: the spawn takes milliseconds
and the row only becomes observable at insert, so two truly concurrent
creates for one createRequestId could BOTH pass newest_live_by check and both
spawn. The registry now carries an in-flight keyed-create reservation
(begin/end_keyed_create); a negotiated handle_create claims the key before
spawning, and a racing create that finds the key reserved briefly waits and
re-checks — adopting the winner's terminal. RAII guard releases on every exit
path (spawn error, socket close); bounded fail-open after ~5s with the §5.4
backstop detector still making any residual duplicate loud. Frozen-client
path untouched (claim only taken on paneReconcileV1 connections).

🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
…t server

PW-RUST specs (raw Node WebSocket synthetic client, no SPA — the §9.2
posture) on the HARNESS-01 owned RustServer fixture:

1. Server restart (Incident 1/2 shape, doubling as the WSL-restart
   equivalence and refreshed-browser analogue): shell pane → fresh, resumed
   CLI pane with an on-disk fixture claude session → respawn with the
   correct sessionRef; completing the respawn converges the next reconcile
   to attach on the spawned terminal.
2. Dead session: a session the index HAS observed disappears from disk →
   explicit dead_session(session_not_on_disk) carrying the identity; the
   session directory is untouched (sibling intact, nothing recreated).
3. Two concurrent reconciling connections (change #1 blocker regression):
   after a restart both tabs reconcile the same createRequestId, both fire
   terminal.create — exactly one live PTY afterwards via /api/terminals,
   second create adopted the first's terminalId.

All three green against target/release/freshell-server (25.8s).

🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
@danshapiro
danshapiro merged commit 4373c8e into main Jul 24, 2026
1 check passed
danshapiro pushed a commit that referenced this pull request Jul 24, 2026
Conflict resolutions (both sides' behavior preserved everywhere):

- crates/freshell-ws/src/lib.rs, terminal.rs, opencode_association.rs
  + 12 tests/*.rs WsState constructors: kept BOTH this branch's
  `activity: None` field and #523's new `session_existence` probe
  field (trivial adjacent-field conflicts). #523's brand-new
  tests/pane_reconcile.rs gained `activity: None` too.
- crates/freshell-terminal/src/registry.rs: kept BOTH the TERM-15/16
  activity tap (ActivityEvent/ActivityObserver, tap field + init, the
  Created-tap fire in create()) and #523's reconciliation additions
  (respawn liveness/cap consts + fields, keyed_create_inflight, the
  §5.4 duplicate-live-PTY backstop — ordered before the Created tap).
  create() gained #523's create_request_id param; this branch's two
  activity-observer tests updated to pass None for it.
- crates/freshell-server/src/main.rs: kept #523's session_index block
  first (ws_state.session_existence references it), then this branch's
  activity-hub construction + registry observer install.
- docs checklist + playwright.config.ts auto-merged (both sides' notes
  and MATRIX/rust-only spec entries retained).

Cross-track integration (#524 truly-idle client): the Rust
terminal.idle emission matches shared/ws-protocol.ts's new
TerminalIdleSchema byte-for-byte ({type, terminalId, at, reason:
'grace'|'queue-empty'}), so the rust leg of
test/e2e-browser/specs/truly-idle-alerting.spec.ts is now ENABLED
(test.fixme removed) — the full production pipeline (PTY submit ->
busy blue -> BEL -> idle + turn.complete -> 2s grace -> terminal.idle
-> one alert edge + tab shade) passes against the real Rust server.

Verification: cargo test --workspace 1656 passed / 0 failed; clippy
clean on touched files; rust-chromium PW (3 activity + truly-idle
rust leg) green 2x (4 passed 1.1m, 4 passed 42.5s); coordinated
npm test full-suite success exit=0 (3982 + 4547 + 350 passed; two
earlier runs each hit one unrelated pre-existing flake — DOM-identity,
sidebar-filter, EADDRINUSE:40301 — each passes in isolation).

🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants