[stacked on #2189] Publish the parity replay harness through the plugin SDK testing kit - #2193
Conversation
The recorded-replay oracle (replayRecording, compareParity, the replay child, rerecord) was private to @bb/provider-parity and keyed on a first-party map: resolveReplayProfile threw UnreplayableProviderError for any provider not in FIRST_PARTY_BRIDGE_MODULES, so a marketplace plugin could record (record mode is public) but never replay or compare. parity.ts is now provider-agnostic: the caller names the recording, the bridge process (resolveProviderBridgeLaunch builds one from a module path through the bootstrap, as the runtime spawns it) and, for a bridge with a provider child, a ReplayProviderProfile. The first-party profiles, module paths and the checkout-keyed wrappers move to testing/first-party-replay.ts (private). The lane writer behind `pnpm rerecord` becomes rerecordCurrentBridgeLane in testing/rerecord.ts; @bb/provider-parity is a thin consumer (launch, per-cell loop, redaction). replayRecording restores the recorded workspace path in the bridge's output, so a bridge that derives paths from cwd compares with its recording. @get-bb/plugin-sdk/provider-bridge/testing re-exports the harness with the experimental_ prefix and ships the bootstrap bundle and the replay child beside its bundle (the harness resolves both from import.meta.url). The echo canary records a real cell (recordings/echo-agent/turn-tools, captured on a dev bb with BB_PROVIDER_BRIDGE_RECORD_DIR and packaged with the recordings scripts) and gains provider-bridge.parity.test.ts: replay through the bridge with zero diffs and every recorded-cell rule green, plus the re-record round trip, importing only the public kit. The bridge's delegation childRef is keyed on the bb thread id instead of the minted provider thread id so a replay can reproduce it. Co-Authored-By: Claude <noreply@anthropic.com>
|
Coordinator review of stack #2196 (#2193 → #2194 → #2195) — APPROVE pending CI. Correctly stacked on the canary #2189 (G-A's acceptance test lives there); G-A — the big one, verified: G-B: canonical types exported by name, pinned in the bundled- Process note (not on this stack): you were right to gate against the Honest residuals noted: Sawyer merges; I do not.
|
Stacked on #2189 (the echo-provider canary). Layer 1 of 3 of the provider-gaps stack (#2193 → #2194 → #2195). Closes gap G-A from #2189.
What was wrong
The recorded-replay oracle — the harness every first-party bridge regression-tests with — was private and first-party-keyed.
replayRecording,compareParity,assembleRecordedEvents,replayRecordedCells, the replay child andrerecordlived in@bb/provider-parityand the protocol package'stesting/parity.ts, none of it re-exported from@get-bb/plugin-sdk/provider-bridge/testing. Worse,resolveReplayProfilethrewUnreplayableProviderErrorfor any provider id outsideFIRST_PARTY_BRIDGE_MODULESandresolveBridgeLaunchonly knew the four first-party module paths inside a bb checkout. Record mode was already public, so a marketplace plugin could record a session and then do nothing with it.What changed
packages/provider-bridge-protocol/src/testing/parity.tsis now provider-agnostic. The caller names the recording, the provider id (keys the assembler's minted ids), the bridge process, and — for a bridge with a provider child — aReplayProviderProfile:resolveProviderBridgeLaunch({ modulePath, pluginId, cwd?, dataDir?, bootstrapPath?, nodeArgs? })builds the bridge process exactly as the runtime spawns it: the bootstrap (bridge-worker-entry), the module, the plugin scope. The bootstrap resolves from beside the harness — the source entry in a checkout, the bundledprovider-bridge-worker-entry.mjsin the published SDK.ReplayRecordingOptionstakesproviderId,bridge: ProviderBridgeLaunch,profile?: ReplayProviderProfile(DEFAULT_REPLAY_PROFILEfor a bridge with no provider child).ReplayProviderProfilelost itsbridgeFamilykey;dialect,env,rewriteRuntimeLine,prepareStatestay.replayRecordedCellstakesrecordingsRootand abridge(cell)callback instead of assuming the committed fixtures and a checkout.replayRecordingrestores the recordedcwdin every line the bridge emits (the replay runs in a fresh temp workspace; a bridge that derives a command's cwd or a read path fromcwdwould otherwise never compare with its recording, and a re-recorded lane would carry the temp path).testing/first-party-replay.ts(new, private):FIRST_PARTY_BRIDGE_MODULES,resolveReplayProfile(codex / claude-code / acp-* / pi),UnreplayableProviderError,resolveBridgeLaunch(checkoutRoot, providerId),firstPartyReplayBridge, andreplayFirstPartyRecordedCells— the checkout-keyed wrapper the three first-partybridge.recorded-conformance.test.tssuites now call.testing/rerecord.ts(new):rerecordCurrentBridgeLane— the lane writer behindpnpm rerecord(anchoring, fractionalseq, recorded request-id restoration), minus the first-party launch and the redaction pass.@bb/provider-parity'srerecord.tsandindex.tsbecome thin consumers (launch, per-cell loop,redact.mjs).@get-bb/plugin-sdk/provider-bridge/testingre-exports the harness with theexperimental_prefix (experimental_replayRecording,experimental_compareParity,experimental_assembleRecordedEvents,experimental_replayRecordedCells,experimental_checkRecordedCellReplay,experimental_rerecordCurrentBridgeLane,experimental_resolveProviderBridgeLaunch,experimental_resolveProviderBridgeBootstrapPath, the normalizers and the recording readers) plus the types.scripts/build-runtime.mjsbundles the bootstrap (with the daemon's ESMrequirebanner — it pulls cross-spawn through@bb/process-utils) and copiesreplay-provider-child.mjsintodist/, where the harness resolves both byimport.meta.url.examples/plugins/echo-provider:recordings/echo-agent/turn-tools/: a real recording. A dev bb's host daemon ran the plugin's built artifact withBB_PROVIDER_BRIDGE_RECORD_DIRin its env,bb plugin install <path>+bb plugin config echo-provider set shout true,bb thread spawn --provider echo-agent, thenscripts/provider-recordings/package-cells.mjs(redacted; 0 survivors). 3 runtime lines, 7 bridge lines; the echo bridge has no provider child, so the provider lanes are empty.provider-bridge.parity.test.ts(the acceptance criterion): imports only the public kit; replays the cell throughsrc/provider-bridge.tsviaexperimental_resolveProviderBridgeLaunch, diffs against the recording's own assembled events, runsexperimental_checkRecordedCellReplay, and round-tripsexperimental_rerecordCurrentBridgeLaneon a copy of the recording (new lane keeps the recordedcwd, names the recordedecho-req-1, assembles to the same events).src/provider-bridge.ts: the delegation'schildRefis keyed on the bb thread id instead of the minted provider thread id.childRefis a value the normalizer does not intern (it interns item and turn ids), so a ref carrying per-process entropy could never reproduce across a replay. Found by the new test.Docs:
docs/provider-bridge-protocol.md(the harness is published, how a profile points a child at the replay script),docs/api_to_audit.md(the testing kit's entry: the new members and three audit items — profile/dialect shape, the two shipped programs, the workspace-path substitution), thebb-plugin-authoringskill (a "Recorded replay" paragraph).No wire change;
HOST_DAEMON_PROTOCOL_VERSIONuntouched.How you verified
pnpm exec turbo run typecheck test --filter=bb-plugin-echo-provider: 5 files, 21 tests pass. The parity self-run: 32 recorded events → 32 replayed, 0 diffs, 0 grammar drops, 0 stalls, exit 0;recorded/turn-tools/{replays, events-schema-valid, grammar, turn-lifecycle, not-empty}allpass. Before thechildReffix the same test failed with the delegation rows only-in-old/only-in-new (the ref carried the recording process's nonce) — the test finds real non-determinism./tmpimportingpackages/plugin-sdk/dist/provider-bridge-testing.js, launching the built artifact (~/.bb-dev/.../plugin-host-artifacts/echo-provider/<digest>/host.mjs) through the bundled bootstrap with no tsx:stalls [] exit 0 events 32 -> 32 passed true diffs 0, five recorded-cell rules pass.pnpm exec turbo run test --filter=@bb/provider-parity: 43/43 cells reproduce their recordings (unchanged).pnpm exec turbo run test --filter=@bb/provider-bridge-protocol: 217 pass.--filter=@get-bb/plugin-sdk: 127 pass (typecheck, bundled-types, package-exports, doc tests).bridge.recorded-conformance.test.tssuites (codex, claude-code, acp) pass throughreplayFirstPartyRecordedCells.apps/serverecho-provider-canary.test.tspasses (thechildRefchange does not affect its assertions).scripts/check-provider-literal-ratchet.mjs): not on this base (it lives on Add the provider-literal ratchet guardrail (G1) #2120); skipped.What a third party still cannot do
json-rpcnor the Claude CLI control protocol:ReplayDialectis those two (audit item 4).@bb/thread-view) is still private, socompareParityfrom a plugin compares events and grammar drops withrows: [].ParityRowProjectoris injected, but nothing public implements it.