Bridges emit typed provider/recovery hints; the runtime acts on the kind (WS4 L1)

[SawyerHood]

WS4 layer 1 of 7 (runtime recovery, process topology, skills, pi, AI services). Design: ideal-state-provider-api.md §"Runtime" — the runtime matches no error text. Additive: every codex-shaped text gate in the runtime stays until L2 deletes it.

What was wrong

The runtime recovered from provider failures by matching provider prose: CODEX_ARCHIVED_SESSION_ERROR_PATTERN gated unarchive-and-retry, CODEX_EMPTY_ROLLOUT_RENAME_ERROR_PATTERN drove a rename retry ladder inside the runtime, and CODEX_ACCOUNT_RESTART_PROVIDER_ERROR_{CATEGORIES,TEXT_PATTERN} decided when to restart a codex process before the next turn. A provider/recovery notification existed in the protocol (#2124) but no bridge emitted it and the runtime only logged it. A third-party bridge had no way to say "this session is archived" or "log in again" that the runtime would act on.

What changed

One payload, two carriers (coordinator decision, 3-lens judge panel):

• A rejected runtime request carries its hint in the JSON-RPC error response as error.data.recovery { kind, message, retryable }. The JSON-RPC id is the correlation; no runtime slot, no ordering rule, no threadId in the payload.
• The provider/recovery notification is for unsolicited hints only (a terminal 401/429 mid-turn). Never both for one event.

Protocol / kit / SDK (packages/provider-bridge-protocol, packages/plugin-sdk)

• errors.ts: providerRecoveryHintSchema, bridgeErrorDataSchema (optional, passthrough). providerRecoveryNotificationSchema is now the hint plus threadId?; same keys, G3 grammar snapshot unchanged.
• createBridgeIo().sendError(id, code, message, data?); BridgeRecoveryError { code, message, recovery } mapped by runBridgeRequest's catch (the ProviderRequestDecodeError → INVALID_PARAMS pattern). JsonRpcResponseError.recovery parsed in settleJsonRpcResponse; a malformed data is a plain failure.
• SDK exports experimental_BridgeRecoveryError, BridgeSendError, providerRecoveryHintSchema, bridgeErrorDataSchema, ProviderRecoveryHint, BridgeErrorData; entry in docs/api_to_audit.md.
• No PROVIDER_BRIDGE_PROTOCOL_VERSION bump (data is optional and additive). No HOST_DAEMON_PROTOCOL_VERSION bump: AgentRuntimeProviderRecoveryHint is unchanged and the daemon's RPC failure errorCode is already a free string.

Runtime (packages/agent-runtime/src/runtime.ts), keyed on kind only:

• sessionArchived (retryable: true): unarchive, retry the request once. The legacy text gate stays beside it (hint wins) until L2.
• authRequired: the rejection becomes AgentRuntimeRecoveryError { code: "auth_required" }; the daemon's getErrorCode reads the string code, so the server sees auth_required with no regex on the way.
• rateLimited: retryable: true → re-send on a bounded ladder (rateLimitRetry.delaysMs, default [2 s, 8 s], injectable so tests run in ms), then a typed rate_limited error; retryable: false → typed error, never re-run.
• staleTurn: steerTurn drops the steer and returns { status: "stale" } (the ACP-gated NO_ACTIVE_TURN branch stays until L2).
• restartRecommended: restart the bridge process and resume every thread it hosted — right away when the hinted thread is idle, otherwise before its next turn. A bridge process hosts every thread of its provider in the environment, so the restart defers while another hosted thread is mid-turn (logged, retried at the next turn): a recommendation never kills another thread's work.
• Rewind staging forks and thread/name/set now pass a recovery context, so an archived source on a rewind recovers through the fork's own rejection.

Codex bridge (plugins/provider-codex)

• sessionArchived as data.recovery on resume (SESSION_NOT_RESTORABLE), fork of an archived source, maintenance, turn/start and steer rejections.

• Unsolicited authRequired / rateLimited on a terminal account error, and the session is marked so the next turn rebuilds the app-server child from its rollout (session/replaced) — the Run Codex app-server per thread #130 account restart moved from the runtime into the bridge that owns the child. Deliberate behavior change in the text fallback (used only when codex reports errorInfo.category unknown):

  	old (runtime, one pattern → restart)	new (bridge)
  pattern	/\b(?:40[19]|429|auth(?:entication|orization)?|credits?|quota|rate[-\s]?limit(?:ed)?|unauthori[sz]ed|usage limit)\b/i	authRequired: /\b(?:40[13]|auth(?:entication|orization)?|unauthori[sz]ed)\b/i — rateLimited: /\b(?:429|credits?|quota|rate[-\s]?limit(?:ed)?|usage limit)\b/i
  401	restart	authRequired + rebuild
  403	no match	authRequired + rebuild
  409	restart	no match — 409 Conflict is neither an auth nor a rate-limit condition, so it no longer triggers a rebuild
  429 / quota / credits / usage limit	restart	rateLimited + rebuild

• The empty-rollout rename retry (50/200 ms ladder) lives in the bridge; exhaustion is a plain error with no hint (a not-ready rollout is no recovery kind). The runtime's copy stays dormant until L2.

• fake-codex-app-server.mjs gains archiveStatePath (archive state across children), turnCursorPath (scripted turns across a rebuilt child), renameEmptyRolloutFailures, and refuses forks of archived sources.

Claude bridge (plugins/provider-claude-code): unsolicited authRequired (authentication_failed, oauth_org_not_allowed) and rateLimited from the SDK's typed assistant error, one hint per turn. The bridge already replaces its CLI child before the next turn; it never asks the runtime for a restart. restartRecommended therefore has no first-party emitter in this layer — kind, runtime action, scripted-echo knob and conformance rule only.

Scripted echo (tests/scripted-echo-provider): failMethods[].recovery, recover:<kind> / recover_now:<kind> prompt tokens, runtime archive state, staleTurn data on its NO_ACTIVE_TURN.

Conformance kit: new rule recovery/session-archived (archive → resume must be rejected with the hint; a bridge with no archive produces no result, like turn/settles-without-activity); the client's error type gains data. Codex and echo reports pin it green.

Docs: docs/provider-bridge-protocol.md "Recovery hints" (action table + carrier rule), docs/provider-plugin-api.md (carrier rule in prose; the G10-mapped block is untouched), docs/api_to_audit.md.

How you verified

New tests (fail before, pass after):

• packages/agent-runtime/src/runtime.recovery.test.ts — one case per kind under provider id fake (the legacy text gates cannot fire), plus the panel's four: two in-flight requests on one thread with different hints, a rewind staging fork of an archived source, thread/fork of an archived source, and a bridge exit as a plain rejection with no action.
• plugins/provider-codex/src/bridge/bridge.recovery.test.ts — terminal 401 → one authRequired notification, child rebuilt (session/replaced) on the next turn; rename ladder pass and exhaustion. bridge.archived-resume.test.ts pins data.recovery on resume and fork.
• plugins/provider-claude-code bridge.test.ts pins exactly one authRequired notification on the SDK auth failure.
• Conformance: recovery/session-archived expected pass in the codex and echo suites.

Oracle (all at --concurrency 4):

• turbo typecheck 11/11; turbo test: provider-bridge-protocol 218, agent-runtime 453, plugin-sdk 127, codex 201, claude-code 323, scripted-echo 1, host-daemon 555, provider-parity 43 — all green.
• Parity pnpm parity --old ~/.bb/parity-old-main --new .: codex 16 passed / 1 skipped, claude-code 13 passed / 1 skipped, zero event and row diffs — the hint is a runtime signal, never a timeline event, so no allowlist entry. Rerecorded lanes: codex archived-resume (the -32002 line gains data.recovery, no inserted line), codex and claude auth-failure (one provider/recovery notification line each).
• A4 corpus: 307 threads, 93,534 rows, zero diffs (rows-last-run.json: diffThreads: []).
• Ratchet --base origin/main: 148 references across 40 core files, unchanged.

Part of the provider-plugin migration (WS4); L2 deletes the runtime text gates this layer makes redundant.

AGENT GENERATED: by Claude Opus 5

[SawyerHood]

Coordinator review — WS4 layer 1 (#2235)

PASS against REVIEW-GATE.md and the carrier decision in specs/WS4-runtime.md. Do not merge: the stack accumulates until the full migration is done.

Verified on the branch

• One payload, two carriers: providerRecoveryHintSchema in errors.ts; bridgeErrorDataSchema { recovery? } for rejections; the notification extends the same hint with threadId? for unsolicited hints.
• Kit: sendError(id, code, message, data?) and BridgeRecoveryError mapped in the runBridgeRequest catch, so a bridge author who throws gets the hint for free.
• Runtime: the five actions switch on error.recovery?.kind. No new branch reads a provider id. The 9 remaining CODEX_PROVIDER_ID references are L2's deletions.
• Tests: 14 cases in runtime.recovery.test.ts, including the four the judge panel required (two in-flight rejections, rewind staging fork, archived fork source, bridge exit → plain rejection).
• No PROVIDER_BRIDGE_PROTOCOL_VERSION or HOST_DAEMON_PROTOCOL_VERSION bump; data is optional and additive.
• Docs: protocol table + carrier rule, plugin-API prose, api_to_audit entry.
• The 409 narrowing is stated side by side in the body. Accepted as a deliberate behavior change.

Oracle (reported, consistent with the diff)

1,921 tests green; parity codex 16/0, claude 13/0 with zero event and row diffs; A4 307 threads, zero row diffs, perf gate clean at load 2.8; ratchet 148/40 unchanged (L2 lowers it).

Process note

The git checkout origin/main -- . incident was reported, replayed, and re-verified before the commit. Good handling.

AGENT GENERATED: by Claude (coordinator)