Skip to content

fix(claude-code-agent): end the turn on the terminal ResultMessage, not stream EOF#15

Open
tmatup wants to merge 3 commits into
mainfrom
fix/turn-end-on-result-message
Open

fix(claude-code-agent): end the turn on the terminal ResultMessage, not stream EOF#15
tmatup wants to merge 3 commits into
mainfrom
fix/turn-end-on-result-message

Conversation

@tmatup

@tmatup tmatup commented Jul 11, 2026

Copy link
Copy Markdown
Member

What

When the SDK stream delivers the terminal ResultMessage of a one-shot query(), communicate() now ends the turn immediately — instead of waiting for stream EOF (i.e. CLI process exit). The subprocess is then reaped with a bounded grace (10 s, so it can flush its session file for resume) followed by SIGKILL.

Why

A successful agent run could be scored 0. If the Claude CLI process lingers after emitting its result — a stray child process holding stdio, a slow shutdown flush — the stream never hits EOF, so the turn watchdog eventually hard-kills it and raises TurnTimeoutError. The task is then recorded final_status=ERROR, score=0 even though the agent finished cleanly.

Observed 3× on live eval sweeps (twice yesterday, once today, all at exactly turn_timeout+4s wall):

05:38:42  final assistant message completes; ResultMessage captured
          (subtype=success, stop_reason=end_turn)
05:47:30  watchdog: Turn timeout (1200s) fired — hard-killing subprocess (pid=77)
          → TurnTimeoutError → task scored 0

~530 s of dead waiting on a process whose result was already in hand.

How

  • communicate() message loop: after state.dispatch(message), break when the terminal ResultMessage has been captured (state.sdk_result_summary set). The ResultMessage is by protocol the last message of a one-shot query — nothing follows it but process exit.
  • New _reap_transport_after_result(): bounded grace (default 10 s) for the CLI to exit on its own — killing instantly could lose the session-file tail the SDK's own close() protects (see its #625 comment) — then SIGKILL. No-op in the common already-exited case. Breaking alone is not enough: async-generator finalization is GC-scheduled, and the SDK's anyio scopes swallow cooperative cancellation (same reason the watchdog exists).
  • test_multiple_assistant_turns reshaped to the real one-shot protocol (single terminal ResultMessage; previously it interleaved two mid-stream ResultMessages, a shape the CLI never produces in one-shot mode).

Tests

  • New: test_communicate_ends_turn_on_result_message_when_cli_lingers (the regression: stream yields result then hangs — must return a clean TurnRecord, not TurnTimeoutError), test_reap_after_result_kills_lingering_process (grace-then-kill + no-op paths).
  • Full suite: 3313 passed, 10 skipped.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LrwDtggu1dG9WKAsgvxfnq

…ot stream EOF

A CLI process that lingers after emitting its result (stray child holding
stdio, slow shutdown flush) kept the SDK message stream open until the
turn watchdog hard-killed it — converting a FINISHED, successful agent run
into a 0-score TurnTimeoutError. Observed three times on live sweeps
(dispute-analyst 2x, lowcode-agent 1x): final assistant message + result
(subtype=success, stop_reason=end_turn) captured ~530s before the 1200s
watchdog fired.

The turn is semantically over when the one-shot query()'s ResultMessage
arrives — nothing follows it except process exit. Break the message loop
there, and reap the subprocess with a bounded grace (it may still be
flushing its session file) then SIGKILL, since generator finalization is
GC-scheduled and the SDK's anyio scopes swallow cooperative cancellation.

test_multiple_assistant_turns is reshaped to the real one-shot protocol
(a single terminal ResultMessage); its old stream interleaved two
mid-stream ResultMessages, a shape the CLI never produces in one-shot mode.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LrwDtggu1dG9WKAsgvxfnq
@github-actions

github-actions Bot commented Jul 11, 2026

Copy link
Copy Markdown

Claude finished @tmatup's task in 1m 22s —— View job


Code Review in Progress

Todo List:

  • Read .github/code_review.md for review standards
  • Read CLAUDE.md for project conventions
  • Review git diff origin/main...HEAD
  • Read full context of changed files
  • Analyze for correctness and edge cases
  • Check cross-file consistency
  • Perform "what's missing" analysis
  • Post structured review feedback

communicate() crossed ruff's PLR0915 statement cap (84 > 80) with the
result-message break added; the per-message logic (deadline guard →
dispatch → terminal-result break) reads better as its own method anyway.
No behavior change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LrwDtggu1dG9WKAsgvxfnq
@tmatup

tmatup commented Jul 11, 2026

Copy link
Copy Markdown
Member Author

Deployment note for the eval images: the images baked at /opt/coder_eval carry 0.7.1 (5c285d6), not current main — overlaying this branch's file into them fails at import (coder_eval.pricing exists only on newer main). The fix is cherry-picked onto 5c285d6 as branch fix/turn-end-on-result-message-071 (no PR — deployment vehicle only), which is what the flow-v2 measurement images now overlay. Both commits are content-identical to this PR.

…t by duck-typing

TaskProgressMessage (sub-agent progress tick) carries BOTH session_id and
usage, so the duck-typed result check misread it as the terminal
ResultMessage. That misread pre-existed silently (corrupted session-id
advance + token backfill whenever a sub-agent ran); once the turn began
ENDING on the terminal result, it truncated every sub-agent-spawning turn
~20s in — the agent was reaped mid-run with zero authored output.

_is_sdk_result_message now matches isinstance(ResultMessage) positively and
rejects the whole SystemMessage family (all task-lifecycle messages subclass
it); the duck-typed fallback remains only for non-SDK test mocks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LrwDtggu1dG9WKAsgvxfnq
@tmatup

tmatup commented Jul 11, 2026

Copy link
Copy Markdown
Member Author

Second commit (0fbed2c): the break exposed a pre-existing misclassification — fixed at its root.

First live run with the turn-end fix truncated every turn that spawned a sub-agent: ~20 s in, zero authored output, empty sandbox. Cause: _is_sdk_result_message duck-types the result as session_id + usage — and the sub-agent lifecycle message TaskProgressMessage carries both. The misread pre-existed (silently corrupting session-id advance and token backfill whenever a sub-agent ran); ending the turn on the "result" merely made it fatal.

The classifier now matches isinstance(message, ResultMessage) positively and rejects the entire SystemMessage family (all task-lifecycle messages subclass it); the duck-typed fallback survives only for non-SDK test mocks. New regression test drives a real TaskProgressMessage through communicate() mid-stream and asserts the turn ends on the true terminal result with the transcript intact. Suite: 3314 passed.

Backport branch fix/turn-end-on-result-message-071 updated with the same commit (050e4ef).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant