feat(acp): stream an agent's reply into the channel as it is written - #3263
Open
noah-shipley-centric wants to merge 1 commit into
Open
feat(acp): stream an agent's reply into the channel as it is written#3263noah-shipley-centric wants to merge 1 commit into
noah-shipley-centric wants to merge 1 commit into
Conversation
ACP delivers an answer as a run of `agent_message_chunk` updates. The harness logged each one to `acp::stream` and dropped it, so a channel saw nothing until the agent posted a finished message. Every token was already arriving — it just never left the log. Publish it instead: post once on the first meaningful text, then edit that same message as more arrives. A stream is one message that grows, not a wall of fragments. This works cleanly because edits (kind 40003) are already excluded from CHANNEL_MESSAGE_EVENT_KINDS, so a streamed answer does not turn into hundreds of unread pings. Relaying every chunk would be worse than not streaming, so `stream_flush` holds a coalescing policy: - the first publish ignores the throttle — delaying the first sign of life is exactly backwards, that is when a reader most needs to know the agent is working; - mid-turn edits need both a minimum delta and a throttle interval; - end-of-turn always flushes, so the tail is never dropped; - an empty turn publishes nothing and never blanks an existing message; - bodies are clamped under build_edit's 64 KiB limit, truncating on a char boundary — a rejected edit would strand the message mid-answer, and a naive byte slice would panic on a multi-byte codepoint. The sender is scope-local in the turn driver on purpose: dropping it IS the end-of-turn signal, so success, cancel, idle timeout and agent exit all flush without any path having to remember to. The driver awaits the streamer so the complete reply is on the relay before the turn reports its outcome. Off by default (BUZZ_ACP_STREAM_REPLIES) — this changes what every channel sees. BUZZ_ACP_STREAM_THROTTLE_MS and BUZZ_ACP_STREAM_MIN_DELTA tune it. 18 tests: every policy branch, plus the post-once-then-edit contract driven through an injected publisher, covering the empty turn, the dropped tail, and a failed post — which must not leave the policy believing there is a message to edit, or the reply would vanish into orphan edits. Signed-off-by: Noah Shipley <nshipley@centricsoftware.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
ACP delivers an answer as a run of
agent_message_chunkupdates.handle_session_updatelogged each one toacp::streamand returned:So a channel sees nothing until the agent posts a finished message. Every token was already arriving — it just never left the log. From the room, the agent goes quiet for a minute and then a whole answer appears at once.
Approach
Post once on the first meaningful text, then edit that same message as more arrives. A stream is one message that grows, not a wall of fragments.
This works cleanly because kind 40003 edits are already excluded from
CHANNEL_MESSAGE_EVENT_KINDS— the existing comment says edits "can land after the last human-visible message and would otherwise create phantom unreads". That is exactly the property streaming needs: a streamed answer does not become hundreds of badge pings.The policy (
stream_flush.rs, pure)Relaying every chunk would be worse than not streaming, so publishing is coalesced:
no_tokens_are_lost_between_the_last_edit_and_turn_end).build_edit's 64 KiB limit, truncating on a char boundary — a rejected edit would strand the message mid-answer, and a naive byte slice would panic on a multi-byte codepoint.No clock, no I/O: the caller supplies
now_msand performs the publish.Lifecycle
The sender is scope-local in the turn driver on purpose — dropping it is the end-of-turn signal, so success, cancel, idle timeout and agent exit all flush without any path having to remember to. This mirrors the existing
ReactionGuardpattern.The driver awaits the streamer before reporting the outcome, so the complete reply is on the relay before anything observes "turn complete" — otherwise a caller could see a finished turn against a message still missing its last edit.
Only channel turns stream; a heartbeat has no channel to write into.
Safety
Off by default (
BUZZ_ACP_STREAM_REPLIES) — this changes what every channel sees, so it is opted into rather than inherited.BUZZ_ACP_STREAM_THROTTLE_MS(1000) andBUZZ_ACP_STREAM_MIN_DELTA(24) tune it. With it unset, behaviour is byte-for-byte what it was: chunks are logged and dropped.Tests
18 tests — every policy branch, plus the post-once-then-edit contract driven through an injected
StreamPublisherfake:Notes
Independent of #3192 and #3196 — different files, no shared commits.
Not yet exercised against a live relay; the publish path is covered by the injected-fake seam rather than an end-to-end run. Happy to add an env-gated integration test if you would rather see one before merging.