Support callable backends and streaming in the exec tool - #46
Merged
Conversation
The exec tool routed every call through the runtime's command path, which only carried a command string and returned drained stdout and stderr. The JavaScript backend accepts a structured input value and returns a structured result, and both command and module backends now take per-run environment variables. The tool could express none of this. Add env and input arguments to the exec tool's input schema and forward them to the runtime. Carry the backend's structured return value out on a result field, present only when the backend produced one. Mark backends callable through ExecBackendDescription so the tool can reject input for a non-callable backend before it reaches the wire, returning the same message the runtime would raise. Surface the callable backends in the tool description so the model knows which ones run their command as module source and read a value back.
The exec tool drained the runtime handle to a single aggregate and returned one object, so the model saw a command's output only after it finished. The runtime already exposes a live event stream, and the AI SDK lets a tool's execute function yield a sequence of results. Turn execute into an async generator. When the runtime handle is async-iterable, iterate it and yield a running snapshot on each stdout or stderr chunk, then a terminal snapshot once the exit event lands; a callable backend's return value rides the final snapshot. Fall back to draining result() when the handle is not iterable, which keeps non-streaming callers working. Running snapshots carry a null exit code so consumers can tell progress from completion. Export ExecStreamEvent, ExecRuntimeHandle, and ExecToolOutput for consumers that build handles or handle the streamed output.
The streamed event model carried a callable backend's return value on its own result event, separate from the exit event that reports the process exit code. The two settle at the same instant: the runtime only emits a result when the run exits zero, and it emits the two together as one terminal batch. Splitting them across two events forced a consumer to collect both and correlate them to learn the outcome of a single run. Fold the value onto the exit event, where it belongs alongside the exit code. The tool still accepts the standalone result event so it keeps working against the current wire, which has not yet been consolidated; consuming both settles the value the same way. Once the wire carries the value on its exit frame the standalone event can go.
The exec tool learned which backends were callable from a `callable` flag on each backend description the caller passed to createAITools. That duplicated a fact the Workspace already holds: it derives the callable set from each backend's own `callable` flag at construction. Declaring it a second time on the tool let the two drift. Expose `isCallable(id)` on the runtime and have the tool ask it, dropping `callable` from the backend description. The runtime answers from the same set that guards its own structured-input check, so the tool and the runtime can no longer disagree about which backends accept input.
The exec tool's ExecStreamEvent carried a standalone result variant alongside the exit event. The runtime folds a callable backend's return value onto its exit event, so no handle the tool consumes emits a separate result event; the variant and the loop branch that read it were dead while still exported for callers to build against. Drop the variant, read the value only from the exit event, and fold the two result tests into one that reflects the wire.
Streaming yielded a snapshot of the whole accumulated output on every chunk, and each snapshot re-encoded that whole buffer to enforce the display cap. A chatty command was quadratic in bytes encoded and in bytes pushed into the model stream, and the accumulator grew without bound regardless of the cap. Coalesce running snapshots to at most one per 100ms, so a chunk-heavy command yields on a wall-clock floor instead of once per chunk; the terminal snapshot always fires. Accumulate each stream through a bounded buffer that counts total bytes incrementally and retains at most streamMaxBytes (512 KiB) of head text, so the truncation marker still reports every byte seen while memory stays bounded and the per-snapshot render no longer re-encodes the entire buffer. The coalescing clock is injectable for deterministic tests.
The runtime's exit event carries its process exit code on `code`. The tool's ExecStreamEvent still read it from `value`, so a streamed exit resolved its code to undefined. Rename the exit variant's payload to `code` and read it there, matching the event the tool consumes.
The tool declared its structured input as z.unknown(), which serializes to an empty JSON Schema that some providers reject under strict function calling, and pushed a non-serializable input past the tool boundary to fail later inside the backend. The runtime call then needed a cast because unknown is wider than the backend's value type. Describe input with a recursive JSON-value schema so it validates at the tool boundary, generates a real schema, and types as the backend's value shape, dropping the cast.
Every exec tool test bound a hand-shaped workspace literal, so the ExecWorkspaceLike contract was only ever checked against fakes written to satisfy it. A real Workspace binding lived only in the examples, which is why a type-level break in that contract surfaced as an example failure rather than a package test failure. Add a test that registers an in-process command backend on a real Workspace and runs the exec tool through it. It covers the binding and the assumption underneath the streaming path: that the real runtime handle is async-iterable.
The streaming exec tool ignored the turn's abort signal: aborting the model turn abandoned the iteration but left the backend execution running. It also built the not-callable rejection from its own string copy. Wire the abort signal to the handle's kill so an aborted turn stops the backend, removing the listener once the run settles, and reject a non-callable input with the runtime's exported message. Cover the abort path with a test that asserts kill fires.
commit: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
exectool in@cloudflare/computerran a shell command, waited for it to finish, and returned one object with the drainedstdoutandstderr. That no longer fits the workspace. The JavaScript backend runs a module, takes a structuredinput, and returns a structured value, and both shell and module backends now take per-run environment variables. None of this was reachable through the tool, and because the tool returned only once at the end, a long-running command produced no visible output until it was done.This change adds the two missing pieces: talking to a callable backend, and streaming output as it happens. It builds on the unified execution backend and should be reviewed on top of that change.
A backend is callable when it accepts a structured
inputand returns a structured value. The workspace already knows which backends are callable, so the tool asks the runtime rather than making the caller declare it a second time:The tool gains
envandinputarguments and forwards them to the runtime. A callable backend's return value comes back on aresultfield, present only when there is one. Aiminginputat a non-callable backend is rejected immediately, before the call reaches the wire, with the same message the runtime would raise.The second half restores streaming. The runtime exposes a live event stream, and the AI SDK lets a tool yield a sequence of results rather than a single one. The tool now yields a fresh snapshot on every output chunk, so the model watches
stdoutandstderrgrow in real time. Running snapshots carry anullexit code; the final snapshot carries the real exit code and, for a callable backend, the return value.sequenceDiagram participant Model participant Tool participant Runtime Model->>Tool: exec("npm test") Runtime-->>Tool: stdout chunk Tool-->>Model: snapshot (exitCode null) Runtime-->>Tool: stderr chunk Tool-->>Model: snapshot (exitCode null) Runtime-->>Tool: exit 0 Tool-->>Model: final snapshot (exitCode 0)A single call now emits several outputs as the work proceeds:
{ "exitCode": null, "stdout": "one\n", "stderr": "" } { "exitCode": null, "stdout": "one\ntwo\n", "stderr": "warn\n" } { "exitCode": 0, "stdout": "one\ntwo\n", "stderr": "warn\n" }Streaming engages only when the runtime handle can be iterated; a handle that exposes only a final result is drained into a single snapshot, so non-streaming callers are unaffected. Output is still truncated on whole-character boundaries, and a mid-stream failure yields a structured
errorsnapshot rather than throwing.A callable backend's value settles at the same instant as the exit code, so the tool reads it from the exit event rather than a separate event a consumer would have to correlate. The runtime wire already carries the value on its exit frame; the tool also accepts a standalone
resultevent so it keeps working against any handle shape that still emits one.The tool publishes the types a caller needs to build a handle or consume the output:
ExecStreamEvent,ExecRuntimeHandle, andExecToolOutput.To verify locally, run the tool tests, which cover both halves — from
envandinputforwarding through streamed snapshots, byte-boundary truncation, and a mid-stream failure:npm test --workspace @cloudflare/computer -- src/tools/ai.test.tsThis works on both major versions of the AI SDK the package supports. The async-iterating tool function and the two-argument
Tooltype it relies on are present in the oldest release named in the peer range, so no version bump is needed. A continuous integration matrix that installs each supported major version and runs the tool tests against both would make that guarantee permanent, and is worth adding as a follow up.