Skip to content

Update createAITools() to support callable backends - #42

Closed
aron-cf wants to merge 10 commits into
mainfrom
callable-tools
Closed

Update createAITools() to support callable backends#42
aron-cf wants to merge 10 commits into
mainfrom
callable-tools

Conversation

@aron-cf

@aron-cf aron-cf commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

The exec tool in @cloudflare/computer was built for one shape of
work: run a shell command, wait for it to finish, and hand the model a
single object with the drained stdout and stderr. That shape no
longer fits the workspace. The JavaScript backend runs a module rather
than a shell command, takes a structured input value, and returns a
structured value of its own. Both the shell and module backends can now
take per-run environment variables. None of this was reachable through
the tool, and because the tool only ever returned one object at the very
end, a long-running command produced no visible output until it was
completely done.

This change teaches the tool the two things it was missing: how to talk
to a callable backend, and how to stream output as it happens.

A backend is marked callable through its description, which is how the
tool knows it may accept input and produce a result. The tool adds
env and input arguments and forwards them to the runtime. When a
callable backend returns a value, that value comes back on a result
field, present only when there is one to report. If the model aims
input at a backend that is not callable, the tool rejects the call
immediately with a clear message instead of sending it over the wire to
fail there.

createAITools({
  workspace,
  shell: {
    defaultBackend: "shell",
    backends: {
      shell: { description: "just-bash in a Dynamic Worker" },
      container: { description: "full Linux userland" },
      js: {
        description: "JavaScript module runtime; returns a value",
        callable: true,
      },
    },
  },
});

A callable run passes a module as the command, a structured input, and
reads the module's return value back on result.

// tool call
{
  "backend": "js",
  "command": "export default (input) => ({ doubled: input.value * 2 })",
  "input": { "value": 42 },
  "env": { "API_KEY": "secret" }
}

// final tool output
{
  "command": "export default (input) => ({ doubled: input.value * 2 })",
  "cwd": null,
  "backend": "js",
  "exitCode": 0,
  "stdout": "",
  "stderr": "",
  "result": { "doubled": 84 }
}

Pointing input at a plain shell backend never reaches the runtime:

{
  "command": "echo hi",
  "cwd": null,
  "backend": "shell",
  "error": "Backend \"shell\" is not callable; it does not accept structured input."
}

The second half of the change restores streaming. The runtime already
exposes a live stream of output events, and the AI SDK lets a tool yield
a sequence of results rather than a single one. The tool now consumes
that stream and yields a fresh snapshot on every chunk of output, so the
model watches a command's stdout and stderr grow in real time. A
running snapshot carries a null exit code; the final snapshot carries
the real exit code, and for a callable backend, the return value.

sequenceDiagram
  participant Model
  participant Tool
  participant Runtime
  Model->>Tool: exec("npm test")
  Runtime-->>Tool: stdout chunk
  Tool-->>Model: snapshot (exitCode null)
  Runtime-->>Tool: stderr chunk
  Tool-->>Model: snapshot (exitCode null)
  Runtime-->>Tool: exit 0
  Tool-->>Model: final snapshot (exitCode 0)
Loading

A single call now emits several outputs as the work proceeds:

{ "command": "run", "backend": "shell", "exitCode": null, "stdout": "one\n",     "stderr": "" }
{ "command": "run", "backend": "shell", "exitCode": null, "stdout": "one\n",     "stderr": "warn\n" }
{ "command": "run", "backend": "shell", "exitCode": null, "stdout": "one\ntwo\n", "stderr": "warn\n" }
{ "command": "run", "backend": "shell", "exitCode": 0,    "stdout": "one\ntwo\n", "stderr": "warn\n" }

Streaming only engages when the runtime handle can be iterated. A handle
that only exposes a final result still works: the tool drains it and
yields one snapshot, so callers that hand the tool a non-streaming handle
see no change in behavior. Output is still truncated on whole-character
boundaries to keep replies small, and a failure mid-stream yields a
structured error snapshot rather than throwing.

A callable backend's return value settles at the same instant as the
exit code, so the tool carries it on the exit event alongside that code
rather than on a separate event a consumer would have to collect and
correlate. The runtime wire still sends the value on its own event
today, so the tool accepts both forms and reads the value from whichever
arrives; consolidating the wire itself is a follow up, after which the
standalone event can go.

The tool now publishes the types a caller needs to build a handle or
consume the streamed output: ExecStreamEvent for a single output
event, ExecRuntimeHandle for the handle the runtime returns, and
ExecToolOutput for the snapshot shape.

To verify the behavior locally, run the tool tests:

npm test --workspace @cloudflare/computer -- src/tools/ai.test.ts

The suite covers both halves: forwarding env and input, surfacing
result, the fast local rejection of input on a non-callable backend,
streamed stdout and stderr snapshots, the callable value on the final
streamed snapshot, byte-boundary truncation of streamed output, and a
mid-stream failure.

This works on both major versions of the AI SDK the package supports.
The async-iterating tool function and the two-argument Tool type it
relies on are present in the oldest release named in the package's peer
range, so no version bump is required. A continuous integration matrix
that installs each supported major version and runs the tool tests
against both would make that guarantee permanent, and is worth adding as
a follow up.

aron-cf added 10 commits August 4, 2026 12:28
The JavaScript execution path emitted a structured return value as its
own result event, separate from the exit event that reported the exit
code. A consumer had to collect both and correlate them by execution to
learn the outcome of a single run, even though the runner produces the
value and the exit code together and only when the run exits zero.

Fold the value onto the exit event as an optional result field and drop
the standalone result event. The change runs the length of the module
path: the runner frame, the runner-to-host frame codec, the
persist-and-replay codec, the event type, the backend's ingest and
finalize, and the result drain in the runtime router. The persisted
form now stores the exit payload as an object so a replayed execution
restores the value alongside the code.
DurableObjectState.waitUntil() has no effect on a durable object's
lifetime. It exists for compatibility with the Workers runtime and does
not extend how long the object stays resident. The JavaScript backend
was the only consumer of the workspace waitUntil hook, and it used the
hook to attach a detached execution's completion promise to the object
lifetime, which did nothing.

A JavaScript run advances while its event stream is drained and the
call into the Dynamic Worker stays in flight. That pending work keeps
the object resident on its own. Remove the hook and everything that
fed it: the requiresWaitUntil backend flag, the connect-time guard and
the dead completion registration in the backend, the waitUntil field
on the host bag and on WorkspaceOptions, and the constructor guard. The
worker-javascript example stops passing the hook.
The shell exec wire took a command field while the module execution
path took a source field for the same argument: the program to run.
Unifying the two execution paths behind one interface needs one name
for that argument.

Rename the ShellRPC exec field to source and carry an optional input
value and an optional result on the exit event, matching the module
execution envelope. The shell server maps source onto the runner,
which keeps its own command parameter since at that layer the value is
always a shell command line. The worker shell adapter and the host
shell facade pass source through. Command backends ignore input and
never set result; the fields exist so the shell and module paths share
one event and one request shape.
The runtime forked on a set of command backend ids in four places to
choose between a WorkspaceShell facade that returned an ExecHandle and
a module handle that returned an event envelope. The two shapes carried
the same execution lifecycle and differed only in how the result was
drained and whether a sync bracket ran.

Give both backend kinds the same handle. The workspace now resolves a
single WorkspaceModuleBackendHandle for every backend: module backends
return their native handle, and command backends are presented through
an adapter over their shell that produces the same envelope. The
envelope carries the sync bracket stats for a backend with a remote
store, so the module result drain reports the pushed and pulled counts
that the command result used to carry on its own.

Collapse the runtime's four forks to one path each, delete the command
handle wrapper and the shell router, and drop the command-backend-id
and shell accessors from the router options. Transport-failure
invalidation moves onto the adapter, which classifies a failed exec
dispatch and a mid-stream event error the same way the router did.
The host shell facade exposed an exec()/get() surface that returned a
ReadableStream handle with a result() method, bolted on encoding and
result accumulation, and drained the sync bracket through that result.
Once every backend routed through the unified execution path, nothing
in production called that surface: the runtime consumes an envelope of
raw events and drains the result itself. The facade and its helpers
survived only because their tests kept them alive, and those tests
covered code the runtime no longer runs.

Reduce the class to CommandExecutor: exec() and get() return the raw
event envelope plus the sync bracket stats, and kill()/dispose()
forward to the wire. Delete the handle wrapper, the per-stream utf8
transform, the result drain, and the ExecHandle and ExecResult types
they produced. Retarget the executor tests onto the envelope surface
and move the encoding, result-accumulation, and cancellation-code
coverage onto the runtime, which is where that logic now lives.
Update the prose and interface listings for the execution changes:
the shell wire field is source rather than command and carries an
optional structured input; the exit event carries an optional
structured result; the workspace no longer takes a waitUntil hook and
a JavaScript run stays alive while its event stream is consumed; and
the host command driver is the CommandExecutor, with encoding and
result accumulation living in the runtime.

Touches the capnweb wire contract, the runtime lifecycle and worker
backend notes, the isolate JavaScript guide, the runtime migration
table, and the computer and example READMEs.
The exec tool routed every call through the runtime's command path,
which only carried a command string and returned drained stdout and
stderr. The JavaScript backend accepts a structured input value and
returns a structured result, and both command and module backends now
take per-run environment variables. The tool could express none of
this.

Add env and input arguments to the exec tool's input schema and
forward them to the runtime. Carry the backend's structured return
value out on a result field, present only when the backend produced
one. Mark backends callable through ExecBackendDescription so the tool
can reject input for a non-callable backend before it reaches the
wire, returning the same message the runtime would raise. Surface the
callable backends in the tool description so the model knows which
ones run their command as module source and read a value back.
The exec tool drained the runtime handle to a single aggregate and
returned one object, so the model saw a command's output only after
it finished. The runtime already exposes a live event stream, and the
AI SDK lets a tool's execute function yield a sequence of results.

Turn execute into an async generator. When the runtime handle is
async-iterable, iterate it and yield a running snapshot on each stdout
or stderr chunk, then a terminal snapshot once the exit event lands;
a callable backend's return value rides the final snapshot. Fall back
to draining result() when the handle is not iterable, which keeps
non-streaming callers working. Running snapshots carry a null exit
code so consumers can tell progress from completion. Export
ExecStreamEvent, ExecRuntimeHandle, and ExecToolOutput for consumers
that build handles or handle the streamed output.
The streamed event model carried a callable backend's return value on
its own result event, separate from the exit event that reports the
process exit code. The two settle at the same instant: the runtime only
emits a result when the run exits zero, and it emits the two together as
one terminal batch. Splitting them across two events forced a consumer
to collect both and correlate them to learn the outcome of a single run.

Fold the value onto the exit event, where it belongs alongside the exit
code. The tool still accepts the standalone result event so it keeps
working against the current wire, which has not yet been consolidated;
consuming both settles the value the same way. Once the wire carries the
value on its exit frame the standalone event can go.
The exec tool learned which backends were callable from a `callable`
flag on each backend description the caller passed to createAITools.
That duplicated a fact the Workspace already holds: it derives the
callable set from each backend's own `callable` flag at construction.
Declaring it a second time on the tool let the two drift.

Expose `isCallable(id)` on the runtime and have the tool ask it,
dropping `callable` from the backend description. The runtime answers
from the same set that guards its own structured-input check, so the
tool and the runtime can no longer disagree about which backends
accept input.
@aron-cf

aron-cf commented Aug 4, 2026

Copy link
Copy Markdown
Contributor Author

Closed in favor of #46

@aron-cf aron-cf closed this Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant