Apps subsystem for self-hosted#1333
Closed
RhysSullivan wants to merge 23 commits into
Closed
Conversation
…descriptor extraction
…iring with end-to-end test
…publish door, skills, ui resources)
… HTTP, workflow, ui, SSE, skills)
…r path) Restructure the publish pipeline so all fallible work (discover, bundle, collect, descriptor assembly, ui/skill blob staging) runs before any persistence. The extracted descriptor is written into the snapshot itself at .executor/descriptor.json and the snapshot is committed last; ui/skill blobs and the descriptor pointer are published only after the commit. A projection failure after commit is self-healing: projections are idempotently re-derivable from the committed snapshot via repair / recompute-on-read. Add key-sorted stableStringify for the determinism byte-compare (no more property-order luck), per-entry ModuleSourceRef provenance, and a toolchainRef in the descriptor.
…idge The author's workflow body now runs inside the QuickJS sandbox (same isolation as tool handlers), not in-process via new Function. The WorkflowRunner seam's start/resume/signal take DATA (scope, workflow, snapshotId, entryPath, input), never an execute(steps) closure, so the seam can back onto an RPC. A new QuickJS WorkflowDriver loads the pinned bundle from the snapshot and drives one replay; each step.* / db.sql call crosses a serializable WorkflowBridge the SQLite runner services against its journal. Suspend signaling is structured (a typed SuspendMarker, not string-matching an error message) and retryable-vs-fatal is a typed discriminator (StepErrorMarker / DriveOutcome.retryable, not error.includes). The journal schema, retry vocabulary and scheduler are unchanged. The SIGKILL kill test still proves exactly-once across a hard kill, now with the sandboxed orchestrator.
…mance suite Make the HandleBridge dispatch strict (grafted from build A): reject an empty/malformed method path, a reserved root, an undeclared root, an out-of-range fan-out index, or an index on a single-connection role, all as typed errors rather than silent resolution. The tool sandbox invoker also rejects any bridge path other than the one reserved channel. Merge A's richer ToolSandbox conformance cases into B's suite: a db.sql write param round-trip (asserting the SQL text + interpolated params cross intact) and a per-index fan-out case (each connections() element addressed distinctly). Add a focused strict-dispatch unit test.
…host Add the apps source plugin to the self-host plugin list (executor.config.ts) over a boot-time runtime singleton, so published tools resolve into tools.list and invoke through the same policy/audit path as any catalog tool. Register the apps MCP surface (publish door, skills, ui:// resources) on the REAL per-session MCP server via a new onServer extension hook threaded through makeMcpBuildServer / createExecutorMcpServer, instead of only a FakeMcpServer in tests. Replace the in-server notImplementedResolver with a real per-request resolver built from the invoking executor context: it resolves the user's connection and credential at the boundary (ctx.connections + resolveValue, OAuth refresh) and dispatches the upstream call over the request HttpClient. runtime.invokeTool now takes a per-request resolver override that the plugin threads from ctx. The arbitrary method-path to REST-endpoint mapping is a documented remaining gap (the resolver covers the REST convention the daily-brief fixture uses and fails with a typed error naming any unmapped method). Harden the plugin: a missing descriptor / unknown tool at invoke is a typed error, not a silent empty-connections default; formalize the scope<->connection mapping in one place.
…Apps host Add a booted-host wire e2e and a sunpeak widget proof, and fix the MCP-Apps serving shape so a real host can mount published UI. Wire e2e (apps/host-selfhost/src/apps-wire.node.test.ts): boots the real self-host app and drives the whole subsystem over a real MCP client (StreamableHTTP): publish over the MCP door, wire the scope into the catalog, invoke the published tool through the catalog path (the GitHub emulator ledger proves the upstream call), start the workflow with a journal, read the ui:// resource, observe the SSE invalidation, list + read the skill. Shape fixes surfaced by bringing serving up to what a real host expects: - ui resource is now a ResourceTemplate (a fixed URI only matched itself, so resources/read of ui://<scope>/<name> 404'd). - ui resource now serves a complete self-booting text/html document (React + a minimal executor:ui runtime + the compiled component + current scope-db rows, all inlined) instead of raw CJS, via AppsRuntime.getUiDocument and the new ui-shell. - add apps_open_ui, a tool carrying _meta.ui.resourceUri so a host renders the view when it runs. - the scope<->apps-connection mapping is now identifier-native (executor normalizes connection names to camelCase, so the old apps/<scope> form did not survive create and resolveTools could not recover the scope). External integration routing is now live in the running server: the resolver reads an integration's base URL from its registered record and dispatches a published tool's calls through the caller's connection + credentials, threaded into both the catalog invoke and the workflow step.tool path. sunpeak harness (e2e/mcp-apps): mounts the published dashboard in headless replicas of the Claude and ChatGPT MCP-Apps hosts and asserts the widget renders scope-db rows. Uses the latest sunpeak with no patch script.
The apps HTTP routes (publish/invoke/workflow/ui/SSE) were mounted as an extension route outside the protected API and never authenticated. Thread the host's Better Auth identity check into the apps handler (session cookie / bearer / bearer-as-api-key), 401 unauthenticated including SSE. The subsystem singleton reads a live auth ref per request and is fail-closed until it is set.
The mounted document JSON.stringify'd rows/title into an inline <script>; a row containing </script> broke out and executed. Serialize via safeJsonForScript, escaping <, >, &, U+2028, U+2029 as \uXXXX (the safe-JSON-in-script pattern).
Concurrent start/resume/signal could execute an unjournaled step.tool twice. Take a per-run SQLite lease so exactly one driver executes a run; a second waits then replays the journal. Separately, resume/signal now rebuild from the run's pinned snapshot and its stored start-time bindings instead of the latest publish and empty bindings, so a republish mid-run never runs different code or drops credentials.
A missing credential binding silently used conns[0], dispatching with another connection's credential. Match the bound connection by name exactly and fail a missing binding with a typed BindingError (role + surface), making no upstream call. Name the emulator connections after their integration so the default binding resolves (the connection-name == integration-slug contract).
…tion Compare-and-swap the git ref on commit (typed conflict on a raced writer, never a silent clobber) and serialize publishes per scope so head and descriptor pointer always agree. Bound publish payloads (256 files, 1MiB/file, 4MiB total) with a typed diagnostic before any work. Validate cron at publish time (step>=1, bounded ranges) and defensively in the scheduler so adversarial strings like */0 never hang the parser.
…ope dbs Dashed and underscored scopes (my-scope vs my_scope) normalized to the same connection identifier and collapsed to the same scope-db file. Store an explicit connection-name -> scope mapping at connect time (exact lookup, no reverse parse), and encode scope-db filenames reversibly (safe scopes verbatim, others hex under an x- prefix) so distinct scopes never share a database.
In a terminal client the tool overpromised inline rendering. Detect the client's MCP-Apps UI capability: capable clients get the _meta.ui resource link, others get a fallback URL to the authenticated ?document=html variant of the ui route (or fallback_unavailable), with an honest description and structured status.
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
executor-marketing | a17a105 | Commit Preview URL Branch Preview URL |
Jul 06 2026, 07:00 AM |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
executor-cloud | a17a105 | Jul 06 2026, 07:00 AM |
Contributor
Cloudflare previewTorn down — the PR is closed. |
@executor-js/cli
@executor-js/config
@executor-js/execution
@executor-js/sdk
@executor-js/codemode-core
@executor-js/runtime-quickjs
@executor-js/plugin-file-secrets
@executor-js/plugin-graphql
@executor-js/plugin-keychain
@executor-js/plugin-mcp
@executor-js/plugin-onepassword
@executor-js/plugin-openapi
executor
commit: |
Collaborator
Author
|
Superseded by the incremental approach — custom tools first (#1332). Branch stays as reference for the workflow/ui/skills slices. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an apps subsystem for self-hosted: user-authored custom tools, durable workflows, UI views, and skills, published from a per-scope git store and executed behind substrate seams (ArtifactStore, ScopeDb, ToolSandbox, WorkflowRunner, LiveChannel) so the Cloudflare backings can drop in later without touching subsystem logic.
Published tools are catalog citizens (resolveTools/invokeTool through the normal policy path). Publishing works over MCP (
apps_publish) and HTTP; UI views serve as MCP Apps resources with a fallback URL for non-Apps clients; workflows are journal-replay durable with cron schedules.Architecture record and proof commands are in
APPS_DESIGN.md. Test coverage includes seam conformance suites, a SIGKILL crash-resume test, a booted-host wire e2e over a real MCP client, and a headless MCP-Apps mount check (sunpeak) against Claude/ChatGPT host sims.