Add BACKEND_RUNTIME=process to run without Docker - #51
Open
akozak-gd wants to merge 20 commits into
Open
Conversation
Adds a BACKEND_RUNTIME variable (docker default | process). In process mode the TUI starts the backend as a detached bare-metal uvicorn process instead of via docker-compose, for users who want a Docker-free backend. Docker is the only OS-level isolation boundary today: the agent CLI runs as root in the container and every other control (bash allowlist, workspace path scoping, PreToolUse guard, credential redaction) is in-process and bypassable. Removing Docker removes that boundary, so process mode substitutes Claude Code's OS-level Bash sandbox (bubblewrap on Linux, Seatbelt on macOS) via ClaudeAgentOptions.sandbox — allow-only network, no unsandboxed escape hatch — layered on top of the existing controls (defense in depth), engaged only in process mode. Fail closed: the sandbox is preflighted at two gates (TUI and the run_generation entrance, which returns SANDBOX_UNAVAILABLE); a run never starts agents unconfined on the host. Supported on macOS and Linux; Windows users stay on docker. Docker mode behaviour is unchanged. See docs/backend/backend-runtime.md.
In process mode the OS Bash sandbox only permits writes to the query
cwd (the workspace) and the session temp dir, but SpecFlow redirects
tool caches (npm/pip/go/gradle/…) to {WORKSPACE_BASE_PATH}/caches/…,
which sits outside cwd. Without granting that subtree, package installs
during generation (npm install, pip install, go mod download) would be
denied and every real run would fail — the unit tests missed it because
they mock the SDK.
The SDK intentionally has no SandboxSettings.filesystem field and routes
filesystem write scope through Edit allow-rules, so extend allowed_tools
with Edit/Write rules for the caches subtree in process mode (empty in
docker; strictly tighter than docker, where bash writes are
unrestricted). The "caches" subdir name is now the single constant
WORKSPACE_CACHE_SUBDIR shared by setup_workspace_cache_directories and
the allowlist so the two cannot drift.
In BACKEND_RUNTIME=process the backend is a detached host process that outlives the TUI, so add an in-app way to stop it. A `k` binding on the dashboard and sessions screens (shown only in process mode; hidden in docker where the container stack is the boundary) confirms first — naming how many generations are in flight, since a hard stop interrupts them — then SIGTERMs the process group via the existing local_env.stop_backend_process and exits the TUI. Relaunching re-runs the startup gate, which already detects the backend is down and offers to start it again. Per the STEEL COMMANDMENTS a stop never releases workspaces: generated code is preserved and retry_generation resumes from the last checkpoint, and the confirmation says so. The confirm-and-stop flow lives once on SpecFlowTUI.stop_backend_flow; a _BackendControlScreen mixin carries the shared binding/action for both screens so there is no second implementation.
On first launch, when the runtime isn't pinned (no BACKEND_RUNTIME env var, no saved choice) and nothing is already running (no containers, no bare process), the TUI asks whether to run the backend in docker or process mode and remembers the pick. If something is already up, it infers the runtime from that instead of asking. The choice is a local-launcher fact, not an MCP-server setting: the MCP server only calls backend_url and is indifferent to how the backend is launched. So it is persisted to .specflow-local/backend-runtime (beside the pidfile/log), NOT mcp-config.json. resolve_backend_runtime now reads flag -> BACKEND_RUNTIME env -> saved file -> default docker, and no longer consults mcp-config.json for the runtime. .env.quickstart.example is corrected: a value sitting only in .env is not read by the runtime gate (it must be an exported env var), and the normal path is the chooser.
Adds an `R` binding on the dashboard and sessions screens to move between docker and process runtimes without leaving the app. The flow preflights the target first (fail-closed sandbox for ->process, docker CLI present for ->docker) and refuses without disturbing the running backend if it can't run. It then confirms — naming how many in-flight generations will be cancelled — and hands off to SwitchRuntimeScreen, which performs the switch in a load-bearing order: cancel the active runs via the current backend's DELETE (before teardown, while the API is still reachable), tear down the current backend (stop_backend_process / docker compose down), persist the new choice to .specflow-local/backend-runtime, then start the target and wait for health. Per the STEEL COMMANDMENTS cancellation preserves workspaces and code (marked CANCELLED, not resumable). Both runtimes bind the same host:port and, in the default sqlite setup, share ~/.specflow/db/specflow.db, so cancelled runs stay visible and the TUI keeps polling the new backend automatically. Adds local_env.stop_containers (docker compose down) and local_env.docker_cli_available (PATH preflight).
Switching runtime cancels ALL in-flight generations and restarts the backend, so it belongs on the sessions overview rather than the single-generation dashboard. Move the `R` binding and action_switch_runtime off the shared _BackendControlScreen mixin onto SessionsScreen; `k` stop backend stays shared by both screens. The switch flow itself is unchanged.
The runtime is a local-launcher fact resolved from the env var / .specflow-local/backend-runtime, never from mcp-config.json, so an editable BACKEND_RUNTIME field in Settings wrote to a location nothing reads. Remove it from EDITABLE_KEYS and add it to the purge list so any stale value an older build left in the env block is swept out on save.
Adds a header sub-title indicator ("docker runtime" / "process
runtime") so the current backend runtime is visible on every screen.
Set once the startup gate finalizes the runtime (pinned, inferred, or
chosen) and updated after a successful runtime switch. The value is
passed explicitly rather than re-resolved, since at startup it may be
inferred from what's already running and not yet saved.
R sat one shift-key away from `r reload`; switch-runtime cancels all in-flight runs and restarts the backend, so a mistype is costly. Move it to `t` (toggle) and update the help text, docs, and test docstring. Addresses PR #51 review feedback.
The bare-metal backend is a machine-wide singleton (one uvicorn on one port, backed by the shared ~/.specflow/db/specflow.db), so its remembered runtime choice (backend-runtime) and pidfile (backend.pid) belong in the per-user ~/.specflow home — not per-checkout .specflow-local. This lets `stop` / `switch runtime` from any clone find the one running backend instead of spawning a duplicate on the same port. The launch log stays per-project at .specflow-local/backend.log. The machine-wide helpers now take an injectable `home` (default Path.home()) instead of `root`; the ~/.specflow dir name is defined once in local_env (SPECFLOW_HOME_DIRNAME) and reused by mcp_clients. Addresses PR #51 review feedback.
Move the process-mode backend lifecycle (pidfile/log paths, start/stop/ detect, env build, sandbox preflight, run-process CLI) out of local_env into services/local_backend_process.py. local_env keeps runtime selection (BackendRuntime enum, saved-choice read/write) and the shared filesystem/ health helpers; the new module depends one-way on it. Callers (TUI, Makefile) and the moved tests are repointed accordingly. Also fixes a stop-process bug: the Makefile target passed $(CURDIR) to stop_backend_process, which after the pidfile moved to ~/.specflow would have looked in the wrong place and never found the running backend. Addresses PR #51 review feedback.
Contributor
|
This is serious. Just after pressing: "Yes" for starting TUI in process mode: I think if we need to write any Specflow related files - we shouldn't do it in root: |
The quickstart .env ships container mount targets (WORKSPACE_BASE_PATH= /workspaces, AGENT_LOGS_BASE_PATH=/agent_logs), and build_process_backend_env used setdefault, so those container paths shadowed the host defaults. In process mode the backend then tried to create /workspaces under the read-only filesystem root and crashed (OSError 30). Override the three container mount points (WORKSPACE_BASE_PATH, AGENT_LOGS_BASE_PATH, SQLITE_DB_PATH) unconditionally with the host equivalents of docker-compose's bind-mount defaults; genuine non-mount choices like DATABASE_TYPE stay respected. AGENT_LOGS_BASE_PATH was not handled before and would have crashed the same way on first log write. Reported on PR #51.
Contributor
Author
|
Good catch — fixed in 27d33a8. Root cause: the quickstart Process mode now overrides those three mount points unconditionally with the host equivalents of compose's bind-mount defaults ( To your point about never writing under |
| """ | ||
|
|
||
| BINDINGS = [ | ||
| Binding("y", "yes", "start"), |
Contributor
There was a problem hiding this comment.
Small Change - but from UX I think it's useful. I would keep the same letter as binding as Label:
start -> s
quit -> q
especially when on previous screen we had docker -> d and process -> p
so I got used to pressing the same key as label
Contributor
Author
There was a problem hiding this comment.
Done in 7692af1 — rebound both gate screens to s (start) and q (quit), matching the labels and consistent with the runtime picker's d/p. Prompts updated to [s] start [q] quit (and the retry variants).
| # env var / .specflow-local/backend-runtime, never from mcp-config.json (the | ||
| # MCP server just calls the backend and is indifferent to how it is launched), | ||
| # so it must not be editable here. See cli.resolve_backend_runtime. | ||
| _LEGACY_EDITABLE_KEYS: list[str] = [ |
Contributor
There was a problem hiding this comment.
do we really need legacy keys if we don't have any "prod" users?
Contributor
Author
There was a problem hiding this comment.
You're right — removed the whole legacy-purge mechanism in 5d52761. It only existed to clean up keys older released builds wrote into mcp-config.json, which don't exist pre-prod. The one key it still targeted (BACKEND_RUNTIME) can only appear via an intermediate commit of this branch and is never read from mcp-config anyway (resolve_backend_runtime uses the env var / ~/.specflow/backend-runtime, and the launcher never loads the mcp-config env block into its own environment), so purging it bought nothing. Normal save/load of the editable keys is unchanged.
| # it lives on this overview screen — not the single-generation dashboard. | ||
| # Bound to "t" (toggle) rather than "R": a shift-of-"reload" is too easy to | ||
| # mistype for so destructive an action. | ||
| Binding("t", "switch_runtime", "switch runtime"), |
Contributor
There was a problem hiding this comment.
I also think that we should move:
- connect_client
- settings
- switch_runtime
- stop_backend
to only the Main Dashboard with Sessions List
The Session Detail should have only controls which control this specific generation and not the whole APP _ I don't even know where this is setup but because those two screens Bindings are somehow blended
Contributor
Author
There was a problem hiding this comment.
Agreed, and fixed in 7692af1. The single-generation dashboard now carries only generation-scoped controls (retry, cancel, clear ws, open ws, events, report). The app-wide controls — settings, connect client, stop backend, switch runtime — now live on the sessions overview only: DashboardScreen no longer extends _BackendControlScreen and its duplicate settings/connect_client bindings + methods are gone. (switch_runtime was already sessions-only.) The "blended" bindings were the shared _BackendControlScreen base — it's now documented as the sessions-overview base and is the single place app-wide backend controls are defined.
Contributor
|
Bigg Issue again with he workflows directory - this Time After all 3 generations finished correctly: agent_logs/est-08e498b353ba/generation-session-workflow/2026-07-28_16-38-29.log 
|
Archival crashed in process mode: ARTIFACTS_BASE_PATH was a second hardcoded container default (/workspaces/artifacts) that the earlier WORKSPACE_BASE_PATH override didn't reach, so artifact_store.archive_workspace tried to mkdir under the read-only filesystem root (OSError 30) after generations finished. Derive it from WORKSPACE_BASE_PATH in the model-validator (alongside CLAUDE_CODE_TMPDIR_PATH) so the process-mode host-path override cascades and there is a single source of truth. The default is byte-identical (/workspaces/artifacts) when WORKSPACE_BASE_PATH is unset, so docker behaviour is unchanged. Reported on PR #51.
Contributor
Author
|
Fixed in d5526a1 — thanks for catching this after a full run. Same read-only- Rather than patch another env var (whack-a-mole), I made Good news on data safety: the log shows all three workspaces pushed to I also audited the other container-path settings while here: |
_LEGACY_EDITABLE_KEYS existed to sweep keys older *released* builds wrote into mcp-config.json. Pre-prod there are no such builds: the only key it still targeted, BACKEND_RUNTIME, can only appear via an intermediate commit of this branch, is never read from mcp-config (resolve_backend_runtime uses the env var / ~/.specflow/backend-runtime and _configure_env never loads the mcp-config env block into the launcher process), and is therefore inert. Remove the constant, the sweep in save_env, and the two tests that asserted it. Normal save/load of EDITABLE_KEYS is unchanged. Addresses PR #51 review feedback.
…bels Two review points from PR #51: - Start/quit gate keys now match their labels — s (start) and q (quit) on StartContainersScreen and StartBackendProcessScreen, consistent with the runtime picker's d/p. Prompts updated to [s]/[q]. - The single-generation dashboard now carries ONLY generation-scoped controls (retry, cancel, clear ws, open ws, events, report). App-wide controls (settings, connect client, stop backend, switch runtime) live on the sessions overview only: DashboardScreen no longer extends _BackendControlScreen and drops its duplicate settings/connect_client bindings and methods. Tests updated: gate key presses, the two stop-backend binding tests moved onto SessionsScreen, and the binding-location test now asserts all app-wide actions live on Sessions and none on Dashboard. Addresses PR #51 review feedback.
The start/retry gate prompts ended with a `[s] start [q] quit` tail. Static renders with Rich markup on, so `[s]` opened a strikethrough span and crossed out the labels. The Footer already shows those keys, so drop the redundant tail and let the Footer own the hints.
mkonopelski-gd
approved these changes
Jul 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a process backend runtime so the TUI can run the backend as a detached bare-metal uvicorn instead of docker-compose, with the default still docker. Because the container was the only isolation boundary, process mode substitutes Claude Code's OS-level Bash sandbox (bubblewrap on Linux, Seatbelt on macOS) and fails closed when that sandbox cannot initialise, so agents never run unconfined on the host.
Entrypoint
Start at
BackendRuntime/settings.BACKEND_RUNTIME(backend/app/core/enums.py,config.py), thenget_agent_sandbox_settingsinbackend/app/agents_sandboxing/os_sandbox.py, whichagent_query(backend/app/services/claude_code.py) passes asClaudeAgentOptions.sandbox. The host-side launch/stop/detect lives inmcp_server/services/local_backend_process.py, wired into the TUI runtime chooser / switch-runtime / stop flows inmcp_server/tui/app.py. Seedocs/backend/backend-runtime.mdfor the end-to-end flow.Details
Fail-closed is enforced at two gates that share one check: the TUI preflight and the
run_generationentrance, the latter via the newSANDBOX_UNAVAILABLErejection code. In process mode the sandbox write scope is widened withEdit/Writerules over the workspacecaches/subtree so npm/pip/go/gradle installs aren't denied (still strictly tighter than docker, where bash writes are unrestricted).The bare-metal backend is a machine-wide singleton, so its pidfile and the saved runtime choice live under
~/.specflow(alongside the shared SQLite db and connected-tools config) while the launch log stays per-project in.specflow-local; the process-control surface is its own module (local_backend_process) split fromlocal_env.Note: the diff includes agent-error-surfacing changes from #53 that were pulled in via a
mainmerge — those are not part of this PR's feature.