Skip to content

[Fix] Self-hosted tasks stuck at Booting environment with fetch failed#563

Draft
roomote-roomote[bot] wants to merge 1 commit into
developfrom
fix/docker-worker-trpc-env-boot-02o0x98rgoe7u
Draft

[Fix] Self-hosted tasks stuck at Booting environment with fetch failed#563
roomote-roomote[bot] wants to merge 1 commit into
developfrom
fix/docker-worker-trpc-env-boot-02o0x98rgoe7u

Conversation

@roomote-roomote

Copy link
Copy Markdown
Contributor

Opened on behalf of Matt Rubens. View the task or mention @roomote-roomote for follow-up asks.

What changed

Self-hosted Docker tasks no longer die immediately at boot with fetch failed when the deployment’s public TRPC_URL cannot be reached from the task sandbox.

When DOCKER_WORKER_NETWORK is set, the controller now injects the in-network API alias (http://api:3001) as the worker TRPC_URL instead of forwarding a public reverse-proxy URL (including cases that already point at the api host but still carry a /_roomote-api path). Local Docker without a control network still rewrites localhost to host.docker.internal.

Spawn also fails fast if required launch env (AUTH_TOKEN, TRPC_URL, R_APP_URL) is empty, and logs the resolved worker TRPC_URL plus injected env key names (not secret values).

Operator note: TRPC_URL / AUTH_TOKEN are still passed via docker exec -e, so they intentionally do not appear in docker inspect Config.Env or a later plain docker exec shell. Missing those surface values alone does not mean spawn skipped injection. Deploy the controller image/service that contains this change (worker image rebuild alone is not enough).

Supersedes the approach in #556 with stricter control-plane URL normalization and launch env guards.

Why this change was made

With a control network, egress policy blackholes the Docker bridge gateway so sandboxes cannot hairpin through the public edge. Production compose defaults TRPC_URL to https://$ROOMOTE_APP_DOMAIN/_roomote-api, which workers cannot reach, producing Job <unknown> failed: fetch failed and tasks stuck at Booting environment.

Impact

Self-hosted Docker task sandboxes can authenticate to the API over the task network and leave boot successfully when the control plane isolates egress. No change to Modal/other providers. Secrets stay off container Config.Env.

Linked work items

Closes #555

Self-hosted sandboxes with DOCKER_WORKER_NETWORK blackhole the docker
gateway, so public TRPC_URL values fail with fetch failed at boot.
Also fail fast when required launch env is missing and log injected keys.
@roomote-roomote

roomote-roomote Bot commented Jul 18, 2026

Copy link
Copy Markdown
Contributor Author

No code issues found. See task

Reviewed 4f5f399

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Worker containers missing TRPC_URL/AUTH_TOKEN env vars — task stuck at 'Booting environment'

1 participant