[Fix] Self-hosted tasks stuck at Booting environment with fetch failed#563
Draft
roomote-roomote[bot] wants to merge 1 commit into
Draft
[Fix] Self-hosted tasks stuck at Booting environment with fetch failed#563roomote-roomote[bot] wants to merge 1 commit into
roomote-roomote[bot] wants to merge 1 commit into
Conversation
Self-hosted sandboxes with DOCKER_WORKER_NETWORK blackhole the docker gateway, so public TRPC_URL values fail with fetch failed at boot. Also fail fast when required launch env is missing and log injected keys.
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Self-hosted Docker tasks no longer die immediately at boot with
fetch failedwhen the deployment’s publicTRPC_URLcannot be reached from the task sandbox.When
DOCKER_WORKER_NETWORKis set, the controller now injects the in-network API alias (http://api:3001) as the workerTRPC_URLinstead of forwarding a public reverse-proxy URL (including cases that already point at theapihost but still carry a/_roomote-apipath). Local Docker without a control network still rewrites localhost tohost.docker.internal.Spawn also fails fast if required launch env (
AUTH_TOKEN,TRPC_URL,R_APP_URL) is empty, and logs the resolved workerTRPC_URLplus injected env key names (not secret values).Operator note:
TRPC_URL/AUTH_TOKENare still passed viadocker exec -e, so they intentionally do not appear indocker inspectConfig.Envor a later plaindocker execshell. Missing those surface values alone does not mean spawn skipped injection. Deploy the controller image/service that contains this change (worker image rebuild alone is not enough).Supersedes the approach in #556 with stricter control-plane URL normalization and launch env guards.
Why this change was made
With a control network, egress policy blackholes the Docker bridge gateway so sandboxes cannot hairpin through the public edge. Production compose defaults
TRPC_URLtohttps://$ROOMOTE_APP_DOMAIN/_roomote-api, which workers cannot reach, producingJob <unknown> failed: fetch failedand tasks stuck at Booting environment.Impact
Self-hosted Docker task sandboxes can authenticate to the API over the task network and leave boot successfully when the control plane isolates egress. No change to Modal/other providers. Secrets stay off container
Config.Env.Linked work items
Closes #555