Skip to content

Guided install for the vLLM Qwen3.8-27B stack (clone, build, prepare) - #4818

Merged
atomantic merged 3 commits into
mainfrom
claim/issue-4767
Aug 22, 2026
Merged

Guided install for the vLLM Qwen3.8-27B stack (clone, build, prepare)#4818
atomantic merged 3 commits into
mainfrom
claim/issue-4767

Conversation

@atomantic

Copy link
Copy Markdown
Owner

Summary

SETUP_ROWS.vllm.install() always refused with a pointer to the docs, so getting this stack running meant leaving PortOS entirely: clone the upstream compose project, hand-write five .env settings, build, prepare, come back and paste an API key. Four of those settings are load-bearing and each fails in a way that points somewhere other than its cause — the wrong tool-call parser answers normally and silently never calls a tool; a missing WSL2 pin-memory flag becomes a quiet compose crash-loop; the wrong allocator config reads as an OOM that isn't one.

The readiness checklist now offers "Clone, build & prepare vLLM (Qwen3.8-27B) (~30 GB), then start" — the same name-the-payload consent shape MTPLX's weights download already used, so the click itself is the consent. It clones, writes the .env, builds the image, runs prepare, and starts the container, streaming every line. Re-running skips whatever already landed.

install() still refuses. Docker, the NVIDIA Container Toolkit and WSL2 are host-level decisions with driver requirements PortOS cannot judge; this provisions the project on a host already capable of running it.

Two judgment calls

  • WSL2 detection counts win32, not just a matching /proc/version. The issue's scope named the feature doc's grep -qi microsoft /proc/version test, which is right for an operator typing inside the distro. PortOS commonly runs as a native Win32 process driving Docker Desktop, whose engine is a WSL2 VM with no /proc/version to read — keying only on the file would omit the two mandatory WSL2 variables on exactly the host that needs them, producing the silent crash-loop they exist to prevent.
  • On Windows the action refuses unless VLLM_QWEN_PROJECT_DIR is set. The default ~/qwen-serving resolves to a Windows home, and ~20 GB of weights read from the WSL2 VM across a 9p share is a mistake that costs 20 GB to discover and 20 GB to undo. The refusal names the UNC-path fix. Setting the override is itself the operator's explicit placement decision, so it only fires when nothing was configured.

Safety properties

  • An existing .env is never overwritten — only keys it does not already mention are appended, and the key that ends up in effect is the one written onto the seeded providers, so an operator's own VLLM_API_KEY / GPU_UTIL / DFLASH_TOKENS all survive.
  • The API key is stored and never logged — only key names are streamed.
  • Cancellation is checked between steps and never handed to the build or the prepare; killing those mid-flight is what leaves a half-written image layer behind.
  • The WSL2 memory ceiling is detected and named, never raised — wsl --shutdown would take down a PostgreSQL container the install may be using.

Supporting work rolled in

  • The provisioning action is a table (PROVISION_STEPS) rather than a hardcoded pull-start, and rows declare provision: { action, run } explicitly — no implicit default that would mislabel a new row's step as a model download.
  • Actions declare what they cover (installs / starts / provisions) instead of the checklist matching substrings of their names. The old setup.action?.includes('pull') silently skipped any runtime whose provisioning step isn't a download.
  • The readiness payload carries provisions, so the client stops re-deriving it from the action name.
  • Orchestration lives in services/vllmQwenManager.js beside mtplxServerManager / llamaServerManager, keeping the registry a table.
  • readRuntimeWeights skips runtimes this platform can't run — a Mac with Docker no longer sweeps for an RTX 3090 project every minute.
  • .env writes go through atomicWrite (it carries a bearer token).

The manual sequence stays in docs/features/qwen38-rtx3090.md as the fallback and the explanation.

Test plan

  • cd server && npm test — 32803 passing. The 13 failures are pre-existing on an untouched main checkout: 12 in imageTo3d/trellis2NormalBake.test.js (shells out to a Python venv absent on this machine) and 1 in askService.test.js, both verified failing on main.
  • cd client && npm test — the 11 a11yConventions.test.js failures likewise reproduce on an untouched main.
  • New: server/services/vllmQwenManager.test.js (24 cases — clone/build/prepare ordering, .env preservation, key propagation and secrecy, WSL2 variables, the Windows placement refusal, docker-not-answering, cancellation never reaching a build), server/lib/vllmQwenProvision.test.js, plus vllmProjectSetupState and client button cases.
  • cd client && npm run lint — clean.
  • Reviewed by the configured local reviewer (LM Studio) across the full branch diff in 13 context-sized chunks; findings applied.

Not validated end-to-end on real hardware yet — the handler and its refusals are unit-tested, but the ~9.5 GB build and ~20 GB prepare have not been exercised on the 3090 box from an empty project directory. That run is what the issue names as the unblocking step.

Closes #4767

…wen3.8-27B stack (#4767)

Getting the vLLM stack running meant leaving PortOS entirely: read
docs/features/qwen38-rtx3090.md, clone the upstream compose project, hand-write
five .env settings, build, prepare, come back and paste an API key. Four of
those settings are load-bearing and each one fails in a way that points
somewhere other than its cause — a wrong tool-call parser answers normally and
silently never calls a tool; a missing WSL2 pin-memory flag turns into a quiet
compose crash-loop; the wrong allocator config reads as an OOM that isn't one.

The checklist now offers "Clone, build & prepare vLLM (Qwen3.8-27B) (~30 GB),
then start" — the same name-the-payload consent shape MTPLX's weights download
already used, so the click itself is the consent. It clones, writes the .env,
builds the image, runs prepare, and starts the container, streaming every line.
Re-running skips whatever already landed.

install() still refuses. Docker, the NVIDIA Container Toolkit and WSL2 are
host-level decisions with driver requirements PortOS cannot judge; this
provisions the project on a host that is already capable of running it.

Two judgment calls worth knowing about:

- WSL2 detection counts win32, not just a matching /proc/version. PortOS
  commonly runs as a native Win32 process driving Docker Desktop, whose engine
  IS a WSL2 VM with no /proc/version to read — keying only on the file would
  omit the two mandatory WSL2 variables on exactly the host that needs them.
- On Windows the action refuses unless VLLM_QWEN_PROJECT_DIR is set. The
  default ~/qwen-serving resolves to a Windows home, and 20 GB of weights read
  from the WSL2 VM across a 9p share is a mistake that costs 20 GB to undo.

An existing .env is never overwritten: only keys it does not already mention
are appended, and the key that ends up IN EFFECT is the one written onto the
seeded vLLM providers — so an operator's own VLLM_API_KEY, GPU_UTIL or
DFLASH_TOKENS all survive. The key is stored and never logged.

Supporting work rolled in: the provisioning action is a table
(PROVISION_STEPS) rather than a hardcoded pull-start, actions declare what they
cover (installs/starts/provisions) instead of the checklist matching substrings
of their names, the payload carries `provisions` so the client stops
re-deriving it, and the orchestration lives in its own manager module beside
mtplxServerManager/llamaServerManager rather than inside the registry table.
readRuntimeWeights now skips runtimes this platform cannot run, so a Mac with
Docker no longer sweeps for an RTX 3090 project every minute.

The manual sequence stays in the feature doc as the fallback and the
explanation. The WSL2 memory ceiling stays manual too — raising it needs
`wsl --shutdown`, which would take down a PostgreSQL container this install may
be using — so PortOS detects it and names the values instead.
…M provisioning tests (#4767)

- `setupHint` now documents that its parameter is one of the `installs` /
  `starts` / `provisions` axes the action table declares.
- Assert the Docker daemon's own words reach the operator, not just PortOS's
  "not answering" wrapper — the underlying message is what says where to look.
- When a provider write fails, assert the generated key really is in `.env`, so
  the "paste it yourself" instruction the run emits is actually followable.
- A `repin()` helper for the two tests that change platform mid-test.
…4767)

#4778 landed `qwenAgentParsers.js` while this branch was open, and its
`vllmExtraArgs()` exists for exactly this transport — the `.env` line the
guided install writes. Read the spelling from that table instead of retyping
it, so a correction there reaches the install without a second edit. Its guard
test enforces this; the two assertions now compare against the table's own
answer rather than a second copy of the string.
@atomantic
atomantic merged commit 8d0891e into main Aug 22, 2026
7 checks passed
@atomantic
atomantic deleted the claim/issue-4767 branch August 22, 2026 04:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Guided install for the vLLM Qwen3.8-27B stack (clone, build, prepare)

1 participant