Front SGLang Qwen3.8-27B as a Hopper/Blackwell local runtime for CoS agents

[atomantic]

Summary

PortOS could not serve Qwen3.8-27B on a Hopper or Blackwell card at all — the only CUDA path was the frozen vLLM + DFlash 2 container on an RTX 3090. The SGLang cookbook publishes no 3090 cell, so the two are complementary rather than alternatives: Ampere 24 GB stays on vLLM, and this adds a third CUDA path beside it on 127.0.0.1:18021.

The launch line is PortOS's own, because SGLang publishes an official image (lmsysorg/sglang:qwen38-27b) but no compose project to inherit flags from. server/lib/sglangQwenRecipe.js is that launch line — a pure table of the cookbook's verified H200 cell plus two Blackwell cells — and the docker-compose.yml is generated from it, so there is one source of truth. A test fails if the feature doc's compose block drifts from the function.

Three flags that fail silently when wrong

• --tool-call-parser — SGLang spells it differently from vLLM for the very same model. Get it wrong and the server starts, answers, and returns raw markup with no tool_calls block: the agent narrates edits it never makes. This is the exact failure the 3090 bring-up hit.

• --reasoning-parser — the same failure mode for the thinking block.

  Both are read from parserFlagsFor('sglang') in server/lib/qwenAgentParsers.js (Encode Qwen-family tool/reasoning parsers in a shared launch-flag table #4778, which landed while this branch was open and which Front SGLang Qwen3.8-27B as a Hopper/Blackwell local runtime for CoS agents #4776 names as the table to consume). Neither spelling is typed in this branch's code — that table ships a guard test failing any tracked JS file that retypes one.

• --mamba-full-memory-ratio — derived (1.201), not left at the cookbook default 0.9, which under-sizes the GDN state pool at CoS prompt lengths and silently clamps max_running_requests.

Hardware gate, before docker

The compute-capability probe decides before anything could pull an image: SM90 → the verified H200 cell, Blackwell splits by VRAM, Ampere is refused with a pointer at the vLLM path, and a wedged nvidia-smi reports "could not read" rather than "no GPU". darwin is refused outright.

Only the H200 cell is transcribed flag-for-flag from the cookbook (via the in-repo research note). The two Blackwell cells carry what that note actually records — NVFP4, --max-running-requests 1 on a 32 GB 5090 — and deliberately omit H200's 32k prefill overrides rather than pinning a number nobody has measured. They are marked verified: false on the record, and the feature doc asks an operator who brings one up to tighten the cell.

Not repeating known bugs

• vLLM-backed OpenCode providers silently drop every generation control #4765 (generation controls silently dropped): THINKING_STYLE.sglang, the client's generationControlsFor, and the lockstep test that walks LOCAL_RUNTIMES all land here. Nothing seeds a thinking default — CoS coding wants it off, but the operator chooses (Run CoS TUI agents on the RTX 3090 via the syv-ai Qwen3.8-27B vLLM stack #4716).
• Local media jobs don't notice the vLLM container holding the GPU #4766 (media jobs ignoring a container that holds the GPU): detectGpuBlockers now covers sglangBacked too. The two probes became one table rather than two copies, so a fix to one can't miss the other.

PortOS never starts the container, never pulls the image, and never downloads weights. Migration 290 seeds disabled OpenCode CLI + TUI wrappers whose API key is optional — SGLang authenticates only behind --api-key, unlike the vLLM stack.

Rebased twice onto main while open, picking up #4778 (the shared parser table, now consumed) and #4767 (which moved the vLLM setup row behind vllmQwenManager and replaced ACTION_LABELS with PROVISION_STEPS/ACTION_CAPABILITIES — the SGLang row keeps its own inline start, since it ships no provisioner).

Test plan

• CI green on this commit: Server tests, Client tests and build, DB tests, Windows server unit tests, and both gates.
• Locally, cd server && npm test — 32,924 pass. Two files fail identically on an untouched main checkout and are unrelated: imageTo3d/trellis2NormalBake.test.js (12) and askService.test.js (1). client/src/a11yConventions.test.js likewise fails identically (10) on main.
• New coverage: the H200 launch line is pinned flag-for-flag; the mamba-ratio arithmetic is pinned at the CoS operating point and asserted to exceed the 0.9 default; every cell is asserted to reproduce the shared table's parser fragment and never vLLM's spelling; the 5090 cell is asserted not to inherit H200's prefill chunks; the hardware gate is covered for Hopper / Blackwell / Ampere / unknown-probe / darwin; the project inspector's tri-state hasWeights is covered in all three states with distinct refusal copy; the setup row is asserted to refuse an Ampere card before it reaches inspectSglangQwenProject; and the compose block in the feature doc is asserted equal to the generator's output.

Reviewed

lmstudio (qwen/qwen3-4b-2507) over the branch diff. Its context window is 8k, so the 43k-token diff was reviewed in 31 file-scoped chunks; all 31 returned findings. Two were acted on — documenting why SGLang's stop command omits vLLM's --profile single, and making sglangComposeYaml throw on a recipe with no --port instead of rendering 127.0.0.1:undefined:undefined. The remaining Blocking items were chunking artifacts (e.g. a claim that detectGpuBlockers leaves row undefined, where the chunk boundary had hidden the destructuring loop headers). Rounds 2 and 3 came back clean.

Closes #4776