test: keep a live jspm outage from redding the required CI job

[vivek7405]

Problem

The required Unit + integration (node --test) CI job fetches from the live jspm CDN (api.jspm.io / ga.jspm.io), so a jspm outage or rate-limit reds CI on a pull request that has nothing to do with vendoring.

Proven on PR #1149 (a documentation-only change touching five prose files): the job failed on a blanket .webjs/ ignore is healed so the pin is committable in test/vendor-cli/vendor-cli.test.mjs:176, asserting assert.equal(code, 0) at L187 when webjs vendor pin exited 1. A re-run of the IDENTICAL commit (91205d71, no code change) then passed all ten checks. During the same window, a local webjs boot logged could not vendor 2 packages via jspm (status 401), and a direct curl to ga.jspm.io returned 200 again shortly after, so the outage was transient and external.

This is a recurrence of the class that #312 already established (De-flake the jspm install-order vendor test (live-CDN dependency), closed 2026-06-03, which reported the vendor tests flaking roughly 1 run in 4 across 18 PRs). #312 fixed ONE instance, in packages/server/test/vendor/vendor.test.js. The root cause, a required CI job depending on a third-party CDN's availability, survives in test/vendor-cli/vendor-cli.test.mjs. The cost is real: a false red blocks a merge, and because the project gates merges on green CI, it either burns an investigation (it burned one on #1149) or trains people to wave red checks through.

Design / approach

The escape hatch already exists and is documented in the header comment of both network-gated files: WEBJS_SKIP_NETWORK_TESTS=1 flips NETWORK_OK to false and skips every live-CDN test. CI never sets it (grep -rn 'WEBJS_SKIP_NETWORK_TESTS' .github/workflows/ scripts/ returns nothing), so every PR run goes to the network. Three directions, not mutually exclusive:

1. Decouple the test from the network where the network is incidental. The #448 block that failed is the strongest case. Its property under test is gitignore HEALING, which is pure local filesystem behaviour: does .webjs/vendor/importmap.json stop being ignored after a pin. It only touches jspm because vendor pin resolves specifiers as a side effect. Either assert the healing without requiring pin to have succeeded against the CDN, or drive the test through a path that needs no network (a --download fixture, or a stub resolver). This fixes the specific flake and leaves genuine CDN-contract coverage alone.

2. Set WEBJS_SKIP_NETWORK_TESTS=1 on the required CI job and cover the live CDN in a separate, non-blocking scheduled job. A third party's uptime then cannot block a merge, while real jspm-contract regressions are still caught daily. Note the tradeoff: a jspm-side breaking change would surface later than it does today.

3. Tolerate a CDN failure rather than hard-failing. If vendor pin fails for a network reason (a 401, a 5xx, a timeout, as opposed to a logic error), degrade the assertion to a skip. De-flake the jspm install-order vendor test (live-CDN dependency) #312's own acceptance criteria asked for exactly this shape: "If the test still touches the network, a CDN hiccup degrades to a skip or a tolerant assertion, never an order-dependent hard fail."

Option 1 plus option 2 is the recommended combination. Option 1 alone leaves the other live-CDN tests able to red the required job on the next jspm hiccup.

Implementation notes (for the implementing agent)

Where to edit:

• test/vendor-cli/vendor-cli.test.mjs: the gate is NETWORK_OK at L22 (!process.env.WEBJS_SKIP_NETWORK_TESTS). The failing block is the describe('webjs vendor pin makes pins committable (#448)', { skip: !NETWORK_OK }) at L165, and the failing assertion is assert.equal(code, 0) at L187 inside the test starting at L176. makeApp() at L39 builds the temp app (it symlinks the repo node_modules and writes an app/page.ts importing picocolors, which is the specifier jspm is asked to resolve). Other network-gated tests in this file: L65, L78, L85, L110.
• packages/server/test/vendor/vendor.test.js: the same NETWORK_OK gate at L404, with roughly a dozen live-CDN tests (L406, L412, L420, L556, L602, L878, L913, L989, L1021, L1068, L1087). This is the file De-flake the jspm install-order vendor test (live-CDN dependency) #312 already touched, so read what it did there before changing its shape again.
• .github/workflows/ci.yml: the Unit + integration (node --test) job, which runs npm ci then npm test (npm test is node scripts/run-node-tests.js, see the root package.json). This is where an env: WEBJS_SKIP_NETWORK_TESTS: 1 would go, and where a separate scheduled live-CDN job would be added.
• scripts/run-node-tests.js if the runner needs to know about a network-test split.

Landmines / gotchas:

• Do not simply delete the live-CDN coverage. The vendor resolver's whole job is to talk to jspm, and dogfood: importmap per-package resolution yields inconsistent dep graph #446 / dogfood: webjs vendor pin writes to gitignored .webjs/vendor (opt-in pins not committable) #448 regressions were found by tests that really hit it. The goal is that a CDN hiccup cannot red a REQUIRED check, not that the contract goes untested.
• De-flake the jspm install-order vendor test (live-CDN dependency) #312 is the prior incident. Read it before designing (it is closed, and its acceptance criteria are directly reusable here). Do not re-fix vendor.test.js's install-order test; that one is already done.
• vendor pin failing is sometimes CORRECT. pinAll: refuses to write empty pin file when every install fails (packages/server/test/vendor/vendor.test.js:989) asserts a failure path on purpose. Any change that makes pin failures non-fatal must not weaken that assertion, and must distinguish a network failure from a logic failure rather than swallowing both.
• Verify the premise before implementing. Reproduce a CDN failure deliberately (point the resolver at an unreachable host, or force a non-200) and confirm the test currently hard-fails, rather than assuming the mechanism from this issue's description.
• A worktree without node_modules cannot run these tests. makeApp() symlinks the repo root node_modules into the temp app, so a fresh worktree needs an install or a symlink first (see AGENTS.md on worktrees and webjs doctor dogfood: a fresh git worktree can't resolve @webjsdev/* (no node_modules) #954). Related trap seen while diagnosing docs: explain the swr cache option on GET server actions #1149: if the worktree's node_modules is symlinked to another checkout, the @webjsdev/* workspace links resolve into THAT checkout's packages/, so a dirty sibling checkout silently changes local test results. Diagnose against a pristine worktree at the base commit before concluding anything from a local run.

Invariants to respect: no bundler or build step may be introduced (AGENTS.md, no-build is permanent). The vendor pin file must stay committable, which is the #448 behaviour the failing test guards, so whatever replaces the assertion must still prove the .gitignore exception is written.

Tests + docs surfaces: this is test/CI infrastructure, so the unit layer is the layer. If the CI job configuration changes, note the split in framework-dev.md (monorepo dev commands and CI) so a contributor knows why a live-CDN test is skipped locally or in the required job. No public API surface changes, so no docs-site or AGENTS.md API update is expected.

Acceptance criteria

• A simulated jspm failure (unreachable host or forced non-200) does NOT fail the required Unit + integration CI job
• The #448 gitignore-healing property is still asserted, and still fails if the .webjs/vendor/ exception stops being written
• A counterfactual proves the healing assertion actually fires (remove the exception-writing code, the test goes red)
• Live jspm contract coverage still runs somewhere (a scheduled or opt-in job), and its failure is visible rather than silently skipped forever
• Repeated CI runs on an unrelated PR stay green without a manual re-run