fix(ci): harden verify/rust-runner against flaky packages.microsoft.com apt source (#2975)

[rysweet]

Steward: CI/GitHub-Actions health — Simard

Standing-goal steward run. Swept the default branch's Actions health, diagnosed
the open failures, and shipped a fix for the one still-unresolved distinct
failure, deduping a stale duplicate PR.

Default-branch failure sweep (diagnosis)

Workflow	Latest default-branch failure	Root cause	Status
Dependabot Updates	run 29375996164 (2026-07-14): dtolnay/rust-toolchain unknown_error	pins targeted diverged per-channel branch HEADs (# stable/# nightly) that Dependabot's shallow clone can't resolve	Already fixed on main by #4077 (pins now e97e2d8c… # v1). No action needed.
verify (pre-commit, e2e-dashboard)	intermittent (issue #2975)	runner image ships packages.microsoft.com apt sources that periodically serve an invalid InRelease (NOSPLIT / "no longer signed") → apt-get update exits 100 → spurious red	Fixed by this PR.
verify, release, advisory-scan, coverage (functional)	none	—	Green.

Verified every SHA-pinned uses: on main now carries a default-branch-reachable
version-tag comment (no diverged per-tool/per-channel pins remain), so the
Dependabot ecosystem is healthy going forward.

This PR — flaky packages.microsoft.com apt hardening (#2975)

scripts/ci-harden-apt.sh (idempotent, non-fatal) strips the Microsoft apt
sources — both sources.list.d/*.{list,sources} and monolithic sources.list —
before any network-fetching apt-get update. Wired into every such consumer:

• .github/actions/rust-runner-prep/action.yml — before the mold sudo apt-get update.
• .github/workflows/verify.yml (e2e-dashboard) — before npx playwright install --with-deps.

(apt-get clean needs no hardening — it fetches no repos.) Our jobs install only
Ubuntu-archive packages (mold, Playwright browser deps), so dropping the Microsoft
sources is safe. The script accepts APT_SOURCES_LIST_D / APT_SOURCES_LIST /
CI_HARDEN_APT_SUDO overrides so its behaviour is testable with no sudo/network.

Dedupe

Supersedes the stale PR #2986 (same steward goal, older instance,
…-1783461322-…), which carried the same idea but is 8 days old, UNSTABLE, and
lacked the regression rigor below. Closing #2986 in favor of this one → one PR
per distinct failure.

---

Merge-ready evidence

1. qa-team scenarios (gadugi-test)

Scenario: tests/gadugi/ci-harden-apt.{yaml,sh} — outside-in; exercises the
script in an isolated fake apt tree (Microsoft-only removal, third-party/Ubuntu
preservation, idempotency), asserts both CI jobs wire it, and runs the Rust guard.

$ gadugi-test validate --file tests/gadugi/ci-harden-apt.yaml
✓ Scenario "Flaky packages.microsoft.com apt source stays hardened" is valid
✓ All 1 file(s) are valid

$ gadugi-test run --directory <dir-with-scenario>
✓ Executing scenario … Flaky packages.microsoft.com apt source stays hardened
Command completed with exit code 0
Test Execution Results:
✓ Passed: 1   ✗ Failed: 0   - Total: 1
✓ All tests passed!

Behavioural assertions (from the scenario run):
OK: removes only Microsoft sources; preserves Ubuntu + third-party ·
OK: idempotent (second run is a no-op) ·
OK: both apt-consuming CI jobs invoke ci-harden-apt.sh ·
OK: ci_harden_apt_wiring regression guard passed · PASS.

2. Docs / changed surfaces

No user-facing surface changes. Changed surfaces are CI-internal only:
scripts/ci-harden-apt.sh (CI helper), two workflow/composite-action wirings,
and two test files. Intent is documented inline in the script header and both
workflow step comments. Internal-only → no external docs update required.

3. quality-audit (≥3 SEEK→VALIDATE→FIX cycles, clean final)

• Cycle 1 — script correctness/security. SEEK: reviewed set -euo pipefail,
  quoting, deb822 vs one-line handling, sudo usage, no injection from untrusted
  input. VALIDATE: behavioural harness (MS removal, non-MS preservation,
  monolithic disable, idempotency) all green. FIX: added env overrides so the
  script is testable without sudo/network.
• Cycle 2 — test robustness. SEEK: the wiring guard used find() first-match,
  which matched the substrings inside the step comments before the real command.
  VALIDATE: 2/3 Rust tests failed — bug caught. FIX: matched the real run commands
  (sudo apt-get update, npx playwright install --with-deps), which don't appear
  in comments → 3/3 pass.
• Cycle 3 — runtime availability + coverage + lint. SEEK: confirmed
  actions/checkout precedes rust-runner-prep in all consumers (verify,
  coverage, advisory-scan) and precedes the e2e harden step, so
  $GITHUB_WORKSPACE/scripts/ci-harden-apt.sh exists at runtime; confirmed all
  network-fetching apt consumers are covered. VALIDATE: cargo fmt --all --check,
  shellcheck (clean), cargo clippy --test ci_harden_apt_wiring -D warnings,
  local pre-commit (rust-only gate + release clippy) and pre-push (race-subset
  test + --all-targets --all-features --locked clippy) all passed. FIX: cargo fmt.

Final cycle clean: zero critical/high; zero medium correctness/security findings.

4. CI

Local mirrors of CI green before push:

• pre-commit hook: rust-only gate ✓, cargo fmt --all --check ✓, cargo clippy --release -D warnings ✓.
• pre-push hook: race-subset cargo test --release ✓, cargo clippy --all-targets --all-features --locked -D warnings ✓.
• cargo test --test ci_harden_apt_wiring → 3 passed; 0 failed.

Full GitHub verify/release CI status is linked on this PR; will be confirmed
100% green (0 failures) before marking ready.

6. Focused diff

6 files, +361/-0: 2 workflow/action wirings, scripts/ci-harden-apt.sh, and 3
test artifacts. No unrelated edits.