diff --git a/CHANGELOG.md b/CHANGELOG.md index 757977e..9c7d86a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -92,6 +92,7 @@ repository still gets a decision, never by following the link; no release carrie - :memo: docs(e11): Shape D justifies E11 unconditionally — an encoded adjacency CEL cannot decode (F01) - :memo: docs(e11): Shape D rests on no-recursion alone — the decode barrier was false (F-01) - :memo: docs(e11): delete the false cost claim; reason 1 carries Shape D alone (F3-01) +- :memo: docs(ci): D-159 recorded a residual that does not exist (ORPH-01..04) ### Features - :sparkles: feat(audit): measure REL-03/REL-07 by real mutation run, not by name @@ -127,6 +128,7 @@ repository still gets a decision, never by following the link; no release carrie - :white_check_mark: test(hack): gate that hack/** bash 4+ features declare a floor (BASH32-F01) - :white_check_mark: test(hack): the bash-3.2-clean assertion could not fail (BASH32-F01) - :white_check_mark: test(hack): close two gate fail-opens found by review (BASH32-F01) +- :white_check_mark: test(ci): wire release-verify-test into task check (D-159) ## [0.3.0] - 2026-08-18 ### Chores diff --git a/Taskfile.yml b/Taskfile.yml index a6096e7..8c66062 100644 --- a/Taskfile.yml +++ b/Taskfile.yml @@ -122,6 +122,18 @@ tasks: # without this line an unpinned `cosign verify-blob` ships silently again, # and SECURITY.md's published issuer/identity pair can drift away from it. - task: release-install-cosign-pin-test + # ORPHAN / D-159: `release-verify-test` is the ONLY test of + # hack/release/verify-artifacts.sh — the checksum/version/cosign procedure + # SECURITY.md:61 publishes to release consumers — and it was invoked by + # nothing: not check:, not CHECK_STAGES, not the push-only release-exitgate. + # A gate invoked by nothing is not a gate (D-124). Placed with the other + # release-gate rows; position is not load-bearing (this stage reads no build + # output and nothing downstream reads its output), unlike dogfood-examples, + # which must follow build. Its `readme` mode covers hack/release/README.md, + # which docs-gates does not read at all — so that file's only pin is here. + # Offline in the three modes wired above; see the task definition for why + # mode `all` is not. + - task: release-verify-test # D-124: the AUD-S06 docs truth-lag gates shipped green and invoked by nothing, # which is the same manual-gate defect they exist to close. Sequential (not a # `deps:` entry) — go-task runs deps in parallel and readme_smoke_test.sh builds @@ -290,10 +302,24 @@ tasks: cmds: - bash hack/release/verify-artifacts.sh + # ORPHAN / D-159: the three OFFLINE modes, named one per line so each is + # separately pinned in the exit gate's STAGE_BODY_PINS. The bare + # `bash hack/release/verify_test.sh` this task used to run means mode `all`, + # which adds test_snapshot_pass -> `task release-snapshot`: `go install + # goreleaser@…` on any machine without it (network), a `go mod tidy` before-hook + # that rewrites go.mod/go.sum in the middle of `task check`, `--clean` on the + # developer's dist/, and a five-target cross-compile. None of that belongs in a + # per-commit gate, so REQ-E9-S12-01 stays where the spec already puts it — + # `task release-snapshot && task release-verify`, an L1 release-time step. + # What runs here is REQ-E9-S12-02/03/04: the tamper rejection, the cosign + # skip-when-absent path, and the README pin. All three are offline and finish + # in well under a second. release-verify-test: - desc: "Verify artifact harness gates (REQ-E9-S12-01..03)" + desc: "Verify artifact harness gates, offline modes (REQ-E9-S12-02/03/04)" cmds: - - bash hack/release/verify_test.sh + - bash hack/release/verify_test.sh negative + - bash hack/release/verify_test.sh cosign-skip-when-absent + - bash hack/release/verify_test.sh readme release-changelog-gate-test: desc: "AUD-S02 gate: the CHANGELOG drift gate is wired and fires (REQ-AUD-S02-01/02)" diff --git a/docs/decisions/decisions.md b/docs/decisions/decisions.md index 3c2e3d1..97393f1 100644 --- a/docs/decisions/decisions.md +++ b/docs/decisions/decisions.md @@ -161,3 +161,4 @@ project/process decisions. | D-154 | 2026-08-23 | **Every `hack/**` script that uses a bash 4+ feature declares a PER-SCRIPT version floor via the shared `hack/lib/require-bash.sh`, because one of them was a silent local FAIL-OPEN under stock macOS bash 3.2 (BASH32-F01).** Substance first: `/bin/bash hack/docs/truthlag_pins_test.sh` on macOS printed 20 `PASS` lines, died at its `declare -A ex_s09_prefix_hint=(…)`, never printed its final `OK: all truth-lag pins green` banner — and **exited 0**. Its caller `task docs-gates`, and therefore `task check`, read that as a green gate, and AGENTS.md rule 4 makes a green local `task check` a per-commit precondition; CI is ubuntu/bash 5, so nothing merges through this path, which is precisely why it survived. Mechanism, measured on bash 3.2.57: 3.2 has no associative arrays, so under `set -u` it re-reads `declare -A x=([k]=v)` as an INDEXED array assignment and evaluates the subscript `k` **arithmetically**, hitting "unbound variable"; the shell terminates the script and the exit status is 0. Two theories were disproved en route and are recorded so they are not re-explored: the `trap … EXIT` status-swallow (3.2 propagates 1 through an EXIT trap correctly) and `set -e` (the script runs `set -uo pipefail`). The other two affected scripts fail CLOSED, but only incidentally: `hack/release/verify-artifacts.sh:202`'s literal is EMPTY, so there is no subscript to evaluate and it degrades to `declare: -A: invalid option`, which its `set -e` turns into exit 2 (measured with a `--dist` that reaches line 202 — an unreachable dist exits 1 much earlier); `hack/validate-schemas-stock.sh`'s `mapfile` degrades to "command not found" and it exits 1. Populating an empty literal or dropping `-e` silently converts a closed failure into the open one, so "how this construct happens to degrade" is not a guarantee worth depending on. Options considered: **(a)** one blanket repo-wide floor (rejected — the binding minimum differs per feature: `declare -A` 4.0, `local -n` 4.3, `mapfile -d ''` 4.4; a blanket floor both refuses shells that could run the 4.0 gates and hides which feature actually binds); **(b)** rewrite the three scripts to be 3.2-compatible (rejected — associative arrays and `mapfile` are the right tools here, a rewrite is a large risky change to scripts whose correctness IS the product, and it does nothing about the next gate author); **(c)** copy the existing inline `BASH_VERSINFO` guard from `hack/audit/aud2_exitgate_test.sh:71` into each script (rejected as the *only* measure — it duplicates the explanation three more times and still leaves the next author to rediscover the hazard); **(d) chosen** — a shared `require_bash [.] ` helper, called with each script's own correct floor (4.0 / 4.0 / 4.4), plus a meta-gate. Enforcement, per D-128, is one place: `hack/lint/bash_version_guard_test.sh`, wired as `task check` stage **20** and pinned in `hack/audit/exitgate_test.sh`'s `CHECK_STAGES` in the same commit. It scans every `hack/**/*.sh` for command-position bash 4+ constructs, requires a guard on each, requires the declared floor to be at least the feature's own minimum, and — the assertion that makes the rest non-vacuous — requires the scan to still find all four known feature-using files, so a typo in a detection pattern reds instead of quietly matching nothing. Its own negative control mutates real copies of the three scripts with the guard lines stripped and asserts they are flagged. `hack/audit/aud2_exitgate_test.sh` keeps its inline guard and is accepted by the gate as guarded; adopting the helper there is a follow-up, not this lane. Stated limits: the gate sees only command-position constructs in `hack/**/*.sh` (not `eval`, dynamically built command names, case-modification expansions, or scripts outside `hack/`), and the exit-0 control needs a real bash 3.2 — present on macOS, absent on CI ubuntu, where it SKIPs **loudly** rather than passing silently. Accepted behaviour change: under 3.2, `hack/release/verify-artifacts.sh --help` now refuses instead of printing usage. Revert: delete the guard calls — `task check` then silently stops certifying the docs truth-lag pins for every contributor whose `bash` is 3.2. | | D-156 | 2026-08-23 | **The tier-1 (CEL) ceiling is recorded (`docs/planning/rego-tier-ceiling.md`, E11-S01 / REQ-E11-S01-01+02) and it NARROWS E11: two of the four named shapes are struck outright and a third in part.** Substance first: of multi-pass / cross-manifest / set-difference / graph-relationship (`openspec/specs/later-phases.md:284`), **cross-manifest and set-difference are struck, the "named intermediate" half of multi-pass is struck, and only fold/aggregate-over-a-collection and in-input graph reasoning survive as justification for a second backend.** Measured against the surface this repo actually binds — `internal/core/aggregate/evaluate.go` `newEvalEnv`, eleven frozen predicate-scope variables, **zero extension libraries** — reproduced in a nested throwaway module pinned to the repo's `cel-go v0.31.0` (root `go.mod`/`go.sum` byte-unchanged; nothing added to the dependency graph, which judgment call (d) has not authorised). **What exceeds tier 1: (1) folds.** `sum`, `reduce`, `math.*` and `lists.*` are all `undeclared reference`; `size()` is the only aggregate in the surface, so a rule counting matched changes is expressible and a rule summing a numeric delta across them is not. **(2) Graph reasoning** — the iteration count of a CEL expression cannot be made data-dependent, so depth is syntactic and **unbounded** reachability has no spelling; a bounded `k`-hop form *is* writable and answers a fixed-depth question, not the rule (detailed below, with the refutations of two sharper claims this row previously made). **What is struck, with the CEL that does the job:** registry membership — `string(new) in facts.registry.topics.value` over a `cardinality: set` fact, already shipped green at `examples/packs/service-catalog/.assent/packs/catalog/rules/ownership.yaml:21`; keyed attribute lookup — either a purpose-built provider (`builtin/resource-owner`, exercised in both polarities by `examples/archetypes/referenced-resource-ownership/`) or `facts.

..value[key].field`, which compiles and is lint-clean because `internal/lint/facts_ref.go`'s D-051 shape check permits arbitrary navigation past `.value` and only forbids indexing `facts` itself; set difference — `oldEntry.acls.filter(a, !(a in entry.acls)).size() == 0`; named intermediates — no `cel.bind`, but re-deriving the sub-expression inside each leaf is semantically identical and compiles. **The sharpest finding is a distinction, not a rule:** same-changeset **cross-file** reasoning fails for **input availability**, not expressiveness — the evaluation unit is one file (`change.ChangeSet` is documented "for one file", `internal/change/diff.go:129`; `assent run` strips one `--subject file:`, `cmd/assent/run.go:266`, and diffs it alone, `:293`) and **REQ-E11-S05-01 pins the Rego module to the identical `EvaluationInput`** with the tier fenced to "declared data, no I/O". **A Rego module therefore fails that rule identically, so it is struck from E11's justification too** — the fix is an input decision, which E11's non-goals fence, not a backend decision. Corroborating: the one committed illustration of the escape hatch, `examples/policies/rego/bounded_change.rego`, is **entirely tier-1 expressible** — both `violations` rules are per-change predicates and `examples/policies/declarative/bounded-change.yaml` is the same rule already in the envelope. **The strongest single piece of evidence, and the one that sizes the epic, is the graph shape.** On the `assent run` path document-mode `walkNode` emits a Change **only where two scalars differ** — a sequence on either side makes the whole ChangeSet **opaque → REVIEW** before any rule runs (`internal/change/diff.go`) — and collection-mode `DiffEntries` is reached only from `internal/adoptertest`, so `old`/`new`/`entry`/`oldEntry` and every `changes[i].old`/`.new` bind **scalars**. That does **not** starve a graph rule, because a flat `set` fact still carries structure: `edges: {type: string, cardinality: set}` returning `["orders|billing","billing|ledger"]` is a **fully declarable, in-contract adjacency inside `EvaluationInput` today** — the declaration is legal (`schemas/provider/v1alpha1/response.schema.json`), `fact.value` carries no JSON-Schema type constraint and `ResolveFactsChecked` cross-checks the *declaration* not the value (`internal/provider/resolve.go`), outputs are operator-authored (`internal/provider/config.go`), the `http` transport is live on the plain run path with **no `--checkout`** (`cmd/assent/provider_host.go` `providerCallFor`), and a set fact binds as a CEL list. Tier-1 CEL nonetheless cannot answer the actual question over it: **CEL has no recursion, fixpoint, fold or user-defined function, so unbounded reachability has no spelling** (`transitiveClosure` → `undeclared reference`; the four comprehension macros iterate one level and cannot call themselves). **Stated precisely, because two drafts of this record overreached and both were caught by independent review:** a *bounded* `k`-hop check **is** expressible and does **not** need the string decomposition CEL lacks — over a finite in-input candidate set (a `nodes` set fact, the same construction as `edges`), decode is replaceable by **encode-and-compare**: `nodes.value.exists(m, nodes.value.exists(n, (string(new)+"|"+m) in edges.value && (m+"|"+n) in edges.value && (n+"|"+string(new)) in edges.value))` compiles **and evaluates** under the real `cel.CostLimit(1_000_000)` to `true/true/true/false` over the fixture above, and `nodes.value.filter(m, (string(new)+"|"+m) in edges.value)` recovers an edge's far end as `[billing]`. Both primitives (`+` on strings, `in`) were already in this record's own function census, so the claim that decoding was a second independent barrier was internally inconsistent as well as false. **The real ceiling is `k`, and a third draft got its cost wrong too** — it claimed `O(|N|^(k-1)·|E|)` "exhausts the cost budget on any real graph". Wrong complexity class and wrong conclusion, because the census had omitted the surface's only value binder, `[expr].all(v, …)`: binding each BFS frontier once per level makes cost additive across levels, and measured under the real `cel.CostLimit(1_000_000)` a ring `|N|=50, deg 5` costs 89,551 at `k=10`, 235,297 at `k=20` and 686,317 at `k=50` — roughly linear in `k`, and at `k=50 ≥ |N|` CEL is **not approximating at all, it decides reachability exactly**, in 69% of budget. The practical `k` collapses around `|N|≈200` (`k=10` at 46% of budget, `k=20` exceeds). Recorded because a reader sizing this epic is entitled to ask "would CEL suffice for *our* graph?" and the previous sentence foreclosed that question with an assertion measurement refutes; it is **not** offered as a second reason. **What carries the verdict is one property, now measured rather than recalled: the iteration count of a tier-1 CEL expression cannot be made data-dependent.** `reduce`, `transformList`, `transformMap`, two-var `all(i,x,…)`, `range`/`lists.range` and `cel.bind` are all `undeclared reference`; `for` is a `reserved identifier`; there is no fold, self-reference or generator; so depth is a **syntactic** property, hard-capped by cel-go's parser recursion limit of 250 (`expression recursion limit exceeded: 250` at depth 260, depth 200 still compiles). That a fixed `k` really is an approximation when the data outruns it is measured too: on a 4-cycle the `k=3` form evaluates `false` and the `k=4` form `true`. **Rego answers the unbounded question with `graph.reachable` — pure and deterministic — and that single gap is the whole justification.** **An input that is in contract and available today, over which CEL can only approximate and Rego decides, is exactly the per-rule evidence D-017's gate demanded — so E11 has TWO unconditional justifications (folds and graphs), neither waiting on any open question.** Deliverability caveat, stated because this row makes the same distinction for sub-shape B1: **no provider in the corpus ships an encoded adjacency today** — that one *can* follows from the provider contract, the same inference used to strike B1, not from a shipped example. Options considered: **(a)** record all four shapes as exceeding, the reading D-141's unlock invites (rejected — three are demonstrably expressible and an unfalsifiable ceiling claim is the defect this story exists to prevent); **(b)** strike set-difference **conditionally**, so that a narrowing answer to OQ-35 would return it to the ceiling (rejected on the second pass — both resolutions strike it: extending the entry binding makes CEL express it, narrowing makes it an input-availability failure that REQ-E11-S05-01 hands to Rego unchanged, so a conditional would have left a downstream story waiting on an answer that cannot change the verdict); **(c)** strike cross-manifest and set-difference **unconditionally**, strike half of multi-pass, keep folds and in-input graphs **both unconditionally** **(chosen)**. **Two rejected drafts are recorded because the same failure produced both, with the sign flipped, and only independent review caught either.** Draft 1 made the graph shape *contingent* on OQ-35/OQ-36 on the premise that "an adjacency needs a mapping-valued fact" — it does not; a set of encoded strings carries one inside the frozen declaration — which came from conflating a *provider-supplied catalog* (sub-shapes B1/B2, struck here as working and shipped) with **B3, which is specifically same-changeset cross-file *diffs***. It would have advised the operator to defer a 14-story epic on a premise that does not hold. Draft 2 replaced that with a second "independent reason" — that CEL cannot decode an edge, killing even a bounded two-hop check — **refuted by execution** (the encode-and-compare leaf above). Draft 3 replaced *that* with the exponential-cost claim above — **refuted by measurement**. The common cause across all three is one habit: **reaching for a second independent reason when one sufficient reason was already in hand**, and then applying an **asymmetric evidentiary standard** to protect it — accepting awkward-but-working spellings when they *struck* a shape (A2's re-derivation, B2's dynamic index) and rejecting them when they would have *narrowed* one this record wanted to keep. The verdict was right in all four drafts; the supporting argument was wrong in three. It now rests on one measured property and no supporting argument at all, which is the form it should have taken first. The correction to A2 runs the same way: the value binder `[expr].all(v, …)` removes the evaluation-cost half of A2's residual, so **A2 is struck more firmly than the first draft struck it**. Binds later stories: **S05** must not be widened to carry cross-manifest data; **S07**'s violation shape must support a fold result and a path/cycle witness, not a cross-manifest reference; **S12** must correct ADR-0002's `rego` bullet, which calls the tier an "escape hatch for **cross-entry checks**" — inaccurate for the shipped input contract; **S11** must label the unquarantined example a shape illustration, never evidence of need. **Binds E11-S04, whose denylist is not yet written: `graph.reachable` MUST be allowed**, with `split` alongside it — both pure and deterministic. `graph.reachable` is the one that carries the justification (it closes the graph at any depth); `split` is a convenience for rebuilding the adjacency. A capability set drafted from "deny anything unfamiliar" would strike out the epic's own strongest justification. Recorded in the S04 section of the epic spec, not only S01's. **That floor is held by REVIEW, not by a gate, and this row says so rather than inventing an enforcement path:** REQ-E11-S04-02's committed golden detects a *change* to the allowed set (so the sandbox cannot silently widen) and cannot detect an *omission* — a golden written without `graph.reachable` stays green forever. It says nothing about judgment call (d), since *which* builtins are callable is orthogonal to *where* the evaluator lives. Two residuals raised, neither decided, **neither gating any verdict here**, and neither the escalated rule-7 (d1)/(d2) question: **OQ-35** (the `entry`/`oldEntry` binding asymmetry — a silent `assent test` / `assent run` divergence; set-difference is struck either way and the graph shape needs no entry tree) and **OQ-36** (the frozen provider declaration has no object/map type, yet the authoring surface and `builtin/repo-file` together permit a mapping-valued fact). Also flagged, not edited (out of lane): `REQ-DEM-S05-04` calls the provider-visible tree the "merged-result checkout"; the code reads `/head`, documented at `cmd/assent/checkout.go:44` as "the MERGE-REQUEST HEAD". This lane writes no Go, adds no dependency and touches no schema. Revert: delete `docs/planning/rego-tier-ceiling.md` and the D-156 narrowing text from the E11 spec — which restores an epic justified by four shapes, three of which this record shows are not real. | | D-158 | 2026-08-23 | **CI-TOOLCHAIN — `verify` was red on every PR from a golangci-lint/Go skew; the remedy is to bump `GOLANGCI_LINT_VERSION` v2.12.2 → v2.13.1 and KEEP `go-version: stable`.** Symptom: PR #85, a six-file Markdown diff with **zero Go bytes**, failed `verify` with `/opt/hostedtoolcache/go/1.27.0/x64/src/crypto/internal/randutil/randutil.go:11:2: could not import math/rand/v2 (… method must have no type parameters) (typecheck)` — the failing path is inside the **Go toolchain's own source tree**, so no author change can fix it and re-running cannot clear it. `Analyze (go)`, `Analyze (actions)` and `CodeQL` all passed on the same run: the tree compiles clean under Go 1.27, only the linter chokes. **Mechanism**: golangci-lint typechecks the standard library with the `go/types` it was **compiled with**, not the `go` on `PATH`. GitHub's `stable` rolled 1.26 → 1.27.0 between this repo's last green `main` run (2026-08-19) and 2026-08-23; the v2.12.2 release binary is built with go1.26.x and cannot read the 1.27 stdlib. **Reproduced locally, not reasoned** (Go 1.27.0 SDK on `PATH`, `GOTOOLCHAIN=local`, whole tree): v2.12.2 → **panic** in `pkg/goanalysis/runner_loadingpackage.go:335` inside `go/types.(*Checker).Files`, exit 2; official v2.13.1 asset (`built with go1.27.0`) → **0 issues**, exit 0; `golangci-lint config verify` → exit 0, so the v2 config schema in `.golangci.yml` needs no migration; v2.13.1 also → **0 issues** against local Go 1.26.6, so the bump is safe for contributors who have not moved; and `go install …@v2.13.1` under Go 1.27.0 → 0 issues, which is the **`release-exitgate`** path (that job builds the binary rather than using the action, and `if: github.event_name != 'pull_request'` / RELSE-08 means no PR ever exercises it). **Diagnosis heuristic worth keeping**: golangci-lint's `go.mod` carries the comment *"the minimum Go version must always be latest-1"*, so its `go` directive is a machine-readable "supports Go 1.N+1" signal — v2.12.2 says `go 1.25.0` (supports ≤1.26), v2.13.1 says `go 1.26.0` (supports ≤1.27), and upstream commit `42a0530` in v2.13.0 is literally "go1.27 support". **Options considered.** (A) *bump the linter* — CHOSEN: treats the root cause, is a **one-line** diff because the version is single-sourced at workflow scope (D-128), fixes BOTH exposed sites (the `golangci-lint-action` in `verify` and the `go install` in `release-exitgate`) at once, and the risk it carried — new findings from a newer linter blocking every merge, since `golangci-lint` runs inside `task lint` inside `task check` — was **measured at zero** on the whole tree under both Go versions. (B) *pin `go-version: 1.26.x`* — REJECTED as the primary: it needs two pins (`verify.yaml:56` and `:171`), leaves the class of bug live, undercuts the stated reason `stable` is there (`# govulncheck needs it`, with `setup-go` exporting `GOTOOLCHAIN=local` so a pinned toolchain can fail to `go run` a tool whose module requires a newer Go — unverified, and moot under (A)), and trades a loud recurrence for **silent indefinite drift onto an unsupported Go** in a repo whose posture is govulncheck + cosign + SLSA + Scorecard. (C) *both* — REJECTED as gold-plating: (A) alone is measured green, and a second simultaneous remedy would make a still-red PR ambiguous. **Per-file sweep of `go-version: stable`, stated rather than silently skipped:** `verify.yaml:56` and `:171` were the only exposed sites and are fixed by the env bump without touching either line; `schemas.yml:57` (runs `go test` + ajv, no linter), `vulncheck.yaml:30` (govulncheck only — `stable` is actively *desirable* there) and `codeql.yaml:45` (CodeQL, which passed on 1.27.0) carry no golangci-lint and are NOT exposed; `release.yaml` uses `go-version-file: go.mod` and is unaffected. `hack/lint/workflow_pins_test.sh` asserts `TASK_VERSION`, action SHA pins and step wiring but **not** `go-version` or `GOLANGCI_LINT_VERSION`; it passes on this change. **Revert is (B), not bare v2.12.2** — reverting the env line alone restores a known-red state: revert = `GOLANGCI_LINT_VERSION` back to `v2.12.2` **and** pin `verify.yaml:56` + `:171` to `go-version: 1.26.x`. **Residual risk, stated plainly:** when `stable` rolls to Go 1.28, v2.13.1 breaks identically. The repo finds out because `verify` reds on **every** PR including zero-Go diffs — loud and immediate, with the signature recorded in `verify.yaml`'s env comment and the fix being one env line. Nothing detects it *earlier*: dependabot's `github-actions` ecosystem updates `uses:` refs, not `env:` literals (backlog `CI-TOOLCHAIN-F01`). Accepted on purpose over (B)'s failure mode. Note also the pre-existing local/CI linter skew — `Taskfile.yml`'s `lint` task is bare `golangci-lint run ./...` against whatever binary a contributor has installed, so this pin has never governed local `task check`; deliberately not changed here (that file was owned by a concurrent lane). | +| D-159 | 2026-08-23 | **ORPHAN — `release-verify-test` was a gate invoked by nothing (D-124 again), and it is wired into `task check` in its three OFFLINE modes, not its default `all`.** Finding: `Taskfile.yml` defined `release-verify-test` — the **only** test of `hack/release/verify-artifacts.sh` — and no caller existed anywhere: not `check:`, not `CHECK_STAGES` in `hack/audit/exitgate_test.sh`, not `.github/workflows/**`, not the push-only `release-exitgate`. That script is not incidental: **`SECURITY.md:61` publishes it as the procedure release consumers run** to check artifact checksums, the stamped `assent version` and cosign signatures, and D-153/AUD2-F01 had just proved that published verification instructions rot silently. Measured before fixing, as the lane required: the gate was **green**, so this is a wiring defect only — nothing was being papered over. **Why not the `all` default.** `verify_test.sh` with no argument adds `test_snapshot_pass`, which shells out to `task release-snapshot`: `go install github.com/goreleaser/goreleaser/v2@…` on any machine without it (**network**, in a gate that runs before every commit), a `before: hooks: go mod tidy` that **rewrites `go.mod`/`go.sum` in the middle of `task check`**, `--clean` on the developer's `dist/`, and a five-target cross-compile (10s here with warm caches and goreleaser already installed; minutes cold). A flaky or network-bound stage is worse than a missing one, so `check:` runs `negative`, `cosign-skip-when-absent` and `readme` — REQ-E9-S12-02/03/04, all offline, together well under a second. **Consequence, stated precisely: REQ-E9-S12-01 (the real snapshot→verify round trip) is not gated per-commit in `task check`; it runs in push-only CI via `hack/release/exitgate_test.sh:33-34` (the `release-exitgate` job, `if: github.event_name != 'pull_request'`) — automated but not PR-visible, i.e. the RELSE-08 blind spot.** Those two lines are literally `task release-snapshot` then `task release-verify`, which is the `Verify:` string the spec gives the requirement (`openspec/specs/p5-e9-distribution/spec.md:441-442`), so nothing here needs a new gate built — an earlier draft of this row claimed the requirement was covered by no automation at all, which was false, and acting on it would have meant a future lane duplicating `exitgate_test.sh:33-34`. What this lane declines to do is move that L1 round trip into a per-commit gate: it cannot be, for the network/`go mod tidy`/`--clean` reasons above. Narrowing the RELSE-08 exposure for this requirement is a separate question from D-159. **Pinning.** `CHECK_STAGES` goes 20 → 21 in the same commit as the `check:` line (the AUD-S18/RELSE-08 lockstep), and `STAGE_BODY_PINS` pins the body **per mode** rather than by the bare script path — a single `bash hack/release/verify_test.sh` pin would stay green if two modes were dropped *and* if the stage were reverted to the unrunnable-in-check default. Three mutation controls prove it: deleting the `check:` line reds `check_check_wiring` by stage name, gutting one of the three modes reds `check_stage_bodies` while the other two still run, and the same deletion applied to the real tree reds the real gate at rc=1. **Also fixed, minimally, because wiring made it matter:** `verify_test.sh` wrote its captured stdout/stderr to fixed `/tmp/verify-{tamper,nosig}.{out,err}` and then **grepped** them; harmless while nothing ran it, but this repo runs several lane worktrees whose `task check` invocations overlap, and a concurrent run truncating `/tmp/verify-nosig.err` at open would fail the other run's `skip.*cosign` grep spuriously. Scratch files now live in each run's own `mktemp -d` (invisible to `verify-artifacts.sh`, which globs only `*.tar.gz`/`*.zip`) and are removed by the existing `RETURN` trap. | diff --git a/hack/audit/exitgate_test.sh b/hack/audit/exitgate_test.sh index 3168005..509f0ca 100644 --- a/hack/audit/exitgate_test.sh +++ b/hack/audit/exitgate_test.sh @@ -140,6 +140,16 @@ CHECK_STAGES=( release-changelog-gate-test release-verify-tag-gate-test release-install-cosign-pin-test + # ORPHAN (D-159): the verify-artifacts harness gate. `release-verify-test` was + # defined in Taskfile.yml and invoked by NOTHING — not check:, not this array, + # not any workflow — while `hack/release/verify-artifacts.sh` is the procedure + # SECURITY.md:61 publishes to release consumers. Added in the same commit as + # its `check:` line, and its body is pinned in STAGE_BODY_PINS by MODE, because + # the stage is check:-safe only in its three offline modes (the `all` default + # shells out to `task release-snapshot`: `go install goreleaser@…` on a cold + # machine, a `go mod tidy` before-hook that rewrites tracked files mid-check, + # and `--clean` on dist/). + release-verify-test docs-gates lint-depguard-test lint-workflow-pins-test @@ -187,6 +197,17 @@ STAGE_BODY_PINS=( 'lint-workflow-pins-test|bash hack/lint/workflow_pins_test.sh|a wired stage with a gutted body is the same defect one level down' 'ci-audit-test|bash hack/release/ci_audit_test.sh|a wired stage with a gutted body is the same defect one level down' 'lint-bash-version-guard-test|bash hack/lint/bash_version_guard_test.sh|a wired stage with a gutted body is the same defect one level down (BASH32-F01)' + # ORPHAN (D-159). Pinned per MODE, not by the bare script path, and that is the + # whole point: `verify_test.sh` with no argument runs mode `all`, which calls + # `task release-snapshot` (network `go install`, a `go mod tidy` before-hook + # that rewrites go.mod/go.sum mid-check, `--clean` on dist/, a five-target + # cross-compile). A single `bash hack/release/verify_test.sh` pin would stay + # green under BOTH regressions that matter here: dropping two of the three + # modes, and reverting the stage to that unrunnable-in-check default. Either + # keeps the stage wired while it stops checking what D-159 wired it for. + 'release-verify-test|bash hack/release/verify_test.sh negative|REQ-E9-S12-02: without it nothing proves verify-artifacts.sh REJECTS a tampered archive, which is the only assertion that makes SECURITY.md:61 worth publishing' + 'release-verify-test|bash hack/release/verify_test.sh cosign-skip-when-absent|REQ-E9-S12-03 / D-110: the unsigned-artifact path must exit 0 AND say it skipped — a silent skip and a real verification are indistinguishable to a consumer' + 'release-verify-test|bash hack/release/verify_test.sh readme|REQ-E9-S12-04: hack/release/README.md must keep documenting the skip-when-absent behaviour the mode above proves' ) # (6) Immutable base ref for the schema freeze. Overridable only to move it @@ -1589,6 +1610,18 @@ mutate "$tf_m" 's|^ - task: lint-workflow-pins-test$| # - task: lint-w expect_red check_check_wiring "a pinned stage was COMMENTED OUT rather than removed" \ "pinned stage 'lint-workflow-pins-test'" "$tf_m" +# ORPHAN (D-159): the control for the line this lane added. Before it, deleting +# `- task: release-verify-test` from check: put the repo back in the state D-159 +# names — verify-artifacts.sh's only test invoked by nothing — with `task check` +# green and no PR able to see it. Anchored on the STAGE NAME, not on the stage +# COUNT: a one-line deletion trips both findings, and the count message also +# fires for unrelated edits, so pinning it would be the weaker assertion. +tf_m="$WORK/Taskfile.noverifytest.yml" +cp "$TASKFILE" "$tf_m" +mutate "$tf_m" '/^ - task: release-verify-test$/d' '- task: release-install-cosign-pin-test' +expect_red check_check_wiring "'- task: release-verify-test' was deleted from check: — verify-artifacts.sh's only test is orphaned again (D-159)" \ + "pinned stage 'release-verify-test'" "$tf_m" + tf_m="$WORK/Taskfile.lowfloor.yml" cp "$TASKFILE" "$tf_m" mutate "$tf_m" 's|^ COVERAGE_MIN: [0-9]*$| COVERAGE_MIN: 90|' 'COVERAGE_MIN: 90' @@ -1651,6 +1684,16 @@ mutate "$tf_m" 's| - bash hack/release/ci_audit_test.sh| - echo skippe expect_red check_stage_bodies "the ci-audit-test: stage body was gutted while the stage stayed wired" \ 'no longer contains: bash hack/release/ci_audit_test.sh' "$tf_m" +# ORPHAN (D-159): gut ONE of the three modes and leave the other two. This is +# the control that proves the three mode pins are not one redundant pin wearing +# three hats — a bare `bash hack/release/verify_test.sh` pin would read this +# mutant as green, because two live invocations of that script remain. +tf_m="$WORK/Taskfile.guttedverifymode.yml" +cp "$TASKFILE" "$tf_m" +mutate "$tf_m" 's| - bash hack/release/verify_test.sh negative| - echo skipped|' ' - echo skipped' +expect_red check_stage_bodies "one of release-verify-test's three modes was gutted while the stage stayed wired and the other two modes still ran" \ + 'no longer contains: bash hack/release/verify_test.sh negative' "$tf_m" + # Vacuity control on the extraction itself: with the task key gone there is no # body to read, and "no pin failed" must not be the answer. tf_m="$WORK/Taskfile.nocoveragekey.yml" diff --git a/hack/release/verify_test.sh b/hack/release/verify_test.sh index 84b68bf..7f9194d 100755 --- a/hack/release/verify_test.sh +++ b/hack/release/verify_test.sh @@ -8,6 +8,16 @@ cd "$ROOT" VERIFY="$ROOT/hack/release/verify-artifacts.sh" MODE="${1:-all}" +# D-159: the captured stdout/stderr below live in each case's own `mktemp -d`, +# never in fixed /tmp paths. `task check` now runs this script (it used to run +# nowhere), and this repo executes several lane worktrees' `task check` at once — +# two concurrent runs sharing /tmp/verify-nosig.err would have one truncate the +# file at open while the other greps it, failing a green tree for no reason. +# Putting them inside the --dist directory is safe and stays safe: verify-artifacts.sh +# reaches files there in exactly two ways, both blind to these names — `find -maxdepth 1` +# restricted to `*.tar.gz`/`*.zip`, and find_sigstore_bundle's four candidates, every +# one of them derived from an archive's own filename. Nothing enumerates the directory. + fail() { echo "FAIL: $*" >&2 exit 1 @@ -50,7 +60,7 @@ EOF # Tamper archive after checksum recorded (fail-closed). echo "tampered" >>"$tmp/assent_1.0.0-test_linux_amd64.tar.gz" set +e - "$VERIFY" --dist "$tmp" --expected-version 1.0.0-test >/tmp/verify-tamper.out 2>/tmp/verify-tamper.err + "$VERIFY" --dist "$tmp" --expected-version 1.0.0-test >"$tmp/verify.out" 2>"$tmp/verify.err" local rc=$? set -e [[ "$rc" -ne 0 ]] || fail "tampered archive expected non-zero exit (REQ-E9-S12-02)" @@ -82,11 +92,11 @@ EOF EOF # No .sigstore.json — cosign branch must skip (D-110 / REQ-E9-S12-03). set +e - "$VERIFY" --dist "$tmp" --expected-version 2.0.0-skip >/tmp/verify-nosig.out 2>/tmp/verify-nosig.err + "$VERIFY" --dist "$tmp" --expected-version 2.0.0-skip >"$tmp/verify.out" 2>"$tmp/verify.err" local rc=$? set -e [[ "$rc" -eq 0 ]] || fail "cosign skip path expected exit 0, got $rc (REQ-E9-S12-03)" - grep -qi 'skip.*cosign\|skipping cosign' /tmp/verify-nosig.err /tmp/verify-nosig.out \ + grep -qi 'skip.*cosign\|skipping cosign' "$tmp/verify.err" "$tmp/verify.out" \ || fail "expected cosign skip message (REQ-E9-S12-03)" echo "OK: cosign skip when bundles absent (REQ-E9-S12-03)" } diff --git a/openspec/specs/backlog.md b/openspec/specs/backlog.md index 925e455..20f805b 100644 --- a/openspec/specs/backlog.md +++ b/openspec/specs/backlog.md @@ -750,6 +750,7 @@ that claim). | AUD2-F01 | **DONE** — SEC-03's twin: `hack/release/verify-artifacts.sh:124` ran the same **unpinned** `cosign verify-blob --bundle` on the maintainer/CI path; `verify_cosign()` now pins the D-153 issuer/identity pair and `hack/release/install_cosign_pin_test.sh` grades it as the **third** file of the one drift gate | **[autonomous]** | AUD2-S03 (D-153's value) | found *by* S03, deliberately left alone there — not an owned path, and out of that story's stated scope | | COUNT1-F01 | Two `go test` cache blind spots left inside `task test` / `task coverage` (`./...` and `./internal/...`): `internal/schemadrift` compares local `schemas/` against `git show origin/main:…` in a **subprocess**, and `internal/provider/isolation_test.go:29` `go build`s `./testdata/maliciousexec` at runtime | **[autonomous]** | — | found *by* the `-count=1` lane, deliberately left alone there: fixing them means carving packages out of `test`/`coverage` and losing an otherwise-honest whole-tree cache | | CI-TOOLCHAIN | **DONE (D-158)** — `verify` was red on **every** PR, including a six-file Markdown diff with zero Go bytes, because `go-version: stable` rolled Go 1.26 → 1.27.0 while `GOLANGCI_LINT_VERSION` sat at `v2.12.2` (built with go1.26.x, so its `go/types` cannot read the 1.27 stdlib); pin bumped to `v2.13.1`, whose "go1.27 support" commit and `go 1.26.0` go.mod directive make it the first release that can | **[autonomous]** | — | not owned by any in-flight lane — the pin is a CI-only workflow env var, and the break blocked all of them | +| ORPHAN | **DONE (D-159)** — `Taskfile.yml`'s `release-verify-test`, the only test of `hack/release/verify-artifacts.sh` (the procedure **`SECURITY.md:61` publishes to release consumers**), was invoked by nothing: absent from `check:`, from `CHECK_STAGES`, and from every workflow — D-124's "a gate invoked by nothing is not a gate", one epic on from AUD2-F01's published-instructions-rot finding. It was **green**, so the defect was wiring only. Now the 14th of 21 `check:` stages, in its three **offline** modes (`negative`, `cosign-skip-when-absent`, `readme` = REQ-E9-S12-02/03/04); the `all` default is deliberately NOT wired — it shells out to `task release-snapshot`, i.e. a networked `go install goreleaser`, a `go mod tidy` before-hook that rewrites tracked files mid-check, and `--clean` on `dist/`. **Not a gap to fill: REQ-E9-S12-01 (snapshot → verify round trip) is already automated** — `hack/release/exitgate_test.sh:33-34` runs exactly `task release-snapshot` then `task release-verify`, the spec's own `Verify:` string, in the `release-exitgate` job. It is push-only (`if: github.event_name != 'pull_request'`), so the exposure is **RELSE-08 visibility, not absence**; do NOT open a lane to build a gate that exists. `CHECK_STAGES` 20 → 21 in the same commit; body pinned **per mode** so dropping two modes or reverting to the bare default reds | **[autonomous]** | — | not owned by any in-flight lane; `Taskfile.yml` + `hack/audit/exitgate_test.sh` are the one `CHECK_STAGES` slot, which only one lane may hold at a time | | CI-TOOLCHAIN-F01 | The linter/toolchain coupling has **no early detector**: dependabot's `github-actions` ecosystem updates `uses:` refs, not `env:` literals, so `GOLANGCI_LINT_VERSION` rots silently until the next Go minor reds every PR at once (~6-monthly). Candidate: a text gate comparing the pinned linter release's go.mod `go 1.N` directive against the toolchain CI resolves, or an updater that watches the literal | **[autonomous]** | D-158 | deliberately out of scope of the fix lane: three lanes were parked behind it, and a new gate belongs with `hack/lint/**`, which was owned by a concurrent lane | | BASH32-F01 | **DONE** — three `hack/**` gate scripts used bash 4+ features with no version guard; under stock macOS `/bin/bash` 3.2 `hack/docs/truthlag_pins_test.sh` died at its `declare -A`, skipped its final `OK:` banner and **exited 0**, so `task docs-gates` (and `task check`) read a gate that certified almost nothing as green. Fixed by a shared, per-script-parameterised `hack/lib/require-bash.sh` floor (4.0 / 4.0 / 4.4), enforced by a new `hack/lint/bash_version_guard_test.sh` wired as `task check` stage 20 (D-154) | **[autonomous]** | — | found *by* the AUD2 wave's macOS runs, not by any audit: CI is ubuntu/bash 5, so the hole is LOCAL-only and no CI lane could have surfaced it | @@ -785,9 +786,10 @@ with the coverage percentage. as the 19th `task check` stage — wired **PR-visibly** in the `verify` job, not only in the push-only `release-exitgate` job, so the RELSE-08 blind spot is not reproduced. **AUD2-F01 is now CLOSED too** — still a follow-up rather than an AUD2 story (D-152), so the five-story claim -stands. It added no stage: `CHECK_STAGES` is unchanged at 19, because the fix extends the +stands. It added no stage: `CHECK_STAGES` was unchanged at 19 *by that fix*, because it extends the existing `release-install-cosign-pin-test` gate instead of starting a second published truth -(D-128). All three traps above were handled: the new sections are offline against the same +(D-128); the array has since moved on — BASH32-F01/D-154 took it to 20 and ORPHAN/D-159 to 21, +so read "19th stage" above as the ordinal at AUD2 time, not a live count. All three traps above were handled: the new sections are offline against the same stubbed `cosign`, and the vacuity trap is closed by making the discriminator the **stub's argv log** rather than the exit code — §5d requires that log to be non-empty and to carry both pinned values, and §5e is the paired control proving a bundle-less (snapshot-shaped) `dist/` @@ -797,7 +799,7 @@ with itself, and — the one that matters — both flags left textually in place branch is never entered. **COUNT1-F01 is now the one OPEN follow-up** — likewise not an AUD2 story (D-152), so the five-story claim is untouched by it either. It is also stage-neutral: the `-count=1` lane that logged it added a *step* to the `verify` job and rewrote `Taskfile.yml` -recipes, not a `task check` stage, so `CHECK_STAGES` stays 19 there too. +recipes, not a `task check` stage, so that work adds no `CHECK_STAGES` entry either (the array itself is 21 as of ORPHAN/D-159, not the 19 this paragraph was written against). ## Phases 3–5