backlog: rewrite BACKLOG #1244 against the ratified coverage boundary, and drop a control that cannot fail - #553
Merged
Conversation
…ruling 2026-08-23) The constraint that made this item hard is retired. It required a public pre-merge check to name no cell id and no count of affected cells, on the reasoning that a public CI log enumerating them hands out coverage by subtraction. A workflow tested that premise against the public engine tree and inverted it: the aggregate coverage the constraint protected is already public in tracked engine files. The constraint was defending a number that was already out. The owner ratified that boundary rather than mounting a redaction campaign, and redrew the private set to three things: the per-cell verdict grid, the residual prose, and the absence claims. Two consequences follow and both delete work this item required: "must not name the count of affected cells" is retired, and "keep the anchor set secret" is dropped -- no public, actionable, pre-merge check naming a file and line can satisfy it, because such a check is a membership oracle by construction. Recorded alongside, so nobody reads permission into silence: the ruling ratified the aggregate and did not rule that a public log may print an id paired with its verdict. That pairing is the vaulted grid. Also in this commit: - Gap 3 is marked CLOSED, with the mechanism (a shrink-only anchor-failure baseline keyed on kind/path/token) and its CI-unverified, unpushed status. Gaps 1 and 2 stay open. - The mechanism is stated rather than lettered. It was never "(a) or (b)"; it is the vault-initiated check-run decided 2026-08-09. - The two conditions that still bind are written into the item: clear the broken anchors before wiring any gate, and settle required-context up front, since an external check-run is advisory unless branch protection names it. - The unmeasured limb is named as unmeasured. Whether anchor density correlates with non-pass verdicts was not measured by the ratification, and no new constraint may lean on it. - The re-scored banner's difficulty rationale cited the retired constraint. Flagged rather than re-scored; a re-score is its own dated act. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…rage objection was retired The ratification removed the reason this option was filed as forbidden. Without a replacement, the item now reads as though the enumeration is back on the table -- the exact "silence is permission" reading the previous commit warned against, arriving one paragraph later. The replacement ground is stronger because it rests on no secrecy argument at all: a manifest publishes the one field the checker does not decide on. check_anchors says so in its own docstring at scripts/asvs/scorecard.py:944 -- a token that is GONE or AMBIGUOUS reds the gate, while a token that is unique and present but sits at a different line is an ADVISORY. The verdict turns on token presence and uniqueness; nothing branches on the line number. So publishing file-and-line pairs pays a real disclosure cost for a field carrying no decision weight, and it is also the field most likely to be wrong: one verifier run carried 1181 advisory DRIFT rows, anchors resolving at a line other than the one recorded. The manifest would be simultaneously the leakiest column and the stalest. Verified against the engine source rather than taken on relay. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The row cited `git ls-files docs/security returns 0` as evidence the engine carries no scorecard data. The CONCLUSION is right and the INSTRUMENT is inert: `.gitignore:170` is `/docs/security/`, and `git check-ignore -v` names that rule for `docs/security/asvs-scorecard.toml`. The command returns 0 whether the scorecard is absent, present-but-ignored, or sitting in that directory right now. It cannot return a different answer, so it distinguishes nothing. Two further scope gaps in the same citation: it interrogates ONE PATH while the sentence claims something about the whole engine, and it counts FILES while "scorecard data" is a claim about CONTENT. Replaced with two controls that can each come back the other way: zero of the 29 tracked .toml files carry a [[cell]] header scorecard.py:2384 declares --scorecard required=True, no bundled default The [[cell]] sweep was run against a synthetic file that HAS one and fired, so the zero means absent rather than broken. IT WAS IN TWO PLACES -- the ranked table at :299 and the item body at :11822 -- and an assertion expecting one occurrence is the only reason both were found. Fixing the item alone would have left the table asserting the retired evidence, which is how a duplicated load-bearing fact drifts (SDS-3.5). The table keeps the short form and its pipe count is unchanged. Also narrows "zero scorecard data" to "zero scorecard CELLS". The unnarrowed phrase reads as a contradiction against this same row's finding that aggregate verdict tallies are already public in tracked engine files, which they are: scripts/docs/asvs_tally_baseline.txt carries them and is not a leak, because the owner ratified aggregate coverage as public and it holds no id-to-verdict pairing. Not a re-score. The author left that to whoever next scores the item and it stays open. Their two commits are unchanged; this correction is mine, not theirs. Surfaced by a five-lens disclosure screen of the row -- 14 findings raised, 0 sustained -- whose fact-check pass caught the citation the lenses were not looking at. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…erted letter The warning read "TODAY'S OWNER RULING DOES NOT COVER OPTION (a)". The owner ruling, read in full today, states that the letters are inverted between documents: .github/workflows/asvs-prove-absences.yml gives "option (a)" to the vault-token-in-public-CI design, while the ruling brief gives that letter to the anchor manifest. So a reader comparing the two documents took this warning to mean the opposite of what it says -- and it is a warning about what is NOT permitted, which is the worst place to invert. Now names the mechanism: publishing the anchor manifest, the enumeration. The ruling's own instruction is quoted inline so the next editor does not reintroduce a letter. No scope change. The item's conclusion, both binding conditions, and the unmeasured cardinality-correlation limb are untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…e sweep never scanned
The 2026-08-08 ledger-coverage sweep that produced the "research an honest pass
for ASVS X.Y.Z" family scanned the partial and fail cells. It did not cover
needs-review. So 186 cells carry an item and the three contested ones carry
none -- two of them for about three weeks.
That is a gap in the sweep's domain rather than a lapse: a needs-review cell has
no agreed target to build toward, because two assessors disagree about what the
requirement demands. Filing them as RESEARCH items fits that -- the deliverable
is a settled reading, and "cannot honestly reach pass" stays a valid finding.
#1350 ASVS 1.3.4 (L2) an image/svg+xml attachment is accepted, stored and
served; the prior na said the engine neither accepts
nor renders SVG, and the accepts half is false. A
layered control IS shipped -- substring MIME
classifier, sandbox CSP, forced attachment
disposition, global nosniff -- so the open question
is the classifier's completeness, not absence.
#1352 ASVS 11.4.4 (L2) the prior na quoted the requirement with its second
sentence truncated, dropping the clause that asks
whether derivation PARAMETERS balance security
against brute force. argon2id via argon2-cffi is
present; whether its parameters were chosen, and
whether the site inventory is complete, is not
established.
#1351 ASVS 10.4.13 (L3) the OIDC authorization request travels the
browser-visible front channel by construction. The
contest is whether the pinned text requires PAR
(RFC 9126) or whether PKCE answers it. No factual
claim in the derivation was refuted.
Table and census: rows added to the CURRENT 2026-08-20 pass only. The 08-03 and
07-10 tables are frozen records of superseded passes and are byte-identical. The
distribution lines are RECOMPUTED from the table rather than adjusted, per the
file's own note that a stale census reads exactly like a current one: 228 -> 231
rows, V4 40->41, V5 58->60, P2 116->118, P3 65->66, fill-in 126->129.
Numbers allocated via alloc.ps1, not chosen by inspection.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
wshallwshall
enabled auto-merge (squash)
August 23, 2026 20:42
Resolved docs/BACKLOG.md. Both sides appended items after #1348 and there is no overlap: main contributed #1349 (from PR 554), this branch contributes #1350-#1352. Kept both, in numeric order. Resolved from the ALLOCATING worktree. #1350-#1352 are pinned to asvs-tracker-carry-on-0ec016 in their alloc records, and ledger_check.py owns() casefold-compares that stored path against the committing repo, so no other seat could have made this commit -- the Lander and the Dispatcher were each refused for that reason before it reached me. Ownership is per-number and pinned to a PATH, not a standing capability of any role. Census RECOMPUTED from the table rather than carried across the merge: 231 rows, P1 18, P2 118, P3 66, DEMAND-GATE 29. Unchanged from this branch's pre-merge figures because #1349 is not in the ranked table yet. Stated explicitly because a recomputed count and a carried one are indistinguishable once written. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ons (BACKLOG #1005, ASVS 12.1.4) Drafted by a bounded workflow and then attacked by four independent refuters: citation checking, runtime re-measurement, house-rule hunting, and one arguing the decision is wrong. Zero blocking findings. Eight repairs applied before filing, each re-verified against the code by hand rather than taken on the refuter's word. DECISION: accept and document both graded directions, plus one build rider the accept reasoning does not cover -- three originating hops that never reach the existing revocation guard, filed by subject and deliberately unallocated per the no-dangling-#N rule. The load-bearing distinction: direction 1 (stapling) is UNBUILDABLE, not unbuilt. No stapling surface exists in the ssl module API at all, measured against live positive controls. The opt-in client-certificate CRL checking that DOES ship is a THIRD combination -- peer revocation on the terminating side -- and moves neither graded direction. That is the single easiest thing in this area to misread, so it gets a two-axis table and a bolded rule. REPAIRS, and the pattern in them is worth recording: Three table rows cited the NON-verifying arm of an `if verify:`. The SMTP row cited tls_policy.py:1218, the `else` branch that goes on to set CERT_NONE, while the verifying context is built at :1216. Same shape in the REST and postgres rows. A table listing verifying hops cited the lines reached only when the hop does not verify. "nine guarded originating hops" where the list has seven, in an ADR that argues elsewhere that a hardened count is a liability. The syslog forwarder was said to "additionally lack" harden_verify_flags, presented as distinguishing. Measured: all three unguarded hops lack it, against a positive control of seven files that carry it. The corrected reading strengthens the rider. "the pinned runtime", twice. THERE IS NO PIN: pyproject.toml:28 is requires-python = ">=3.14", a floor with no implementation constraint, and no .python-version, .tool-versions or runtime.txt exists. Now scoped to what is true and sufficient -- every interpreter this project builds against -- with the re-check trigger deliberately not keyed to CPython. This defect was caught by the owner, not by the four refuters; the sibling ADR's refuters found the identical phrase and killed it. Index row added in this commit, per the ledger gate. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… population is ungraded The rider named three unguarded originating hops. One of them does not exist as a TLS site: messagefoundry/transports/smart.py contains ZERO ssl usage -- no create_default_context, no SSLContext, no ssl. anywhere -- against a control of 153 such hits package-wide. Lines 183-184 are an opener and a redirect handler; the file rides urllib's own context. Found by the Dispatcher while scoping the rider for a build lane, checked before queuing rather than after, and re-verified here independently. TWO CONSEQUENCES, and the second matters more than the first: The two remaining hops are confirmed exactly as written -- auth/oidc_http.py:99-101 and logging_setup.py:310 are both real create_default_context calls. "Three" also understated the population. At least six further unguarded context constructions exist (apiclient/client.py:214, postgres.py:729, rest.py:275, :296, soap.py:202, tls_policy.py:1218) and NONE has been graded -- several are outbound client contexts where the answer may differ. The rider is therefore not a well-specified three-hop job; it is an ungraded population. Scoping it as small would have understated the work. This citation survived four adversarial refuters, eight hand repairs and an owner ruling. It is the same class as the SMTP if/else already corrected here: a citation that reads correct and points at the wrong construct. What the repair pass checked was whether cited lines say what the ADR claims -- not whether the cited FILE is the kind of thing being claimed about. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…es in five voices The ninth repair withdrew a rider hop that is not a TLS site and stated the population is larger than two and ungraded. It corrected ONE of the six places the ADR asserted that count. The other five still said "three", so the document contradicted itself -- and the worst of them, section 4.3's harden_verify_flags measurement, re-cited transports/smart.py as a measured member of the population immediately after the withdrawal. Found by the Lander, who held the commit rather than landing it. FIXED BY REMOVING THE RESTATEMENTS, not by editing the number into five more places. CLAUDE.md section 11, SDS-3.5: state a load-bearing fact ONCE and link to it. The count now lives in section 4.3 alone; the other five sites say "the unguarded hops" and point there. A fact restated six times gets corrected in one. :196 now "TWO CONFIRMED ... population larger and UNGRADED, see 4.3" :237 count and scope delegated to 4.3 :360 smart.py REMOVED from the measured list, with the reason :373 "three" dropped :383 points at 4.3 THE THIRD LAYER, and the Lander named it exactly. Nine repairs fixed CITATIONS: lines pointing at the wrong construct. This is the COUNT those citations supported, restated across the document. Four refuters, nine repairs and an owner ruling each stopped one layer short of the next -- the refuters checked whether cited lines say what is claimed, the ninth repair asked whether the cited FILE is the kind of thing being claimed about, and neither asked whether the CLAIMS DEPENDING ON THE CITATION were updated with it. No claim widened. The six further unguarded context constructions remain ungraded and this commit does not grade them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…hat refutes its own researcher
Owner-directed research ran as a 10-agent dispatcher workflow with two refuters
per finding. The rows are written here rather than by the dispatcher because
ledger_check.py takes base='origin/main' as a default argument: #1350-#1352 are
on PR 553, not on main, so from any worktree they read as new headings and
owns() pins them to this one. Measured by the dispatcher, not assumed.
#1350 ASVS 1.3.4 -- cannot honestly pass without work. Neither refuter broke it.
The browser-active classifier is a FOUR-TOKEN DENY-LIST (html, xml, script,
svg). Reproduced here: application/hta is not flagged, and
mimetypes.guess_extension("application/hta") returns .hta FROM THE WINDOWS
REGISTRY, so the served extension is host-derived.
AND IT CORRECTS THIS ITEM'S OWN PREMISE, which was mine: the item asked
whether the sandbox holds on every path, assuming the CSP is reached THROUGH
the classifier. It is not. _ATTACHMENT_CSP is re-asserted on every attachment
response (app.py:720, :749) and set directly at :3571; Content-Disposition is
unconditional. The classifier downgrades the declared Content-Type; it does
not gate the sandbox.
#1351 ASVS 10.4.13 -- THE RESEARCH VERDICT DOES NOT HOLD. Engine facts right,
disposition wrong, and one refuter broke it. APPLICABILITY WAS NEVER ASKED:
ASVS-ASSESSMENT-METHOD.md:44-45 makes scope rule 1, ahead of every control
question. V10.4 is the OAuth AUTHORIZATION SERVER section and this engine
hosts none. The item text skipped that rule too, so this is a defect in the
row as much as in the research. The honest next step is reading the 10.4.13
text and applying rule 1 -- not building PAR against a requirement that may
not apply.
#1352 ASVS 11.4.4 -- cannot honestly pass without work. Both refuters
reproduced the control. ALL FIVE PINNED ARGON2ID VALUES ARE BYTE-IDENTICAL TO
argon2-cffi 25.1.0's DEFAULTS, verified here against the installed library,
under a comment at passwords.py:14 stating they are pinned explicitly RATHER
THAN relying on those defaults. The values meet OWASP guidance; what is
missing is any measurement behind them. A refuter timed the heavy profile at
696.5ms against 34-36ms shipped.
Every load-bearing engine claim above was re-verified in this worktree rather
than taken from the findings file.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… as one The ADR put ASVS 12.1.4 in quotation marks as "proper certificate revocation, such as OCSP stapling" and asserted that sentence "covers two independent behaviours". Both are wrong against the primary source, which the vault already tracks and which needs no network to read: asvs-5.0.0-source/OWASP_..._5.0.0_en.csv line 254 "Verify that proper certification revocation, such as Online Certificate Status Protocol (OCSP) Stapling, is enabled and configured." level 3 One sentence. No "client", no "peer", no "server". ASVS uses the same `such as` construction one row up (12.1.1, "such as TLS 1.2 and TLS 1.3") for a plainly non-exhaustive example, so the operative noun is proper certification revocation and stapling is an instance of it. The client-side duty is a DIFFERENT requirement -- 12.3.2, L2 -- and it grades validation, not revocation. So the two-direction split is this ADR's reading, not the requirement's structure. It is kept, because the engine's surfaces do differ that way and the two do have different remedies; it is now labelled an interpretation instead of being presented as the standard's wording, and section 7 tells a citing cell to carry that caveat. The verdict does not move and this commit does not move it. A contested reading resolves to the worse verdict, and reinterpreting a requirement to turn a red green is the move this same cell already declined. Whether the reasoning resting on the split gets amended is an open owner decision. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Recovered work. The ASVS Tracker seat closed with these two commits committed and unpushed; they are rebased onto current main with authorship preserved. The third commit is mine and is described below.
Rescued rather than rewritten: the seat is gone, so nothing was going to push these on its own.
What the two recovered commits do
They rewrite BACKLOG #1244 against the owner's ratified coverage boundary of 2026-08-23. The constraint that made the item hard -- a public check must name no cell id and no count of affected cells -- was retired, because the aggregate it protected is already public in tracked engine files. The private set is now three things: the per-cell verdict grid, the residual prose, and the absence claims.
"Keep the anchor set secret" is dropped as unsatisfiable rather than expensive. Any public, actionable, pre-merge check that names a file and a line is a membership oracle by construction.
The published-anchor-manifest option stays rejected, on a stronger ground that needs no secrecy argument: a manifest publishes the one field the checker does not decide on.
check_anchorsturns on token presence and uniqueness; the line number is advisory. One run carried 1181 advisory drift rows, so a manifest would be the leakiest column and the stalest at once.#1244 stays OPEN. Gap 3 is recorded closed but CI-unverified; gaps 1 and 2 are open.
Two things the author flagged as theirs, not the owner's
Both survive this landing with that labelling intact, and they are the right shape:
They also did not re-score the item, and said so in the text rather than leaving the stale rationale to be discovered.
The third commit is mine: the row cited a control that cannot fail
The rationale cited
git ls-files docs/security returns 0as evidence the engine carries no scorecard data. The conclusion is right and the instrument is inert..gitignore:170is/docs/security/, andgit check-ignore -vnames that rule for the scorecard path, so the command returns 0 whether the scorecard is absent, ignored, or sitting in that directory right now.Two further scope gaps in the same citation: it interrogates one path while the sentence claims something about the whole engine, and it counts files while "scorecard data" is a claim about content.
Replaced with two controls that can each come back the other way:
.tomlfiles carrying a[[cell]]headerscorecard.py:2384--scorecard required=True, no bundled defaultIt was in two places -- the ranked table at :299 and the item body at :11822. An assertion expecting one occurrence is the only reason both were found; fixing the item alone would have left the table asserting the retired evidence.
This is not a re-score. The author reserved that and the item stays open.
Disclosure screen
This rewrites the ASVS boundary in a public file, so it was screened by five independent lenses -- one per private limb, one on the subtraction channel, one on vault metadata -- each with a skeptic prompted to refute, none permitted to open the vault.
14 findings raised, 0 sustained. The fact-check pass alongside it is what caught the inert citation, which none of the disclosure lenses were looking at.
Ledger check
parse_itemsagainst main as control: 354 items / 258 open / 0 duplicate fields, unchanged. Nothing added, nothing lost, no status flips, #1244 still open. Pure amendment.Co-Authored-By: Claude Opus 5 noreply@anthropic.com
Added after the disclosure screen: BACKLOG #1350, #1351, #1352
3f18051bfiles the three contested ASVS cells the coverage sweep never scanned. Owner-directed, numbers allocated throughalloc.ps1.Rows were added to the current 2026-08-20 table only. The 08-03 and 07-10 tables are frozen records of superseded passes and are byte-identical.
The census was recomputed from the rows, twice, on two different bases — 228 to 231 — rather than incremented. That distinction is stated because a recomputed count and an adjusted one are indistinguishable once written. The ordering invariant was checked end to end: zero violations.
The cherry-pick conflicted, positionally: 553 had gained
#1347and#1348from a main merge and both sides appended after#1346. Both sides were kept.This commit was authored from the allocating worktree, not cherry-picked by me. The ledger gate pins the allocating path, not the person, so a scratch worktree has a different toplevel and is refused exactly as a foreign one is. That is the same owner ruling applied as in PR 564 — restore the condition the gate tests rather than bypass it.
Ledger check against main as control: 356/260 to 359/263, zero duplicate banner fields, added 1350, 1351, 1352, lost none.