Skip to content

Complete the first-country configuration product#508

Open
jeremi wants to merge 21 commits into
mainfrom
codex/first-country-config-latest
Open

Complete the first-country configuration product#508
jeremi wants to merge 21 commits into
mainfrom
codex/first-country-config-latest

Conversation

@jeremi

@jeremi jeremi commented Jul 26, 2026

Copy link
Copy Markdown
Member

Summary

This delivers the Phase 1–3 first-country configuration product as one governed adopter journey:

  • make Registry projects, environments, integrations, entities, fixtures, service policy, Relay consultations, and Notary claims explicit authoring contracts
  • add deterministic schema authority, safe diagnostics, migration candidates, fixture coverage, semantic comparison and impact, preflight, and promotion evidence
  • generate configuration reference, diagnostic catalogs, standard spreadsheet/OpenAPI/HTTP/script/snapshot/Notary journeys, and CI routing from reviewed sources
  • distinguish authored intent from generated Relay and Notary product inputs, and bind review, signing, verification, promotion, activation, restart, recovery, and rollback responsibilities
  • close topology edge cases so Relay-only, Notary-only, and removed-product baselines retain the right product-owned obligations
  • document the current source-built workflow honestly relative to v0.13.0 and make required readiness gates explicit

Review corrections

The final branch includes all actionable PR review findings:

  • independently authored governed requests now prove request-to-consultation binding instead of treating downstream fixture input as end-to-end evidence
  • semantic and compile-stage check failures retain safe field addresses and structured JSON output
  • file-backed CSV, XLSX, and Parquet entity providers participate in offline preflight with private-file posture and Relay's bounded default source-file ceiling
  • a registry.id change requests the required Relay and Notary review, signing, activation, and restart actions without reporting either value
  • invalid promotion baselines return a schema-valid, value-free blocked report
  • Relay and Notary startup diagnostics preserve cause-specific static codes and remediation while dropping paths, URLs, credentials, identifiers, parser text, and country values
  • documented journey, canonical-source, readiness, runtime-evidence, navigation, release-provenance, archive-link, and CI prerequisite claims are executable and accurate

Security, data minimization, and scope

  • portable reports exclude country values, secret values, and ambient filesystem paths
  • secret material remains environment- or file-bound and is validated by preflight rather than embedded in authored projects
  • --show-authored-values remains an explicit human-only trusted-local terminal option; it fails closed and never includes secrets or fixture values
  • generated references and journeys are schema-driven, deterministic, independently falsifiable, and checked for drift
  • signed product inputs remain separately owned by Relay and Notary, with conservative baseline handling for product retirement
  • the PostgreSQL diagnostic expansion exposes only a closed static cause class and remediation, never the underlying driver or database error

This is intentionally one cross-owning-area PR because the product contract spans the registryctl authoring model, generated Relay and Notary inputs, product-owned runtime diagnostics, release evidence, and the documentation generated from those same authorities. Splitting the corrections would temporarily publish contradictory schemas, generated references, runtime codes, and adopter instructions.

Verification

Cumulative verification through final head 3808fc39:

  • cargo fmt --all -- --check
  • cargo check --locked --workspace --all-targets
  • cargo clippy --locked --workspace --all-targets -- -D warnings
  • cargo test --locked --workspace
  • docs npm test: 197 passed and 1 intentional executable-journey skip
  • docs npm run check, including build, links, OpenAPI, frontmatter, style, archive, SEO, and machine-readable corpus checks
  • gitleaks dir --config .gitleaks.toml --no-banner --redact --timeout 120 .
  • first-country acceptance validator: 26 tests plus source-packet check
  • gate inventory: 166 gates plus 68 mutation/contract tests
  • PostgreSQL 16, 17, and 18 Notary conformance, including all eight negative modes
  • final CI-remediation delta: actionlint, 24/24 CI-routing tests, 112/112
    host project-authoring tests, the separately forced Linux-only portability
    test, three-shell parsing, tutorial dry run, shell lint, and value-free
    tutorial regression
  • independent correctness, security, maintainability, developer-experience, and final no-findings review

No hosted onboarding, country system, deployment, or production signing action was performed.

Remaining external acceptance

The implementation is locally verified. Fresh-reader and country-developer exercises, candidate-based signing and verification, real country semantics and interoperability, and operator recovery and rollback evidence remain release and adoption gates. This PR does not claim those human, country, governance, candidate, or operational outcomes in advance.

jeremi added 9 commits July 26, 2026 19:39
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
…onfig-latest

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
@codecov

codecov Bot commented Jul 26, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.23810% with 4 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
crates/registry-platform-ops/src/lib.rs 95.23% 4 Missing ⚠️

📢 Thoughts on this report? Let us know!

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
@jeremi

jeremi commented Jul 26, 2026

Copy link
Copy Markdown
Member Author

Review: first-country configuration product

Two rounds of review. I reviewed d192dfc3 first, re-reviewed ef01c22e after the fixes landed, and checked the ef01c22e..4e86feb4 delta. Every build and test below ran in an isolated detached worktree pinned to the commit under review, never in a live working tree, so failures are attributable to the commit rather than to concurrent edits.

Verdict: this is a strong response to review. The blocking problem is gone and the two highest-value findings were fixed properly. There is one correctness bug I would fix before merge, two stale numbers that need correcting, and red CI.


What is genuinely good

The baseline went from red to green. At d192dfc3 the branch did not compile. At ef01c22e, verified on a pristine tree:

Gate d192dfc3 ef01c22e
cargo fmt --check FAIL PASS
cargo check --locked --workspace --all-targets FAIL, 2x E0425 PASS
cargo clippy --locked --workspace --all-targets -- -D warnings FAIL PASS, zero warnings
cargo test --locked -p registryctl could not build PASS, 504 passed, 0 failed
docs npm ci && npm test FAIL, 13 ENOENT PASS, 194/195, 1 pre-existing skip

At PR head 4e86feb4, cargo fmt --check and cargo check --locked --workspace --all-targets both pass on a clean worktree as well.

More important than the count is the method. The two findings I cared most about were not patched, they were made structurally impossible:

  • The fabricated introduced_in provenance is now pinned at three layers: the schema constrains the release-history fields to type: null / maxItems: 0, the emitter writes None, and AuthoringConfigurationReference.astro throws at build time if any field carries a non-null introduced_in or a non-empty version_history. The page renders "Release history: Not verified" instead of inventing a version. The claim cannot come back without the build failing.
  • The prose-coverage metric now publishes reviewed_intent_assignments_using_reused_intent_count: 1248, which admits that 72% of assignments share boilerplate intent text, alongside the sentence "Assignment coverage means that every documented path resolves to reviewed intent. It does not mean that every path has unique prose." Publishing that number was not required by anything. It is the most candid thing in the change set.

The negative control in crates/registryctl/src/project_authoring/diagnostics.rs:2091 is the best test in the PR. It asserts the control parser error carries the sentinel before asserting the diagnostic does not, so it proves the leak path exists before proving it is closed. every_production_current_format_ingress_routes_through_schema_authority does the same thing for a grep-based gate by mutating the source to remove a route and asserting rejection. Both patterns are worth copying to the remaining unfalsifiable gates.

Also verified: the redaction default is byte-for-byte unchanged (Public count still 6), the new --show-authored-values path is opt-in, human-format only, fail-closed, and excludes secret-reference and fixture-sourced values; registryctl gives real file:line:col diagnostics for syntax errors, which makes the "use registryctl check instead of raw startup errors" delegation legitimate; all nine commits are DCO-signed.


Fix before merge

1. check reports success for a project that cannot work

This is the one I would fix first in the whole review. Take the http starter unmodified and change the consultation input mapping to reference an identifier nothing defines:

input:
  person_id: request.target.identifiers.ghost_field   # was registry_person_id
$ registryctl check --project-dir proj --environment local
Registry Stack project: fictional-citizen-registry (valid)
Fixtures: 24/24 passed
$ echo $?
0

Full green, exit 0, on a project whose only consultation can never bind its input.

The cause is the fixture boundary. Fixtures inject at the consultation input directly (input: { person_id: AB-123456 }), which is downstream of the request.target.identifiers.* mapping. No fixture in the starter references registry_person_id or identifiers at all, so the one piece of wiring an adopter is most likely to get wrong is exercised by zero of the 24 fixtures whose passing count is printed as reassurance.

A green check on a broken project is worse than any artifact-honesty problem in this review, because the adopter has no reason to look further. Suggested fixes in order of value: validate at check time that every request.* path in an input mapping resolves against the request contract; let fixtures optionally supply a full request so the mapping is covered; at minimum, qualify the printed count so it does not read as end-to-end.

2. Semantic-validation diagnostics carry an empty JSON pointer

Field-addressed errors are the headline UX promise here. That promise is met for parse and schema errors and unmet for the entire semantic phase, which is where real authoring mistakes land. Five distinct mistakes, all producing byte-identical output:

Mistake Pointer
credential_profiles.claims names an unknown claim ""
consultations.integration names an unknown integration ""
a claim cel references an unknown consultation ""
disclosure set to an invalid mode ""
validity set to an unparseable duration ""
{ "code": "registryctl.authoring.project.invalid",
  "addresses": [ { "file": "registry-stack.yaml", "pointer": "" } ],
  "phase": "semantic_validation",
  "cause": "The project declaration is invalid." }

In human format there is no line or column either. Worth separating this from the redaction policy, because it is not a redaction decision: a pointer such as /services/person-verification/consultations/person_record/integration contains only key names the adopter typed into their own file and discloses no value. The pointer is empty because the semantic validator does not track where it was.

3. check --format json returns empty stdout on compile-stage errors

$ registryctl check --project-dir proj --environment local --format json
        (empty stdout)
$ echo $?
1
$ registryctl check ... --format json 2>&1 >/dev/null
Error: direct claim references an unknown output

No schema_version, no code, no file. The documented contract is registryctl.project_diagnostics.v1 and this path emits no document at all, so registryctl check --format json | jq .status fails to parse rather than reporting "invalid". That is a fail-open for tooling. Raised from compiler/notary.rs:619 as a bare anyhow!. I probed six adopter-reachable mistakes and one hit this path, so it is narrow but real. Either convert compile-stage failures into diagnostics, or add a catch-all so --format json always emits a valid document on non-zero exit.

All three were verified at ef01c22e and their code paths (project.rs, compiler/notary.rs, fixtures.rs) are untouched by ef01c22e..4e86feb4, so they stand at PR head.


Two numbers that need correcting

The parity keyword count is now 9, not 22. The rule_coverage block attributing 22 keywords was deleted rather than backed by tests, and replaced with per-case expected_failing_keywords that are genuinely verified by executing each mutation and harvesting the keywords that actually fail. That is the right correction and I would rather have a true 9 than a decorative 22. But the observed-and-declared union is now additionalProperties, const, minItems, minimum, not, oneOf, pattern, required, type, and the 30-keyword vocabulary inventory is never compared against the observed set, so 21 remain unexercised. Any doc or summary that inherited "22 keywords covered" now states a false number.

The command offered as proof of schema/Rust agreement does not detect divergence. Verified twice on a pristine tree: delete "services" from project.schema.json's required array and

$ cargo test --locked -p registryctl --test project_authoring_schema_parity
test result: ok. 8 passed; 0 failed

stays green. The test that catches it is canonical_schema_language_is_structurally_contained_by_derived_dtos in crates/registryctl/src/project_authoring/schema_authority.rs:

schema_authority.rs:1658:37: project containment failed:
project: DTO requires fields absent from published requirement ["services"]

The protection is real and the prover is sound (roughly 20 negative controls, and it rejects DTO if/then/else rather than guessing). Only the pointer to it is wrong. Anyone told "run the parity target to verify schema and Rust agree" gets a false green.


Smaller items

  • Dimension labels in parity-coverage.json are still unverified free text. Nothing ties a case's declared dimension to its observed keyword: project-wrong-type sits in dimension type while empirically tripping const. Same defect class that was just fixed for keywords. Only the tail keyword is recorded, so the JSON pointer is discarded and two unrelated sites tripping the same keyword are indistinguishable.
  • Two vacuous asserts were missed by the redaction sweep, at crates/registry-relay/tests/config_schema.rs:1112 and crates/registry-notary/tests/config_schema.rs:791: assert!(!text.contains("SCHEMA_MUST_NOT_READ_OR_EMIT_THIS_SECRET_VALUE")) where text never receives the marker. The other use of that same marker at relay :972 and notary :273 is correctly planted as an env var, so this reads as an oversight rather than a policy. Test hygiene only; the underlying redaction behaviour is separately verified working.
  • release/exercises/product-input-lifecycle/ still contains only schema and template files, so the CI lifecycle validator passes over zero records: product-input lifecycle discovery passed: 1 non-evidence template(s), 0 candidate evidence record(s), exit 0. There is nothing to mutate, because the input set is empty. Either populate it or drop the gate.
  • docs/site/scripts/registryctl-release-provenance.test.mjs (new, 87 lines) has 19 assert.match calls and zero execFile. It asserts that the docs say /The `v0\.13\.0` binary cannot complete this workflow/ without running that binary. A test whose subject is a claim about an executable's behaviour, implemented as a regex over prose.
  • dto-shape-contract.v1.json is consumed only by its own drift test. It regenerates byte-identically and the drift gate is falsifiable, but the containment prover uses the live DTO schema, not the committed bytes, so the 102 KB file is a reviewability snapshot rather than a load-bearing input. Worth knowing before treating it as a contract. Its failure output also dumps a raw byte array, roughly 846 KB of unreadable diff.
  • Duplicated generated output: six byte-identical pairs, 5,070,758 bytes, between docs/site/public/generated/ and docs/site/src/data/generated/, written to four destinations by generate-authoring-reference.mjs. No .gitignore entry and no build-time copy step.
  • ops.md should say that an adopter who hand-edits a compiled config rather than the project has no diagnostic path, since registryctl only understands authored source.

Process notes

CI is red. Seven jobs failing at 4e86feb4, including Secret scan, Project authoring portability and determinism on both aarch64-macos and x86_64-linux, Rust tests (notary), and Notary state plane on PostgreSQL 16/17/18. The determinism job is directly relevant to several byte-reproducibility claims in this PR, and the Secret scan failure should be triaged first. Worth confirming these are pre-existing or environmental rather than introduced here.

Scope. AGENTS.md asks that a change stay scoped to one owning area. This PR spans 13: crates/registryctl (102 files), docs/site (87), crates/registry-relay (19), crates/registry-notary (13), release (11), crates/registry-notary-server (5), .github (4), products/notary (4), crates/registry-config-report (3), crates/registry-platform-ops (2), crates/registry-notary-core (2), products/manifest (1), crates/registry-notary-client (1). Not a blocker for a product-completion branch, but it makes review and revert harder, and it is worth an explicit note in the description saying why it is one PR.

Security-sensitive change needing explicit review notes. --show-authored-values changes what the tool will print to a terminal. The implementation is the right shape and I verified it fail-closed, but AGENTS.md asks for explicit review notes on data-minimization changes and I did not find them in the commit messages.


I have a longer written report behind this with per-finding evidence, mutation experiments, and the goal-level suggestions, which I am happy to share. Two items in the ef01c22e..4e86feb4 delta (the CI tutorial-selection change and the semantic_impact.rs additions) are still being verified and I will follow up if anything there changes the picture above.

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>

@jeremi jeremi left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comparative configuration-product follow-up at 4e86feb4.

The PR has the right major pieces: governed authoring, layered checks, semantic comparison, preflight, generated artifacts, promotion, and task-oriented docs. The remaining problem is that several published guarantees do not survive the real adopter path. In particular, preflight can report ready without a declared data file, a registry identity change does not request product re-sign/activation/restart, promotion can leave JSON automation with empty stdout, and the primary Verify route stops before performing the advertised evaluation. Inline comments contain direct repros and proposed contract-preserving fixes.

I also confirmed the current red CI has deterministic PR-local causes beyond the inline compile and OpenAPI-path comments: the registryctl tutorial checker still expects the renamed Inspect the generated contract heading; the docs test invokes zsh on an Ubuntu job that does not install it; docs tests start concurrent cargo run processes in an environment that has not prepared the Rust toolchain, producing rustup download/rename races; and the secret scan reports 11 matches in new intent, generated, and lifecycle-test data that need narrowly scoped fixture changes or allowlisting. The Notary test/state-plane failures still need separate triage.

I am leaving this as a comment review, not an approval. The complete object graph is substantially clearer, but the validation, evidence, and lifecycle boundaries below should be aligned before merge.

@@ -1959,7 +2103,10 @@ fn public_rhai_commands_accept_the_released_contract_for_an_unknown_product() {
})
.expect("product-neutral Rhai project builds");
assert_eq!(build_report.status, "built");
let output = PathBuf::from(build_report.output.expect("build output"));
let output = resolve_build_output(
&project_directory,

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P0] This test does not compile at the PR head. ProjectBuildOptions takes ownership of project_directory here, then resolve_build_output borrows it below, producing E0382 in the registryctl test, clippy, and both portability jobs. Pass project_directory.clone() into ProjectBuildOptions or otherwise retain a borrowable path, then rerun the workspace and portability gates.

}

for (entity_id, entity) in &environment.entities {
if let RecordProvider::Postgres { connection, .. } = &entity.provider {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Preflight omits file-backed entity providers. This adapter only records RecordProvider::Postgres; CSV, XLSX, and Parquet paths are validated as runtime paths elsewhere but never enter OfflinePreflightInput. I reproduced this with snapshot-exact: with secrets and the Notary token satisfied but /var/lib/registry/population.csv absent, preflight --format json returned status: "ready" and no diagnostic. Please add these provider paths as runtime-file checks and a missing-provider regression that must return not_ready.


fn comparison_ignores_field(scope: &SnapshotScope, instance_field: &str) -> bool {
matches!(scope, SnapshotScope::Project)
&& (instance_field == "/registry/id"

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Ignoring registry.id drops required product actions. Changing only this field changes generated Relay publisher/title and deterministic lock keys, but compare --format json surfaces only generated review/approval drift. Those synthetic fields have no Relay or Notary consumer, so the report says signing, activation, and restart are all none. Please compare /registry/id as a product-impacting field or attach affected products to the generated projection changes, and add a regression asserting the Relay actions.

preflight_project_rhai_scripts(&loaded)
.map_err(|_| anyhow!("promotion state could not be validated safely"))?;

let baselines = load_verified_promotion_baselines(options, &loaded)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] promote --format json exits before its published report contract on baseline failures. Tampered, legacy, malformed, or wrong-product signed baselines are mapped to a generic error here, and the tests currently assert empty stdout. Automation therefore cannot read a blocked disposition or closed reason. Please return a redacted registry.project.promotion.v1 report with a blocked disposition for these verification failures, or explicitly narrow the documented JSON contract.

Comment thread docs/site/astro.config.mjs Outdated
{ label: 'Integration patterns', slug: 'explanation/integration-patterns' },
{ label: 'DPI safeguards', slug: 'explanation/dpi-safeguards-alignment' },
{ label: 'Overview', slug: 'verify' },
{ label: 'Evaluate a registry-backed claim', slug: 'journeys/registry-backed-notary-claim' },

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] This makes an incomplete journey the primary Verify task. The generated journey initializes, checks, builds, and starts a Notary project, but never loads the evaluator key, calls /v1/evaluations, or shows active, pending, no-match, and ambiguity outcomes. Those runtime evaluation steps remain only in the draft detailed tutorial, which this sidebar no longer exposes. Keep the primary route on the complete tutorial until the generated journey includes the runtime interface, or add those executable steps and link the tutorial as a deep dive.

0,
{
id: "lifecycle-preflight",
kind: "readiness_gate",

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] This required readiness gate is neither passing nor exercised. Following the lifecycle commands through compare leaves three secrets and the Notary-to-Relay token file unprovisioned, so this preflight exits 1. The clean-temp test also filters to kind === "command", excluding every readiness_gate. Please make this an explicitly expected-failing diagnostic step, provision a passing example state, or add separate executable evidence for the passing preflight before claiming the journey reaches offline readiness.

proves: Static snapshot and evidence contracts are coherent.
does_not_prove: Runtime snapshot files are available.
- name: materialization
proves: One runtime loaded a concrete snapshot generation.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] This gate claims runtime evidence that the journey explicitly does not have. The snapshot journey stops at fixture/check/compare/build, while its runtime evidence is not_claimed; no runtime loads a concrete generation. Move materialization to the production delta or does_not_prove, or add a real runtime step and CI evidence. Otherwise the page crosses the authored/generated/runtime boundary it is meant to clarify.

error!(error = %err, "registry-relay exiting with failure");
if let Some(failure) = err.downcast_ref::<OperatorSafeConsultationActivationFailure>() {
failure.emit();
} else if err.downcast_ref::<ReportedConfigLoadFailure>().is_none() {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] The stable config-load diagnostic boundary is inconsistent across commands. The serve path wraps loader failures as ReportedConfigLoadFailure, but openapi calls config::load directly and emits both the specific loader diagnostic and this generic fallback. explain-config parses before the loader and emits only the generic fallback for malformed YAML. Route those paths through the same already-reported wrapper, and add child-process tests for the exact diagnostic codes of openapi and explain-config.

evidence_lifecycle: main_unreleased
availability: { status: current_unreleased, proof: source_tree, release: null }
canonical_sources:
- docs/site/openapi/registry-relay.openapi.json

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] This canonical source does not exist in a clean checkout. The committed contract is crates/registry-relay/openapi/registry-relay.openapi.json, so the clean-temp standard-journey test fails with ENOENT when it resolves this path. Update both this source entry and the artifact entry below, and keep the clean-checkout test as the guard.

@@ -101,7 +101,7 @@ URL, or monorepo path is part of the journey.
inspect the topology and call a Notary evaluation route directly.
- [Run a protected registry API locally](../../tutorials/publish-spreadsheet-secured-registry-api/):
generate a smaller Relay project from a sample workbook.
- [Evaluate a claim with Registry Notary](../../tutorials/verify-claim-registry-api/):
- [Evaluate a claim with Registry Notary](/journeys/registry-backed-notary-claim/):

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Keep this link relative or docset-base-aware. A root-relative /journeys/... target escapes a future /v/<release>/ archive and sends readers to the current docs root; the archive link checker intentionally rejects that pattern for normal internal links. Use the relative link form used by the surrounding task links or the shared base-aware mechanism.

@jeremi

jeremi commented Jul 26, 2026

Copy link
Copy Markdown
Member Author

Follow-up: CI triage, and a correction to my baseline claim

Correction first. My comment above reported the baseline going red to green from local runs. That verification was macOS-only and it was structurally blind to the thing that matters most here: this branch does not compile on Linux, and has not compiled on Linux at any commit on the branch. cargo check --locked --workspace --all-targets passes for me and fails in CI because the broken test sits behind #[cfg(target_os = "linux")]. The table I posted is accurate about what it measured and gives a misleading impression of the branch's actual state. Treat this section as superseding it.

CI has since finished the rest of its jobs. It is 14 red, not the 7 I saw mid-run. Triage below separates what this branch introduced from what is runner infrastructure.

1. Blocker: the branch does not compile on Linux

crates/registryctl/tests/project_authoring.rs:2107, E0382:

2079 |     let exercise = |project_directory: PathBuf| {
2099 |             project_directory,      <- value moved here
2107 |             &project_directory,     <- value borrowed here after move

project_directory is moved into ProjectBuildOptions at 2099 and borrowed by resolve_build_output at 2107. One-line fix: clone at 2099, or hoist the resolve_build_output call above the move.

This one error fails three jobs: Rust format and clippy, Rust tests (registryctl), and Project authoring portability and determinism (x86_64-linux).

I checked all nine commits on the branch. The move-then-borrow is present in every one of them, starting at ba60456a. Combined with the #[cfg(target_os = "linux")] gate on the enclosing test, that means public_rhai_commands_accept_the_released_contract_for_an_unknown_product has never compiled and has never run, at any point in this branch's history.

That is worth stating separately from the mechanical fix. It is the same pattern as the rest of my review, in its purest form: not an assertion that cannot fail, but an assertion that never executed at all. A #[cfg(target_os = "linux")] test authored on macOS needs one Linux run before it can be cited as evidence of anything.

2. The value-free diagnostics rewrite broke the operator error contract

This is the finding I would rank hardest in the whole review, above everything in my first comment except the green-check-on-a-broken-project issue.

Notary state plane fails on PostgreSQL 16, 17 and 18:

notary PostgreSQL conformance failed: read-only startup did not report the expected closed error

What the runtime actually emitted:

ERROR notary.runtime.activation_failed: Registry Notary runtime activation failed;
next action: restore the governed runtime dependency or integrity condition, then retry activation

products/notary/scripts/postgresql-conformance.sh is untouched by this PR. The runtime message changed underneath it. That script asserts six distinct operator-facing messages across eight startup failure modes:

Failure mode Expected message
read-only database Notary PostgreSQL database is unavailable
catalog drift Notary PostgreSQL state schema is incompatible
fingerprint drift Notary PostgreSQL state schema is incompatible
permission drift Notary PostgreSQL state schema is incompatible
incompatible role Notary PostgreSQL runtime role is incompatible
unsupported major Notary PostgreSQL server major is unsupported
database unavailable Notary PostgreSQL database is unavailable
TLS failure Notary PostgreSQL database is unavailable

All of them now collapse into one generic sentence asking the operator to "restore the governed runtime dependency or integrity condition."

This is the same defect class as finding #2 in my first comment, the empty JSON pointer on semantic validation, but worse on two axes: it is the production operator path rather than the authoring path, and it collapses distinguishable causes into a single indistinguishable message. An operator running an unsupported Postgres major and an operator with a read-only database now receive identical text. Neither can act on it, and state doctor is a separate command they have no reason to reach for when the message does not hint that the database is the problem.

Data minimization is a legitimate goal for these messages. But Notary PostgreSQL server major is unsupported contains no operator data. The redaction looks applied at the wrong granularity: the cause was removed along with the values. Keeping a stable code plus a cause-specific static string satisfies both goals, and that is exactly the shape the authoring diagnostics already use.

Worth saying plainly: a pre-existing, genuinely falsifiable gate caught a real regression. The conformance script is doing exactly what a good gate is supposed to do. I would treat its failure as the signal here, not as an obstacle to route around.

3. A new redaction test fails

crates/registry-notary/tests/startup_redaction.rs:282, in signed_bundle_boot_failure_does_not_expose_governed_paths_or_values:

assertion failed: combined.contains(&format!("result=\"{}\"", code.as_str()))

The file is new in this PR (288 lines, absent from origin/main), so this is introduced rather than pre-existing. The test asserts the stable code is present in the output and it is not, which is consistent with item 2.

4. Tutorial heading drift, red since the first commit

registryctl tutorials from source fails on a heading mismatch:

  • rendered: Inspect the editable local scaffold
  • expected: Inspect the generated contract

The prose was renamed in ba60456a; docs/site/scripts/check-registryctl-tutorials.sh still expects the old heading at lines 307, 357 and 359. Mechanical, but it means this gate has also been red for the branch's entire life.

5. A second prerequisite-ordering bug

Project authoring portability and determinism (aarch64-macos) fails for a different reason than its Linux twin:

ENOENT: no such file or directory, realpath '.../docs/site/openapi/registry-relay.openapi.json'
  at canonicalRepositoryPath (generate-standard-journeys.mjs:1149)
  at assertCanonicalSources (generate-standard-journeys.mjs:1170)

docs/site/openapi/ is not tracked in the tree, so assertCanonicalSources requires the OpenAPI documents to have been generated before the standard-journeys test runs, and this job does not generate them.

This is the same class as the cargo build --locked -p registryctl --bin registryctl ordering fix in the delta, which was a good catch. There is a second instance of it here.

6. Secret scan: 11 findings, no credentials

I triaged this locally in a clean checkout of the head commit rather than speculating, and deliberately did not reproduce any matched values. All 11 are false positives and none is a credential exposure, which is why I am raising it here rather than through SECURITY.md:

  • release/scripts/test_validate_product_input_lifecycle.py:590-591 (2 findings, rules aws-access-token and jwt): deliberately planted sentinels inside test_secret_and_location_sentinels_are_rejected_before_retention, whose entire purpose is to prove the validator rejects them. This is exactly the negative-control pattern I praised in my first comment. Good test, unlucky scanner.
  • 9 findings on a "key_path" field, three each in docs/site/src/data/generated/configuration-reference.json, docs/site/public/generated/configuration-reference.v1.json, and crates/registry-notary-core/config/documentation-intent.json: dotted configuration key paths in generated documentation, matched by the generic-api-key rule.

Fix is scoped .gitleaks.toml allowlist entries, narrowed by path and rule. Please avoid a blanket rule disable, since the planted sentinels are precisely the thing you want the scanner to keep catching elsewhere.

7. Docs checks: mostly runner infrastructure

3 of 195 failed. Two are the runner failing to download a rustup component:

error: component download failed for clippy-x86_64-unknown-linux-gnu:
could not rename 'downloaded' file ... No such file or directory (os error 2)

hitting authoring-reference-docs.test.mjs:107 and diagnostic-reference-docs.test.mjs:78. Not yours.

The third, generate-standard-journeys.test.mjs:570, reports zsh rejected rendered commands: undefined with null !== 0. A null exit status means the process was signalled or never started, not that zsh rejected anything, so this also looks environmental. The assertion message is misleading enough to be worth a guard: if zsh is missing or killed, the test should skip or say that, rather than blame the rendered commands.

8. The two delta items I owed you

start/quickstart.mdx is still orphaned. Zero inbound references anywhere in docs/site/src or astro.config.mjs at PR head. The Start sidebar section lists four items and does not include it, and there is no autogenerate. The delta only retargeted an outbound link inside the page. start/credential-tour.mdx is orphaned by the same check.

I confirmed no generic orphan detection exists by adding a page linked from nowhere and running the suite: information-architecture.test.mjs reported 7 pass / 0 fail, and full npm test stayed green at 194 pass / 1 skip / 0 fail. The probe was removed afterwards. The closest existing test runs the reverse direction only, sidebar slug to page existence.

authoring-reference-sources.json is genuinely verified, and I want to correct any impression that everything in this change set is self-referential. It is cross-checked from several independent directions, each check is provably falsifiable, and test_ci_changes.py carries a planted negative control. This is real evidence and it is the pattern the rest of the artifacts should follow. One narrow gap: ci_inputs.sample entries are never checked for existence on disk, so a path that does not exist still passes (24/24 Python, 8/8 node). A three-line Path(sample).exists() loop closes it.

One honest correction to my own round-1 wording on the standard-journeys test: I said it "could not fail". That was too broad. Its source_label and availability assertions are falsifiable literals, and availability genuinely derives from the YAML. The specific heading and id assertions I flagged do remain incapable of failing, because they compare imported constants against themselves.


Net. Items 1 through 5 are all introduced by this branch and all are currently red. Items 6 and 7 are noise you can clear cheaply. Item 2 is the one I would look at hardest, because it is not a mechanical break: it is a design question about how much a diagnostic may withhold from an operator before it stops being a diagnostic, and the answer this branch currently gives is "too much".

jeremi added 8 commits July 27, 2026 00:40
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7437841cf3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/scripts/ci_changes.py Outdated
Comment on lines +524 to +525
registryctl_tutorial = (
complete or tutorial_infrastructure or bool(affected & TUTORIAL_PACKAGES)
complete or tutorial_infrastructure or tutorial_source_under_test

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Restore dependency-based tutorial routing

When a PR changes tutorial-critical code outside this short allowlist—such as crates/registryctl/src/project_authoring/commands.rs, fixtures.rs, Relay consultation code, or the platform config/auth/crypto crates—the classifier now sets registryctl_tutorial to false even though check-registryctl-tutorials.sh builds and exercises those components. The previous affected & TUTORIAL_PACKAGES condition covered these dependencies, so this regression lets changes to the end-to-end first-country journey merge without running the workflow job gated by this output; retain dependency/package-based routing rather than enumerating only a few source files.

Useful? React with 👍 / 👎.

Comment thread .gitleaks.toml
'''let key = Pkcs1RsaPrivateKey''',
'''chain_key_epoch_id:\s*[a-z0-9-]+-chain-1''',
'''"dci_crvs_api":\s*"5e31d1e381d4bd8c7c74112d714fd49d263c6df7"''',
'''"key_path":\s*"oid4vci\.(?:enabled|nonce\.ttl_seconds|proof\.max_age_seconds)"''',

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Add the required DCO sign-off

The reviewed commit contains no Signed-off-by: trailer, so it lacks the repository-required DCO attestation and must be recreated with git commit -s before it can be accepted.

AGENTS.md reference: AGENTS.md:L70-L70

Useful? React with 👍 / 👎.

Comment thread docs/site/package.json
@@ -14,7 +14,8 @@
"assemble:archives": "node scripts/assemble-archives.mjs",
"archive:snapshot": "node scripts/archive-lock.mjs snapshot",
"preview": "astro preview",
"generate": "node scripts/generate-data.mjs && node scripts/generate-project-starters.mjs && node scripts/fetch-openapi.mjs && node scripts/sync-repo-docs.mjs && node scripts/generate-sidebar.mjs",
"generate": "node scripts/generate-data.mjs && node scripts/generate-project-starters.mjs && node scripts/generate-authoring-reference.mjs && node scripts/generate-diagnostic-references.mjs && node scripts/fetch-openapi.mjs && node scripts/generate-standard-journeys.mjs && node scripts/sync-repo-docs.mjs && node scripts/generate-sidebar.mjs",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Split the change by owning area

This commit simultaneously changes crates/, products/, docs/site/, and release/—in addition to root CI—despite the repository requiring each change to remain within one owning area. Split these changes so the product implementation, generated documentation, and release evidence can be reviewed and rolled back independently.

AGENTS.md reference: AGENTS.md:L77-L78

Useful? React with 👍 / 👎.

Comment thread .github/workflows/ci.yml
Comment on lines +643 to +644
sudo apt-get update
sudo apt-get install --yes zsh

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Avoid apt-get on the macOS matrix leg

When project-authoring-determinism runs its declared macos-14/arm64 matrix leg, this unconditional step invokes Ubuntu's apt-get, which is not available on the macOS runner, so every registryctl change selecting this job fails before the journey tests execute. Restrict this installation to Linux or skip it on macOS, where the requested shell is already part of the runner environment.

Useful? React with 👍 / 👎.

Comment thread .github/workflows/ci.yml Outdated
Comment on lines +743 to +744
- name: Validate product-input lifecycle records
run: python3 release/scripts/validate-product-input-lifecycle.py --discover release/exercises

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Prepare candidate assets before lifecycle discovery

As soon as the documented workflow adds a real candidate_evidence record under release/exercises/product-input-lifecycle/, this CI command always fails because it supplies no --candidate-asset-root; validate_candidate_assets explicitly rejects candidate evidence when that argument is absent. The later upgrade-asset preparation does not help because it runs afterward and scans only top-level upgrade records, so CI currently validates only the template and cannot ever accept the first real product-input lifecycle record.

Useful? React with 👍 / 👎.

def load_json(path: Path, *, max_bytes: int = MAX_RECORD_BYTES) -> Any:
require_regular_file(path, max_bytes=max_bytes)
try:
return json.loads(path.read_text(encoding="utf-8"))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Reject duplicate fields in acceptance records

If a first-country acceptance record repeats a security-sensitive field such as record_kind, an outcome, or an evidence digest, plain json.loads silently keeps the last occurrence and the validator may accept an object that another JSON consumer interprets differently. This makes the release-evidence record ambiguous at exactly the provenance boundary the validator is meant to close; load with a duplicate-detecting object_pairs_hook, as the sibling product-input lifecycle validator already does.

Useful? React with 👍 / 👎.

Comment thread docs/site/package.json
@@ -14,7 +14,8 @@
"assemble:archives": "node scripts/assemble-archives.mjs",
"archive:snapshot": "node scripts/archive-lock.mjs snapshot",
"preview": "astro preview",
"generate": "node scripts/generate-data.mjs && node scripts/generate-project-starters.mjs && node scripts/fetch-openapi.mjs && node scripts/sync-repo-docs.mjs && node scripts/generate-sidebar.mjs",
"generate": "node scripts/generate-data.mjs && node scripts/generate-project-starters.mjs && node scripts/generate-authoring-reference.mjs && node scripts/generate-diagnostic-references.mjs && node scripts/fetch-openapi.mjs && node scripts/generate-standard-journeys.mjs && node scripts/sync-repo-docs.mjs && node scripts/generate-sidebar.mjs",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep current-source generators out of archived builds

When buildDocsetArchive sets DOCS_DOCSET to an archived release, it still calls this shared generate script, and the newly added authoring-reference and standard-journey generators ignore the selected docset and read the current checkout instead (generate-standard-journeys.mjs even hard-codes Main source (unreleased)). As a result, rebuilding or bootstrapping an immutable v0.13 archive injects current unreleased registryctl contracts and journeys into the historical site, causing either archive-lock mismatches or historically incorrect archive content; make these generators docset-aware or exclude them from archived generation.

Useful? React with 👍 / 👎.

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3808fc39df

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

def load_json(path: Path, *, max_bytes: int = MAX_RECORD_BYTES) -> Any:
require_regular_file(path, max_bytes=max_bytes)
try:
return json.loads(path.read_text(encoding="utf-8"))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Read acceptance records through one no-follow descriptor

When the record resides in a directory that can be modified concurrently, an attacker can replace it after require_regular_file() performs lstat() but before this second path-based open; read_text() may then follow a symlink or read bytes whose size was never checked, so the validator can accept a different record than the bounded regular file it inspected. Open once with no-follow semantics, validate the descriptor metadata, and parse bytes from that same descriptor, as the sibling lifecycle validator does.

AGENTS.md reference: AGENTS.md:L79-L81

Useful? React with 👍 / 👎.

Comment on lines +413 to +416
"for-each-ref",
"--format=%(contents)",
"--count=1",
f"refs/tags/{version}",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Require the version ref to be an annotated tag

When refs/tags/<version> is a lightweight tag whose target commit message uses the closed release-candidate binding format, git for-each-ref --format=%(contents) returns that commit message, and parse_tag_binding accepts it even though no annotated tag object exists. The lifecycle validator can therefore report a valid annotated-tag provenance binding without the artifact it claims to verify; check the ref's object type and require tag before parsing its contents.

AGENTS.md reference: AGENTS.md:L79-L81

Useful? React with 👍 / 👎.

review_classes: field.review_classes.clone(),
affected_subjects: subjects.clone(),
occurrences: 1,
requirements: requirements_for_consumers(&field.consumers),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Limit semantic actions to enabled products

For a single-product project, this derives requirements directly from the schema-wide field catalog rather than the current and baseline product topology. For example, changing registry.id in the maintained relay-only-records project emits notary_config plus actions to re-sign, reactivate, and restart Notary even though that project generates only Relay inputs. These machine-readable actions are incorrect for adopters and automation; filter consumers, generated artifacts, and requirements using the union of the current and baseline topologies so removed-product obligations are still retained.

Useful? React with 👍 / 👎.

async fn run(args: Args) -> Result<ExitCode, Box<dyn std::error::Error>> {
let env_report = load_env_file_arg(args.env_file.as_deref(), args.env_file_override)?;
let server_startup = args.command.is_none();
let doctor_command = matches!(&args.command, Some(Command::Doctor { .. }));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Classify env-file failures for state doctor

When registry-notary state doctor is invoked with a missing or malformed --env-file, this predicate does not recognize the nested doctor command, so the error takes the raw branch below and bypasses the new NotaryActivationFailure catalog. The command then emits an OS- or parser-dependent message instead of the stable notary.configuration.invalid diagnostic expected from the cause-specific state-doctor boundary; include Command::State { command: StateCommand::Doctor } in the value-free configuration-failure path.

Useful? React with 👍 / 👎.

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d01915c1a1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +1205 to +1206
let current_number = current.as_f64();
let baseline_number = baseline.as_f64();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Compare integer bounds without f64 rounding

When comparing integer limits above 2^53—which are valid for fields such as entity max_recordsValue::as_f64() can round distinct integers to the same value. For example, lowering max_records from 9007199254740993 to 9007199254740992 makes these floats equal, so the upper-bound branch reports the change as Widened rather than Narrowed; this reverses the machine-readable risk classification. Compare the JSON integer representations exactly before falling back to floating-point values.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant