Bind release conformance evidence to immutable candidates#459
Conversation
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 72046facc3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| for value, label in ((source_ref, "source_ref"), (target_commit, "source_commit")): | ||
| resolved = subprocess.run( | ||
| ["git", "rev-parse", "--verify", f"{value}^{{commit}}"], |
There was a problem hiding this comment.
For candidate evidence records this loop only proves that target_release.source_commit is a valid commit and that source_ref is an ancestor; it never resolves refs/tags/{target_release.version} to that commit. If an operator points the record at an untagged descendant containing a matching manifest, the manifest/artifact checks can still pass while the record claims the versioned release candidate, so the upgrade evidence is not actually tied to the immutable tag. Resolve the version tag and compare it with target_commit before accepting the record.
AGENTS.md reference: AGENTS.md:L79-L81
Useful? React with 👍 / 👎.
| for path in records: | ||
| data = json.loads(path.read_text(encoding="utf-8")) | ||
| validate_record(data, allow_template=data.get("record_kind") == "template") |
There was a problem hiding this comment.
When a candidate_evidence JSON is added under release/exercises, the CI step invokes --discover, but this branch calls validate_record without require_all_passed and ignores --require-pass entirely. That means a committed record with candidate_frozen false, failed/not_run checks, or P/T artifact drift is reported as discovery-passed instead of failing the release-provenance gate; pass the promotion flag for non-template records or add a separate promotion discovery mode.
AGENTS.md reference: AGENTS.md:L79-L81
Useful? React with 👍 / 👎.
| for kind in ("binaries", "image_inputs"): | ||
| if len({artifacts[f"{phase}{run}_{kind}"] for phase in ("p", "t") for run in (1, 2)}) != 1: | ||
| raise ExerciseError(f"--require-pass rejects P/T {kind} drift") | ||
| if artifacts["p_release_inputs"] != artifacts["t_release_inputs"]: | ||
| raise ExerciseError("--require-pass rejects P/T release-input drift") |
There was a problem hiding this comment.
candidate_artifact_set now records separate p_notary_layouts/t_notary_layouts and p_relay_layouts/t_relay_layouts, but --require-pass only compares binaries, image inputs, and release inputs. A promotion record can therefore pass all structural and promotion checks while the retained image layout identities differ between P and T, leaving the release evidence unable to prove the candidate image artifacts were reproduced unchanged; include the layout pairs in the P/T drift check.
AGENTS.md reference: AGENTS.md:L79-L81
Useful? React with 👍 / 👎.
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cd293a07c9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| value = images[component] | ||
| if ( | ||
| not isinstance(value, str) | ||
| or re.fullmatch(rf"{re.escape(repository)}@sha256:[0-9a-f]{{64}}", value) |
There was a problem hiding this comment.
When relay-oidc-smoke.py run --image-lock is given a locally edited lock, this check only proves the image strings look like ghcr.io/registrystack/...@sha256:<hex>; it never ties the lock bytes to the signed release asset, capsule, or SHA256SUMS. A forged lock can keep the correct tag/manifest refs but swap in a different Relay digest, after which the smoke verifies the running container against that forged value and emits a passing report for a non-release image. Please bind the lock to the immutable release asset before accepting its image digests.
AGENTS.md reference: AGENTS.md:L79-L81
Useful? React with 👍 / 👎.
| endpoint = urllib.parse.urlsplit(args.suite_offer_endpoint) | ||
| if ( | ||
| (endpoint.scheme, endpoint.netloc) != (base.scheme, base.netloc) | ||
| or endpoint.scheme not in {"http", "https"} |
There was a problem hiding this comment.
When an operator points --conformance-server and --suite-offer-endpoint at an http:// suite origin, this allowlist still accepts it and cmd_submit_offer sends the inline credential_offer query value, which read_offer has already confirmed contains a Notary pre-authorized code. That code is then exposed over cleartext transport even though the adapter is meant to keep the owner-only offer on the pinned suite origin; require HTTPS, or restrict HTTP to a verified loopback-only endpoint, before submitting real offers.
AGENTS.md reference: AGENTS.md:L79-L81
Useful? React with 👍 / 👎.
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 12652ff96f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| "source_ref": source_ref, | ||
| "source_tag": target["version"], | ||
| } | ||
| if not isinstance(stack, dict) or any(str(stack.get(key)) != value for key, value in expected.items()): |
There was a problem hiding this comment.
In the release-tool path I inspected, --discover promotes non-template records with require_all_passed, but this identity check never constrains stack.status. If a target tag is cut from a manifest that still says draft (or any other non-candidate lifecycle state) while the version/source fields match, the record can satisfy --require-pass and be accepted as promotion evidence even though the target release was not frozen as a release candidate. Require release-candidate (or the reviewed released closeout) before accepting the target manifest.
AGENTS.md reference: AGENTS.md:L79-L81
Useful? React with 👍 / 👎.
| expected = { | ||
| "manifest": record["target_release_manifest"]["sha256"], | ||
| "relay_image": record["target_release"]["relay_image_digest"], | ||
| "notary_image": record["target_release"]["notary_image_digest"], | ||
| } |
There was a problem hiding this comment.
For a candidate evidence record, this expected-coordinate check verifies the manifest and product image digests but leaves candidate_artifact_set.artifacts.image_lock as any syntactically valid SHA-256. A record can therefore pass --discover/--require-pass with an arbitrary image-lock digest while the rest of the target coordinates are correct, so the upgrade evidence no longer proves which registryctl lock pinned the exercised images. Compare the image-lock digest to the exact release image-lock asset (or capsule/SHA256SUMS binding) before accepting promotion evidence.
AGENTS.md reference: AGENTS.md:L79-L81
Useful? React with 👍 / 👎.
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c429e2edbf
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| python3 release/scripts/validate-upgrade-exercise.py --template | ||
| release/exercises/upgrade-exercise-v1.template.json | ||
| - name: Validate committed upgrade exercise records | ||
| run: python3 release/scripts/validate-upgrade-exercise.py --discover release/exercises |
There was a problem hiding this comment.
Checked the release-tool workflow: this discovery command does not pass --candidate-asset-dir, but non-template discovery now calls validate_image_lock_binding, which fails immediately when that argument is absent. As soon as a real candidate_evidence JSON is committed under release/exercises, CI will reject it before the promotion record can be validated, so the release-provenance gate cannot accept reviewed candidate evidence unless the workflow downloads/provides the authenticated release asset directory or uses a separate structural-only mode.
AGENTS.md reference: AGENTS.md:L79-L81
Useful? React with 👍 / 👎.
| ) | ||
|
|
||
| capsule_sha256 = sha256(capsule_path) | ||
| verify_release_authenticity(asset_dir, tag, (lock_name, capsule_name)) |
There was a problem hiding this comment.
When the candidate asset directory is writable during the run, this verifies the image-lock and capsule by pathname after hashing/parsing earlier reads, then only checks that the paths have the original hashes at the end. A local swapper can present the signed release subjects while cosign/slsa-verifier run and swap the previously parsed forged files back before the final hash check, so the smoke/upgrade evidence can be bound to bytes that were never authenticated. Copy the closed asset set through no-follow descriptors (as the integration evidence runner does) or otherwise hold stable file descriptors before parsing and authenticity verification.
AGENTS.md reference: AGENTS.md:L79-L81
Useful? React with 👍 / 👎.
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
| import json | ||
| import sys | ||
| import tempfile | ||
| import unittest |
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2b777d9638
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if manifest_bytes == tagged_manifest_bytes: | ||
| return |
There was a problem hiding this comment.
If a candidate tag is cut after the release manifest already says stack.status: released, the local and tagged manifest bytes are identical and this early return accepts it without proving the reviewed release-candidate-to-released closeout transition. That lets Relay smoke or upgrade evidence bind to a tag that was never frozen as a release candidate; require the tagged manifest to be release-candidate when bytes match, and allow released only through the explicit one-line closeout comparison. Fresh evidence since the prior status comment is that the new closeout helper now returns before checking the tagged status when the bytes are equal.
AGENTS.md reference: AGENTS.md:L79-L81
Useful? React with 👍 / 👎.
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
2 participants
What changed
Why
A structurally complete report was not enough to prove that evidence came from the candidate being promoted. The release packet must fail closed when Git lineage, manifest bytes, image locks, or evidence identity drift.
Implementer impact
Maintainers get a candidate-neutral preparation workflow with explicit structural and promotion checks. Live conformance remains an owner-run release exercise, and sensitive outputs remain outside the public evidence record.
Validation
python3 -m unittest release/scripts/test_validate_upgrade_exercise.py release/scripts/test_relay_oidc_smoke.py release/scripts/test_openid_conformance_runner.py release/scripts/test_check_gates_inventory.py(96 passed)python3 release/scripts/check-gates-inventory.py(80 gates, no inert nested workflows)git diff --check origin/main..HEADIssue status
Progresses #203 and #205 but does not close them. Closure still requires an immutable published candidate, the real P/T upgrade and recovery exercise, owner-run OIDF/Relay execution, reviewed redacted evidence, and promotion approval.