Skip to content

feat(contracts): verify registry-fetch smoke check + update vendored $id to resolvable Pages URL#9

Merged
OgeonX-Ai merged 3 commits into
mainfrom
feat/registry-fetch-smoke-check
Jul 8, 2026
Merged

feat(contracts): verify registry-fetch smoke check + update vendored $id to resolvable Pages URL#9
OgeonX-Ai merged 3 commits into
mainfrom
feat/registry-fetch-smoke-check

Conversation

@OgeonX-Ai

Copy link
Copy Markdown
Contributor

Summary

Wires cas-evals as the consumer-verification repo for REQ-1.4.11 (schema registry publishing). Two additive changes:

  1. Updates the offline vendored $id expectations in verify_vendored_contract() and the re-vendored vendor/cas-contracts/v0.1.0/*.schema.json copies (+ recomputed provenance.json sha256) to the new resolvable Pages registry URL convention (https://coding-autopilot-system.github.io/cas-contracts/registry/...), matching cas-contracts#18.
  2. Adds a new cas_evals.registry_check module (stdlib urllib, no new deps) + unit tests + a dedicated registry-smoke CI job that performs live HTTP GETs against the Pages registry and asserts HTTP 200, proving the registry actually resolves — something the existing offline vendored-contract check cannot prove by design.

Link to cas-contracts#18

This PR is a companion to Coding-Autopilot-System/cas-contracts#18 (fix/registry-resolvable-id), which rewrites all 22 schema $id values in cas-contracts from the dead schemas.coding-autopilot.dev namespace to the live Pages registry URL.

The vendored $id update in this PR is independent of whether #18 has merged — the vendored copy under vendor/cas-contracts/v0.1.0/ is a local, deliberately re-vendored snapshot edited to match the portfolio-wide $id convention; it does not fetch from cas-contracts at build or test time.

The new registry-smoke CI job also does not depend on #18 merging: it checks paths (index.json, v0.1/manifest.json, v0.1/common.schema.json, v0.1/evaluation-result.schema.json) that already resolve 200 today via the existing pages.yml publisher from prior phases. Merging #18 only changes the $id field text embedded inside those already-resolving files — it does not change which URLs exist. Verified locally: python -m cas_evals.registry_check exits 0 with all four paths returning 200 against the live registry today, pre-merge of #18.

Deferred post-merge verification (cannot be completed until cas-contracts#18 merges)

  • Re-run python -m cas_evals.registry_check after #18 merges and confirms continued 200s (expected no-op in observed behavior — paths already resolve, only the $id field text inside those files changes).
  • Fetch https://coding-autopilot-system.github.io/cas-contracts/registry/v0.1/common.schema.json directly and confirm its $id field now reads the new github.io value (proves the merged PR's content actually deployed, not just that the path still 200s).
  • Confirm verify_vendored_contract()'s expected $id and the live-fetched schema's $id agree post-merge (both were updated to the same convention in this milestone, so they should match once #18's content is live).

What changed

  • src/cas_evals/registry_check.py: new module — check_registry_urls(base_url, paths, timeout_seconds=10) and main(argv) CLI entry point (python -m cas_evals.registry_check), following reference_product.py's urllib pattern (capped reads via MAX_RESPONSE_BYTES, typed exception translation to RegistryCheckError).
  • tests/test_registry_check.py: unit tests covering all-200, one-404 (HTTPError), network-unavailable (URLError), and CLI exit-code paths (mocking urlopen).
  • src/cas_evals/contracts.py: verify_vendored_contract()'s two hardcoded $id equality checks updated to the new Pages registry URL.
  • vendor/cas-contracts/v0.1.0/{common,evaluation-result}.schema.json: $id field rewritten to the new convention; this changes the file bytes and thus the sha256.
  • vendor/cas-contracts/v0.1.0/provenance.json: sha256 fields recomputed to match the re-vendored files.
  • releases/v0.2.0/manifest.json: provenanceDigest regenerated — it is a sha256 of provenance.json, which changed as a direct, mechanical consequence of the re-vendor above (not new scope; downstream digest consistency required by the existing release-manifest invariant).
  • .github/workflows/ci.yml: new registry-smoke job, isolated from the offline verify matrix (separate job, not a blocking step) so a transient network failure doesn't block unrelated merges. SHA-pinned actions, permissions: contents: read, timeout-minutes: 5, matching the repo's existing CI conventions.

Test plan

  • python -m unittest discover -s tests -v — 37/37 pass, including the 6 new test_registry_check tests and the existing test_contracts.py suite (now expecting the new $id).
  • python -m cas_evals.registry_check — exits 0, all four paths return 200 against the live Pages registry today.
  • Deferred post-merge checks listed above.

Not merging this PR per environment constraint — PR-only, leaving open for review.

Aitomates added 3 commits July 6, 2026 20:13
- RED phase: test_registry_check.py covers all-200 success, one-404
  failure, network-unavailable distinction, HTTP 5xx errors, and CLI
  exit-code paths for the not-yet-implemented cas_evals.registry_check
- REQ-1.4.11
- GREEN phase: check_registry_urls() performs live GETs against the Pages
  registry, raising RegistryCheckError naming the first failing path on
  non-200, HTTPError, or network-unavailable conditions
- main() is the python -m cas_evals.registry_check CLI entry point,
  checking index.json, v0.1/manifest.json, v0.1/common.schema.json, and
  v0.1/evaluation-result.schema.json by default; exits non-zero on failure
- Follows reference_product.py's urllib pattern: capped response reads
  (MAX_RESPONSE_BYTES), typed exception-to-domain-error translation
- REQ-1.4.11
…$id to resolvable Pages URL

- Update vendored v0.1.0 common/evaluation-result schema $id values from
  the dead schemas.coding-autopilot.dev namespace to the resolvable Pages
  registry URL, matching the portfolio-wide convention from cas-contracts
  plan 32-01
- Recompute sha256 in provenance.json for the re-vendored files
- Update verify_vendored_contract()'s two hardcoded $id equality checks
  to match the new value
- Wire a new registry-smoke CI job invoking cas_evals.registry_check
  against the live Pages registry, isolated from the offline verify matrix
  so a transient network failure doesn't block unrelated merges
- Regenerate releases/v0.2.0/manifest.json: its provenanceDigest is a
  sha256 of provenance.json, which changed as a direct consequence of the
  re-vendor above (Rule 3 auto-fix: this is a blocking downstream effect
  of the provenance.json edit required by this task, not new scope)
- REQ-1.4.11
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@OgeonX-Ai OgeonX-Ai enabled auto-merge (squash) July 8, 2026 17:02
@OgeonX-Ai OgeonX-Ai merged commit 4652531 into main Jul 8, 2026
13 checks passed
@OgeonX-Ai OgeonX-Ai deleted the feat/registry-fetch-smoke-check branch July 8, 2026 17:51
OgeonX-Ai added a commit that referenced this pull request Jul 8, 2026
* docs(36-02): append freshness footer to README

* docs(36-03): add docs/wiki tree (Home, Architecture, Operations, Decisions)

- Architecture.md documents the evidence gate flow (offline + live
  reference-product modes) and the in-progress registry-fetch smoke
  check (PR #9, companion to cas-contracts PR #18)
- Operations.md verified CLI/gate/CI commands
- Decisions.md indexes phase history and open PRs

---------

Co-authored-by: Kim Harjamäki <kim.harjamaki@prosimo.fi>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants