Skip to content

ci: restore basic and migration checks#92

Merged
mrjf merged 4 commits into
mainfrom
codex/restore-basic-migration-ci
May 28, 2026
Merged

ci: restore basic and migration checks#92
mrjf merged 4 commits into
mainfrom
codex/restore-basic-migration-ci

Conversation

@mrjf
Copy link
Copy Markdown
Contributor

@mrjf mrjf commented May 28, 2026

TL;DR

Restores an active PR CI surface and adds a migration-specific parity and benchmark workflow. This gives PRs like #91 actual checks plus a manual workflow_dispatch path when maintainers need a "run CI" button.

Problem

  • Active workflows were too narrow: normal source changes on the Crane migration PR did not produce checks.
  • The Python-to-Go migration needs non-vacuous Python-vs-Go parity evidence and benchmark evidence before it can be treated as complete.
  • Reviewers need artifacts that show the migration score, Go test events, and Python-vs-Go latency instead of relying only on Crane comments.

Approach

Area Change
Basic PR CI Add active CI workflow for Ruff, Python unit tests, and Go tests.
Migration gate Add active Migration Parity and Benchmarks workflow for Go parity score and benchmark evidence.
Benchmark helper Add scripts/ci/migration_cli_benchmark.py to compare Python and Go CLI latency on smoke commands.
Lint bootstrap Apply the formatter-only fix needed for the restored Ruff format gate to pass on current main.

Implementation

  • .github/workflows/ci.yml runs on pull_request, merge_group, and workflow_dispatch. It restores lint, Python unit tests, and Go tests with APM_PYTHON_BIN set to the checked-out Python CLI.
  • .github/workflows/migration-ci.yml runs on migration-relevant paths and manual dispatch. A lightweight detection job skips the heavy parity/benchmark jobs when a PR only changes workflow wiring. When migration paths change, it installs the Python reference CLI, runs go test ./..., computes .crane/scripts/score.go, uploads parity evidence, runs benchmark tests, and uploads benchmark evidence.
  • scripts/ci/migration_cli_benchmark.py benchmarks Python and Go CLI smoke commands, compares return codes, enforces a configurable Go/Python median latency ratio, and emits JSON plus Markdown summaries.
  • tests/unit/test_crane_workflow_prompt.py gets a formatter-only trailing blank-line cleanup so restored ruff format --check starts green.

Diagram

The migration workflow now produces both correctness evidence and benchmark evidence from the same PR head.

flowchart LR
    A["Migration PR"] --> B["Python-vs-Go Parity Gate"]
    B --> C["Migration Score Artifact"]
    B --> D["Migration Benchmarks"]
    D --> E["Benchmark Artifact"]
Loading

Trade-offs

  • The migration parity workflow requires migration_score == 1.0; this is intentionally strict for completion-oriented migration PRs.
  • Benchmarks use a smoke-command latency ratio instead of a full performance lab. That keeps PR feedback practical while still catching obvious regressions.
  • The broad CI workflow is intentionally basic rather than resurrecting every disabled workflow at once.

Benefits

  1. Normal PRs get visible CI checks again.
  2. Migration PRs get a hard Python-vs-Go parity gate.
  3. Benchmark evidence is uploaded as artifacts and summarized in the job output.
  4. Maintainers get manual workflow_dispatch entry points for CI and migration checks.

Validation

ruby YAML parse: ok .github/workflows/ci.yml
ruby YAML parse: ok .github/workflows/migration-ci.yml
.venv/bin/ruff check scripts/ci/migration_cli_benchmark.py: All checks passed
.venv/bin/ruff format --check scripts/ci/migration_cli_benchmark.py: already formatted
.venv/bin/ruff check tests/unit/test_crane_workflow_prompt.py: All checks passed
.venv/bin/ruff format --check tests/unit/test_crane_workflow_prompt.py: already formatted
.venv/bin/python -m py_compile scripts/ci/migration_cli_benchmark.py: passed
git diff --check: passed

Go tests were not run locally because the local workspace does not have a Go toolchain installed.

How to test

  1. Open a PR that changes Python source: confirm the CI workflow runs.
  2. Open or update a PR that changes cmd/**, internal/**, go.mod, or .crane/**: confirm Migration Parity and Benchmarks runs.
  3. Use the Actions tab to manually run either workflow via workflow_dispatch.
  4. Inspect uploaded parity and benchmark artifacts on the migration workflow run.

@mrjf mrjf force-pushed the codex/restore-basic-migration-ci branch from 0cbab3e to cfe9db1 Compare May 28, 2026 03:43
@mrjf mrjf merged commit 23f7da4 into main May 28, 2026
13 checks passed
@mrjf mrjf deleted the codex/restore-basic-migration-ci branch May 28, 2026 04:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant