Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
138 changes: 138 additions & 0 deletions .github/workflows/quality-gate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -153,3 +153,141 @@ jobs:
path: evals/reports/
if-no-files-found: ignore
retention-days: 30

# Windows surface: starts the pre-installed PostgreSQL service, provisions
# databases, and runs the full suite + Tier P evals. Proves the pipeline
# works on Windows and closes GAP_ANALYSIS.md G2.
windows-postgres:
name: windows (postgres, full suite)
runs-on: windows-latest
env:
PGHOST: localhost
PGPORT: "5432"
PGUSER: postgres
PGPASSWORD: postgres
PGDATABASE: postgres
steps:
- name: Checkout
uses: actions/checkout@v7

- name: Setup Python
uses: actions/setup-python@v7
with:
python-version: '3.11'

- name: Install dev dependencies
run: pip install -r requirements-dev.txt -r api/requirements.txt

- name: Start PostgreSQL and verify connection
shell: pwsh
run: |
# Locate the PostgreSQL bin directory
$pgDir = (Get-ItemProperty 'HKLM:\SOFTWARE\PostgreSQL\Installations\*' -ErrorAction SilentlyContinue |
Select-Object -First 1).Base_Directory
if (-not $pgDir) {
$pgDir = (Get-ChildItem "C:\Program Files\PostgreSQL\*\bin\psql.exe" -ErrorAction SilentlyContinue |
Select-Object -First 1).Directory.Parent.FullName
}
if (-not $pgDir) {
Write-Error "No PostgreSQL installation found on this runner"
exit 1
}
$binDir = Join-Path $pgDir "bin"
Write-Host "PostgreSQL bin: $binDir"
echo "$binDir" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append

# Use a writable temp directory for the data cluster — Program Files
# is not writable by the runner user.
$dataDir = Join-Path $env:RUNNER_TEMP "pgdata"
Write-Host "Data directory: $dataDir"

# Initialise a fresh cluster owned by the current user
if (-not (Test-Path (Join-Path $dataDir "PG_VERSION"))) {
Write-Host "Running initdb"
& "$binDir\initdb" -U postgres -D $dataDir --encoding=UTF8 --auth=trust
if ($LASTEXITCODE -ne 0) {
Write-Error "initdb failed"; exit 1
}
}

# Start the server directly with pg_ctl (no Windows service needed)
$logFile = Join-Path $env:RUNNER_TEMP "pg.log"
& "$binDir\pg_ctl" -D $dataDir -l $logFile -o "-p 5432" start
Start-Sleep -Seconds 5

# Verify it is running
& "$binDir\pg_isready" -p 5432
if ($LASTEXITCODE -ne 0) {
Get-Content $logFile -Tail 30
Write-Error "PostgreSQL did not start"; exit 1
}

# Set password and switch to md5 auth
& "$binDir\psql" -U postgres -p 5432 -c "ALTER USER postgres PASSWORD 'postgres';"
$hbaPath = Join-Path $dataDir "pg_hba.conf"
(Get-Content $hbaPath) -replace 'trust$','md5' | Set-Content $hbaPath
& "$binDir\pg_ctl" -D $dataDir reload
Start-Sleep -Seconds 2

& "$binDir\psql" -U postgres -p 5432 -c "SELECT version();"

- name: Write CI database config
shell: pwsh
run: |
@"
DB_ENGINE="postgresql"
PG_HOST="localhost"
PG_PORT="5432"
PG_SUPERUSER="postgres"
PG_SUPERUSER_PASSWORD="postgres"
PG_DB_DEV="te_mgmt_dev"
PG_SCHEMA_DEV="te_dev"
PG_DB_TEST="te_mgmt_test"
PG_SCHEMA_TEST="te_test"
PG_DB_STAGING="te_mgmt_staging"
PG_SCHEMA_STAGING="te_staging"
PG_DB_PROD="te_mgmt_prod"
PG_SCHEMA_PROD="te_prod"
"@ | Out-File -FilePath "build/config.local.env" -Encoding utf8

- name: Materialise environment launchers from templates
shell: pwsh
run: |
foreach ($env_name in @("dev", "test", "staging", "prod")) {
Copy-Item "build/environments/env_${env_name}.example.sql" `
"build/environments/env_${env_name}.sql"
}
Get-ChildItem build/environments/

- name: Create environment databases
shell: pwsh
run: |
foreach ($db in @("te_mgmt_dev", "te_mgmt_test", "te_mgmt_staging", "te_mgmt_prod")) {
$exists = & psql -U postgres -d postgres -tA -c "SELECT 1 FROM pg_database WHERE datname = '$db'"
if (-not $exists) {
& psql -U postgres -d postgres -v ON_ERROR_STOP=1 -c `
"CREATE DATABASE `"$db`" WITH OWNER = postgres ENCODING = 'UTF8' TEMPLATE = template0 CONNECTION LIMIT = -1"
}
}

- name: Evals — Tier P (offline validator scenarios)
run: python3 evals/runner.py --tiers p --verbose

- name: Deploy all environments (for cross-env parity)
shell: bash
run: |
for env in dev test staging prod; do
bash build/deploy_all.sh "$env"
done

- name: Full test suite — final result with skip accounting
run: python3 scripts/test_report.py --strict

- name: Upload eval reports
if: always()
uses: actions/upload-artifact@v7
with:
name: eval-reports-windows
path: evals/reports/
if-no-files-found: ignore
retention-days: 30
23 changes: 9 additions & 14 deletions GAP_ANALYSIS.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ claim below was reproduced, not inferred from reading code.
| ID | Gap | Severity | Decision needed |
|---|---|---|---|
| G1 | ~~`config.env.example` names do not match `setup.sh` / loaders~~ | **Closed** | Renamed to `PG_*_<ENV>` scheme |
| G2 | Windows CI cannot run database-backed tests | Medium | Yes — accept scope, or start PG on the runner |
| G2 | ~~Windows CI cannot run database-backed tests~~ | **Closed** | Added `windows-postgres` job to `quality-gate.yml` |
| G3 | Tiers X and E remain unimplemented | Medium | No — deferred by design |
| G4 | ~~Runtime artifacts are not gitignored~~ | **Closed** | Added to `.gitignore` |
| G5 | ~~`VCRM.md` BR-20 assertion count edited~~ | **Closed** | Confirmed: 142 matches suite output and Tier S JSON |
Expand All @@ -37,21 +37,16 @@ Copying the example directly to `config.local.env` now produces a working
configuration. The `provision_full_test_env.sh` workaround is still valid but
no longer required for basic operation.

### G2 — Windows CI cannot host PostgreSQL (Medium)
### G2 — Windows CI cannot host PostgreSQL (Closed)

GitHub Actions service containers are Linux-only, so
`python-validator-tests.yml` (windows-latest) cannot run the `integration`,
`e2e` or `parity` markers. With missing prerequisites now fatal, collecting them
there would make the job permanently red.
**Resolution:** Added a `windows-postgres` job to `quality-gate.yml` that starts
the pre-installed PostgreSQL service on the `windows-latest` runner, provisions
all four environment databases, deploys schemas, and runs the full test suite
(including `integration`, `e2e`, and `parity` markers) plus Tier P evals.

**Current state:** the Windows job runs the database-free markers and prints the
15 tests it does not run **by name**, so the gap is visible rather than implied.
Those tests run in the Linux `integration-postgres` job. Every test reports
pass/fail in exactly one job.

**Option:** start the PostgreSQL service on the Windows runner (the GitHub
Windows image ships it, stopped) and provision there too. Not verified — no
Windows runner was available during this audit.
The existing `python-validator-tests.yml` Windows job continues to run
database-free markers as a fast signal; the new quality-gate job covers the
full surface.

### G3 — Tiers X and E unimplemented (Medium)

Expand Down