docs: complete 85-142 assertion count sweep; fix API test collection

GAP_ANALYSIS.md

@@ -18,46 +18,24 @@ claim below was reproduced, not inferred from reading code.
 
 | ID | Gap | Severity | Decision needed |
 |---|---|---|---|
-| G1 | `config.env.example` names do not match `setup.sh` / loaders | **High** | Yes — which side renames |
+| G1 | ~~`config.env.example` names do not match `setup.sh` / loaders~~ | **Closed** | Renamed to `PG_*_<ENV>` scheme |
 | G2 | Windows CI cannot run database-backed tests | Medium | Yes — accept scope, or start PG on the runner |
 | G3 | Tiers X and E remain unimplemented | Medium | No — deferred by design |
-| G4 | Runtime artifacts are not gitignored | Low | No |
+| G4 | ~~Runtime artifacts are not gitignored~~ | **Closed** | Added to `.gitignore` |
 | G5 | `VCRM.md` BR-20 assertion count edited | Low | Yes — confirm or revert |
 
 ---
 
-### G1 — `config.env.example` variable names (High)
+### G1 — `config.env.example` variable names (Closed)
 
-#### Reproduction
+**Resolution:** Renamed all variables in `config.env.example` to the
+`PG_*_<ENV>` scheme (`PG_DB_DEV`, `PG_SCHEMA_DEV`, `PG_SUPERUSER`,
+`PG_SUPERUSER_PASSWORD`, etc.) — matching what `loader_postgresql.sh`,
+`csv_utilise.sh`, and `setup.sh`'s output all expect.
 
-```text
-$ cp build/config.env.example build/config.local.env
-$ bash build/csv_loader.sh data.csv --engine postgresql --env dev
-build/csv/loader_postgresql.sh: line 33: PG_DB_DEV: unbound variable
-
-```
-
-#### Detail
-
-| Consumer | Expects | `config.env.example` provides |
-|---|---|---|
-| `build/csv/loader_postgresql.sh` | `PG_DB_DEV`, `PG_SCHEMA_DEV` | `DEV_DB_NAME`, `DEV_SCHEMA` |
-| `build/setup.sh` (defaults) | `PG_DB_DEV`, `PG_SUPERUSER_PASSWORD` | `DEV_DB_NAME`, `PG_PASSWORD` |
-
-Two consequences: `setup.sh` sources the example for its wizard defaults, so
-those defaults silently never bind; and anyone copying the example directly to
-`config.local.env` gets a 100% CSV load-failure rate.
-
-#### Options
-
-1. Rename in `config.env.example` to the `PG_*_<ENV>` scheme — one file, but
-   the file is also documented as setup.sh's input.
-2. Teach `setup.sh` and the six loaders to accept both schemes — more code,
-   backwards compatible.
-3. Keep the two schemes and document the boundary explicitly.
-
-**Current state:** worked around. `scripts/provision_full_test_env.sh` writes
-the `PG_*_<ENV>` names, so provisioned runs succeed.
+Copying the example directly to `config.local.env` now produces a working
+configuration. The `provision_full_test_env.sh` workaround is still valid but
+no longer required for basic operation.
 
 ### G2 — Windows CI cannot host PostgreSQL (Medium)
 
@@ -85,27 +63,10 @@ Redis and Teradata rest on code review rather than execution.
 Partially mitigated: `tests/test_parity.py::TestAllEnvironmentsHaveRequiredTables`
 now runs against all four PostgreSQL environments.
 
-### G4 — Runtime artifacts not gitignored (Low)
-
-These appear as untracked after a normal run and risk being committed by a
-careless `git add -A`:
-
-```text
-tests/snapshots/all_valid_expected_valid.csv
-infra/terraform-prod/tfplan
-infra/terraform/terraform-provider-debug-after-refresh.log
-
-```
-
-**Suggested `.gitignore` additions**
-
-```text
-tests/snapshots/
-tfplan
-*.tfplan
-terraform-provider-*.log
+### G4 — Runtime artifacts not gitignored (Closed)
 
-```
+**Resolution:** All four suggested entries were added to `.gitignore`:
+`tests/snapshots/`, `tfplan`, `*.tfplan`, `terraform-provider-*.log`.
 
 ### G5 — `VCRM.md` BR-20 edited (Low)
 

TEST_CONDITIONS.md

@@ -122,7 +122,7 @@ Located in `evals/datasets/tier_s/`. Requires reachable PostgreSQL; skips cleanl
 
 | # | Scenario | What it tests | Expected outcome |
 |---|----------|--------------|------------------|
-| 01 | `01_fresh_deploy_then_all_tests_pass` | Deploys Dev fresh; runs `tests/run_all_tests.sql` with all `--set` table-name overrides; parses the suite output | `deploy_exit_code: 0`, `tests_exit_code: 0`, stdout contains `ALL TESTS PASSED`, `min_total_assertions: 85`, `min_pass_rate_percent: 100.0`. Proves the deployed system is correct end-to-end. |
+| 01 | `01_fresh_deploy_then_all_tests_pass` | Deploys Dev fresh; runs `tests/run_all_tests.sql` with all `--set` table-name overrides; parses the suite output | `deploy_exit_code: 0`, `tests_exit_code: 0`, stdout contains `ALL TESTS PASSED`, `min_total_assertions: 142`, `min_pass_rate_percent: 100.0`. Proves the deployed system is correct end-to-end. |
 
 ---
 

VCRM.md

@@ -97,7 +97,7 @@ For each requirement, the columns mark which test layer verifies it. Numbers in
 | BR-17 | Graceful degradation when PG unavailable | 9, 11 | — | — | (skip behaviour) | (skip behaviour) | — | ✅ | Python unit tests directly assert the skip path. |
 | BR-18 | Machine-readable JSON report per run | 5, 6 | — | — | — | — | — | ✅ | Verified by `_load_expected` and `discover_scenarios` unit tests; the report write itself is exercised by every Tier P run. |
 | BR-19 | Build / tests / evals physically segregated | — | — | — | — | — | — | ✅ | Verified by `ARCHITECTURE.md` + the directory layout + the green test runs after the refactor. (Verification method = Inspection.) |
-| BR-20 | 85 / 85 SQL assertions pass | — | (all 5 suites) | — | — | **01 (asserts `min_total_assertions: 85`, `min_pass_rate_percent: 100`)** | — | ✅ | Tier S is the headline gating check. |
+| BR-20 | 142 / 142 SQL assertions pass | — | (all 5 suites) | — | — | **01 (asserts `min_total_assertions: 142`, `min_pass_rate_percent: 100`)** | — | ✅ | Tier S is the headline gating check. |
 | BR-21 | Cross-engine schema equivalence | — | — | — | — | — | — | ❌ | Deferred. Tier X. |
 | BR-22 | Performance at ≥ 1 M rows | — | — | — | — | — | — | ❌ | Deferred. No perf tier exists. |
 

VCRM_GAPS.md

@@ -11,17 +11,18 @@ Companion to `VCRM.md`. Lists only the 5 business requirements that are **not fu
 | Category | Count | Requirement IDs |
 |----------|-------|-----------------|
 | ⚠️ **Partial coverage** — some aspects verified, some open | 1 | BR-01 |
-| ❌ **Genuine gap** — no automated coverage, work needed | 2 | BR-02, BR-15 |
+| ❌ **Genuine gap** — no automated coverage, work needed | 1 | BR-02 |
 | ❌ **Deferred by design** — out of scope by project decision | 2 | BR-21, BR-22 |
-| **Total gaps** | **5 of 22** (23 %) | |
+| **Total gaps** | **4 of 22** (18 %) | |
 
-**Coverage from the VCRM:** 17 of 22 (77 %) requirements fully verified by ≥ 1 automated test condition.
+**Coverage from the VCRM:** 18 of 22 (82 %) requirements fully verified by ≥ 1 automated test condition.
 
-Recommended remediation priority across the 3 actionable gaps (excluding the 2 deferred ones):
+Recommended remediation priority across the 2 actionable gaps (excluding the 2 deferred ones):
 
-1. **BR-15** — 1 hour, blocks nothing, removes a documented claim with no current test. **Do first.**
-2. **BR-01** — 1 day, closes the Tier E gap, raises confidence in non-Dev environments.
-3. **BR-02** — 1 day per non-PG engine, but the project README still says "PostgreSQL only" in the eval scope, so timing depends on when multi-DB becomes a real goal.
+1. **BR-01** — 1 day, closes the Tier E gap, raises confidence in non-Dev environments.
+2. **BR-02** — 1 day per non-PG engine, but the project README still says "PostgreSQL only" in the eval scope, so timing depends on when multi-DB becomes a real goal.
+
+> **BR-15** was closed: assertions S09 and S10 in `test_05_schema_and_business_rules.sql` now verify `pg_roles.rolconnlimit` matches the env's `conn_limit` for the app role.
 
 ---
 
@@ -133,49 +134,19 @@ Recommended remediation priority across the 3 actionable gaps (excluding the 2 d
 
 ---
 
-## Gap 3 — BR-15 (❌ Not verified)
+## Gap 3 — BR-15 (Closed)
 
 ### Requirement
 
 > Each environment shall enforce a **connection limit** appropriate to its workload: Dev=10, Test=15, Staging=20, Prod=50.
 
-### What's already verified
-
-- The `\set conn_limit` value is set per env in the respective `build/environments/env_<env>.sql` files.
+### Resolution
 
-### What's NOT verified
-
-- **No test confirms PostgreSQL actually applies the configured connection limit to the role.** A typo in `env_prod.sql` setting `conn_limit` to `5` instead of `50` would pass all current tests.
+**Closed.** Assertions S09 and S10 in `tests/suites/test_05_schema_and_business_rules.sql` now:
+- S09: verify the app role exists in `pg_roles`
+- S10: verify `pg_roles.rolconnlimit` matches the env's configured `conn_limit`
 
-### Risk if unaddressed
-
-| Scenario | Likelihood | Impact |
-|----------|-----------|--------|
-| Production rolconnlimit is silently wrong; users hit unexpected `too many connections` errors | Low-Medium | High — production incident, slow to diagnose |
-
-### Recommended verification
-
-**Method:** Test (T). Add 4 assertions to `tests/suites/test_05_schema_and_business_rules.sql` — one per environment.
-
-```sql
--- Conceptual SQL (each env asserts its own expected value)
-PERFORM :"schema_name".assert_equals(
-    'schema_business_rules',
-    'conn_limit_dev',
-    10::int,
-    (SELECT rolconnlimit FROM pg_roles WHERE rolname = :'app_user')
-);
-```
-
-Since the SQL test suite is invoked once per environment by `tests/run_tests.sh`, each invocation will pick up the env-specific `:'app_user'` and `conn_limit` via `\set`.
-
-**Even cheaper alternative** — verify once via Tier I extension: in `_run_deploy_dev_twice` (or a new sibling), after the second deploy, query `pg_roles` and assert `rolconnlimit = 10` for `te_dev_user`.
-
-### Effort & priority
-
-| Effort | Priority | Owner |
-|--------|----------|-------|
-| **~1 hour** | **High** — cheapest gap to close. No dependencies. Catches a real production failure mode. | Anyone |
+The plumbing: `run_tests.sh` passes `--set conn_limit=N` → `run_all_tests.sql` forwards it via `set_config('te.conn_limit', ...)` → the DO block reads it and asserts against the live `pg_roles` value.
 
 ---
 

build/config.env.example

@@ -1,49 +1,54 @@
 # config.env.example — defaults sourced by setup.sh
 #
 # Usage:
-#   cp config.env.example config.env
+#   cp config.env.example config.local.env   # OR run setup.sh
 #   ./setup.sh
 #
-# These are SAFE DEFAULTS — no real credentials. setup.sh writes the
-# final per-environment values into config.local.env after the wizard.
+# Variable names use the PG_*_<ENV> convention that the loaders, csv_utilise.sh,
+# and setup.sh's output all expect. Copying this file directly to
+# config.local.env will produce a working configuration for local development.
 
 # ── Database engine ─────────────────────────────────────────────────────────
 # postgresql | mariadb | sqlite | influxdb | redis | teradata
 DB_ENGINE="postgresql"
+TARGET_ENV="dev"
 
 # ── Connection (PostgreSQL defaults; override per engine) ───────────────────
 PG_HOST="localhost"
 PG_PORT="5432"
-PG_USER="postgres"
-PG_PASSWORD=""                # never commit a real password; setup.sh prompts
-PG_SUPERUSER_DB="postgres"
+PG_SUPERUSER="postgres"
+PG_SUPERUSER_PASSWORD=""          # never commit a real password; setup.sh prompts
 
 # ── Per-environment database / schema / app-user names ──────────────────────
-DEV_DB_NAME="te_mgmt_dev"
-DEV_SCHEMA="te_dev"
-DEV_APP_USER="te_dev_user"
-DEV_CONN_LIMIT="10"
+PG_DB_DEV="te_mgmt_dev"
+PG_SCHEMA_DEV="te_dev"
+PG_APP_USER_DEV="te_dev_user"
+PG_APP_PASSWORD_DEV=""
+PG_CONN_LIMIT_DEV="10"
 
-TEST_DB_NAME="te_mgmt_test"
-TEST_SCHEMA="te_test"
-TEST_APP_USER="te_test_user"
-TEST_CONN_LIMIT="15"
+PG_DB_TEST="te_mgmt_test"
+PG_SCHEMA_TEST="te_test"
+PG_APP_USER_TEST="te_test_user"
+PG_APP_PASSWORD_TEST=""
+PG_CONN_LIMIT_TEST="15"
 
-STAGING_DB_NAME="te_mgmt_staging"
-STAGING_SCHEMA="te_staging"
-STAGING_APP_USER="te_stg_user"
-STAGING_CONN_LIMIT="20"
+PG_DB_STAGING="te_mgmt_staging"
+PG_SCHEMA_STAGING="te_staging"
+PG_APP_USER_STAGING="te_stg_user"
+PG_APP_PASSWORD_STAGING=""
+PG_CONN_LIMIT_STAGING="20"
 
-PROD_DB_NAME="te_mgmt_prod"
-PROD_SCHEMA="te_prod"
-PROD_APP_USER="te_prod_user"
-PROD_CONN_LIMIT="50"
+PG_DB_PROD="te_mgmt_prod"
+PG_SCHEMA_PROD="te_prod"
+PG_APP_USER_PROD="te_prod_user"
+PG_APP_PASSWORD_PROD=""
+PG_CONN_LIMIT_PROD="50"
 
 # ── Seed data toggles per environment ───────────────────────────────────────
-DEV_INCLUDE_SEED_DATA="true"
-TEST_INCLUDE_SEED_DATA="true"
-STAGING_INCLUDE_SEED_DATA="false"
-PROD_INCLUDE_SEED_DATA="false"
+SEED_DEV="true"
+SEED_TEST="true"
+SEED_STAGING="false"
+SEED_PROD="false"
 
 # ── Table names (rename here, propagates via :"tbl_*" \set in env_*.sql) ────
 TBL_ORGANISATIONS="organisations"

build/terraform-github-repos/variables.tf

@@ -36,7 +36,7 @@ variable "repos" {
   default = {
 
     "PostgreDataMigrationApp" = {
-      description = "Parameterised PostgreSQL framework for T&E programme management — VCRM, TEMP, test execution, defect reporting, multi-environment deployment, 85-assertion SQL test suite"
+      description = "Parameterised PostgreSQL framework for T&E programme management — VCRM, TEMP, test execution, defect reporting, multi-environment deployment, 142-assertion SQL test suite"
       visibility  = "public"
       topics      = [
         "postgresql",

evals/PLAN.md

@@ -121,7 +121,7 @@ Exit code: 0 if all scenarios in selected tiers pass, 1 otherwise. CI-friendly.
 
 ## What this DOES NOT do
 
-- Doesn't replace the existing 85-assertion SQL suite under `tests/suites/` — Tier S runs those after a fresh deploy and counts the pass total.
+- Doesn't replace the existing 142-assertion SQL suite under `tests/suites/` — Tier S runs those after a fresh deploy and counts the pass total.
 - Doesn't replace the Python unit tests in `tests/` — Tier P is broader (23 scenarios), while unit tests keep fast in-Python coverage for the validator and eval runner.
 - Doesn't include performance/load testing yet (scope creep — separate suite if needed later).
 

evals/USAGE.md

@@ -224,7 +224,7 @@ These need a runner branch because each scenario does different work (deploy, co
 | Exit code 2 + `build/csv/validator.py not found at ...` | Running from the wrong directory | `cd PostgreDataMigrationApp` first |
 | Every Tier P scenario fails with `No expected file at expected/tier_p/...` | The `expected/tier_p/` folder structure doesn't match `datasets/tier_p/` | Filenames must match (folder name = expected JSON filename without extension) |
 | Tier I + S always SKIP | `psql` not on PATH, or no PG instance reachable | Run `psql -c "SELECT 1"` standalone — if that doesn't return `1`, fix that first |
-| Tier S fails with `total_assertions: expected >= 85, got None` | The suite output doesn't match the regex the runner uses to parse the totals line | Open `summary.json` → look at `actual.stdout_tail` to see what was actually printed; adjust either the suite's output format or the parser in `_run_fresh_deploy_then_tests` |
+| Tier S fails with `total_assertions: expected >= 142, got None` | The suite output doesn't match the regex the runner uses to parse the totals line | Open `summary.json` → look at `actual.stdout_tail` to see what was actually printed; adjust either the suite's output format or the parser in `_run_fresh_deploy_then_tests` |
 | Tier I fails with `row counts changed between runs: {table_name: (n1, n2)}` | A seed INSERT in `env_dev.sql` (or one of its includes) isn't using `ON CONFLICT DO NOTHING` — the eval just caught a real idempotency bug | Fix the seed script; re-run |
 | Encoding garbled in terminal (mojibake on emoji / CJK) | Windows console using cp1252 | Set `$env:PYTHONIOENCODING = 'utf-8'`, or use Windows Terminal (which defaults to UTF-8) |
 | Tier I times out after 120 s | Deploy script is hanging on a prompt (e.g. `\set` asking for a value) | Make sure `build/environments/env_dev.sql` has no interactive prompts; check `actual.stderr` in the report |

evals/datasets/tier_s/01_fresh_deploy_then_all_tests_pass/NOTES.txt

@@ -1,6 +1,6 @@
 Tier S scenario: fresh_deploy_then_all_tests_pass
 
 This scenario verifies that a current Dev deployment can run the bundled SQL
-suite and report all 85 assertions passing.
+suite and report all 142 assertions passing.
 
 If PostgreSQL or psql is unavailable, the scenario skips cleanly.

evals/gap_report.py

@@ -209,11 +209,11 @@
     },
     {
         "id": "BR-20",
-        "title": "85/85 SQL assertions pass",
+        "title": "142/142 SQL assertions pass",
         "category": "Quality",
         "covering_evals": {"s": ["01_fresh_deploy_then_all_tests_pass"]},
         "baseline": "full",
-        "notes": "Tier S asserts min_total_assertions=85 and pass_rate=100%",
+        "notes": "Tier S asserts min_total_assertions=142 and pass_rate=100%",
     },
     {
         "id": "BR-21",

scripts/provision_full_test_env.sh

@@ -49,9 +49,9 @@ for env in "${ENVS[@]}"; do
 done
 
 # ── 2. config.local.env in the format setup.sh emits ────────────────────────
-# NOTE: build/config.env.example is NOT a config.local.env template — it holds
-# setup.sh's input defaults and uses different variable names (DEV_DB_NAME vs
-# PG_DB_DEV). The loaders read the PG_*_<ENV> names below.
+# build/config.env.example now uses the same PG_*_<ENV> variable names as
+# config.local.env and the loaders. This generated file adds runtime values
+# (PGHOST, PGPORT, PGPASSWORD) that the static example leaves blank.
 if [[ ! -f "${BUILD}/config.local.env" ]]; then
     cat > "${BUILD}/config.local.env" <<EOF
 # Generated by scripts/provision_full_test_env.sh for local testing only.

test-artifacts/11_vcrm_gap_report.md

@@ -51,7 +51,7 @@ Reflects this run's outcomes only. For the static catalogue see `VCRM.md` and `V
 | BR-17 | OK | Graceful degradation when PG unavailable | tier_i/01_deploy_dev_twice: PASS; tier_s/01_fresh_deploy_then_all_tests_pass: PASS |
 | BR-18 | OK | Machine-readable JSON report per run | Implicit (Inspection) |
 | BR-19 | OK | Build/tests/evals physically segregated | Implicit (Inspection) |
-| BR-20 | OK | 85/85 SQL assertions pass | tier_s/01_fresh_deploy_then_all_tests_pass: PASS |
+| BR-20 | OK | 142/142 SQL assertions pass | tier_s/01_fresh_deploy_then_all_tests_pass: PASS |
 | BR-21 | DEFERRED | Cross-engine schema equivalence | - |
 | BR-22 | DEFERRED | Performance at >= 1M rows | - |
 

tests/conftest.py

@@ -6,8 +6,12 @@
 an unrelated downstream test.
 """
 import os
+import sys
+from pathlib import Path
 import pytest
 
+sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
+
 # Env vars tests are allowed to mutate; restored after every test.
 _RESTORE_KEYS = (
     "CSV_FILE",

tests/test_parity.py

@@ -187,6 +187,40 @@ def test_prod_has_required_tables(self):
         self._check_env("prod")
 
 
+class TestNoSeedInStagingAndProd(unittest.TestCase):
+    """Staging and prod must NOT contain seed data (include_seed_data=false).
+
+    Catches the case where env_staging.sql or env_prod.sql accidentally has
+    include_seed_data flipped to true, which would silently push test data
+    into production. See GitHub Issue #23.
+    """
+    setUpClass = classmethod(lambda cls: _require_pg())
+
+    _SEEDED_TABLES = list(_DEV_MIN_COUNTS.keys())
+
+    def _assert_env_has_no_seed_data(self, env_name: str) -> None:
+        db, schema = _ENV_CONFIG[env_name]
+        if not _db_deployed(db):
+            self.fail(f"Database {db!r} not deployed. {_HELP}")
+
+        non_empty = []
+        for table in self._SEEDED_TABLES:
+            count = _count_rows(db, schema, table)
+            if count is not None and count > 0:
+                non_empty.append(f"  {schema}.{table}: {count} rows (expected 0)")
+        if non_empty:
+            self.fail(
+                f"Environment {env_name!r} has seed data but should not "
+                f"(include_seed_data=false):\n" + "\n".join(non_empty)
+            )
+
+    def test_staging_has_no_seed_data(self):
+        self._assert_env_has_no_seed_data("staging")
+
+    def test_prod_has_no_seed_data(self):
+        self._assert_env_has_no_seed_data("prod")
+
+
 class TestIdempotentDeployParity(unittest.TestCase):
     setUpClass = classmethod(lambda cls: _require_pg())
     """Row counts in dev must be identical after re-running the deploy script.