Skip to content

Optimize test suite and release validation - #5077

Merged
Widthdom merged 22 commits into
mainfrom
codex/test-suite-ci-optimization-20260811
Aug 11, 2026
Merged

Optimize test suite and release validation#5077
Widthdom merged 22 commits into
mainfrom
codex/test-suite-ci-optimization-20260811

Conversation

@Widthdom

Copy link
Copy Markdown
Owner

Summary

  • streamline the release validation matrix so the full .NET 8 suite runs once while publish-only lanes use the production SDK and restore path
  • retain runtime-specific smoke coverage for the produced Linux, Windows, and macOS artifacts
  • move independent database fixtures and parser-only checks out of oversized non-parallel test collections, with explicit console and SQLite isolation where shared state requires it
  • scale TRX telemetry to handle the full suite efficiently and deduplicate CI matrix contract assertions
  • document the resulting parallelism boundaries in the bilingual testing guide and add internal changelog fragments

Why

The suite was paying for redundant release lanes and serializing independent tests behind broad SQLite pool-sensitive classes. The TRX telemetry reader also retained a smaller file-size ceiling and repeatedly sorted its slow-test list. Together these increased CI/release wall-clock time and limited useful full-suite telemetry.

Impact

The full Release/net8 suite improved from 24m 08s to 21m 07s, a reduction of 3m 01s (about 12.5%), while preserving all 11,355 test outcomes and runtime artifact smoke validation.

During full-suite validation, concurrent MCP request diagnostics exposed a stderr ownership gap. The emitting MCP audit integration tests are now explicitly console-sensitive, and nested batch captures retain their own non-parallel boundary.

Validation

  • dotnet build CodeIndex.sln --configuration Debug -p:UseSharedCompilation=false --nologo — 0 warnings, 0 errors
  • full Release/net8 suite — 11,348 passed, 7 skipped, 0 failed (11,355 total) in 21m 07s
  • focused net9 changed/workflow suite — 258 passed, 2 skipped, 0 failed
  • changelog validation — 15 fragments validated
  • git diff --check origin/main...HEAD
  • repository CodeIndex status check — fresh and matched to HEAD
  • separate-agent adversarial review — no blocking/actionable issues found

@Widthdom
Widthdom marked this pull request as ready for review August 11, 2026 16:11
@Widthdom
Widthdom merged commit b8c955f into main Aug 11, 2026
12 checks passed
@Widthdom
Widthdom deleted the codex/test-suite-ci-optimization-20260811 branch August 11, 2026 16:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant