Skip to content

Decompose large indexing and query orchestration - #5078

Merged
Widthdom merged 45 commits into
mainfrom
codex/split-large-types-20260812
Aug 13, 2026
Merged

Decompose large indexing and query orchestration#5078
Widthdom merged 45 commits into
mainfrom
codex/split-large-types-20260812

Conversation

@Widthdom

Copy link
Copy Markdown
Owner

Summary

  • Decompose large symbol extraction, reference extraction, database query, MCP, and CLI implementations into cohesive partials and helpers.
  • Consolidate mutable ownership in update and full-scan orchestration, removing duplicate context/result state and callback handoffs.
  • Preserve behavior with characterization and parity coverage while fixing four correctness defects found during the decomposition.
  • Reduce hot-path allocations in pattern extraction and JVM triple-string masking.

Why

Several core paths had grown into very large classes and methods with duplicated mutable state, callback-heavy context objects, and difficult lifetime boundaries. This series makes those boundaries explicit while preserving transaction, cancellation, schema-fallback, and output behavior.

The branch contains 45 focused commits so each extraction or correction remains independently reviewable.

Correctness fixes

  • Use one grammar-aware C# base-list parser so generic constraints and nested type syntax are resolved consistently.
  • Replay accepted but undispatched parallel batch inputs after an output-limit stop instead of losing them.
  • Apply C# partial-family unused-symbol filtering before paging without per-candidate queries, while preserving legacy read-only schema fallbacks.
  • Keep full-scan FTS mutation state monotonic and defer shared worker-resource cleanup until all peer workers stop.

No open issue is auto-closed by this PR.

Validation

  • Release solution build: 0 warnings, 0 errors.
  • Full Release test suite, net8.0: 11,395 passed, 7 skipped, 0 failed.
  • Full Release test suite, net9.0: 10,924 passed, 417 skipped, 0 failed.
  • Full-scan focused suite: net8.0 181 passed / 1 skipped; net9.0 179 passed / 3 skipped.
  • dotnet format CodeIndex.sln whitespace --no-restore --verify-no-changes
  • dotnet run --project tools/CodeIndex.Changelog -- check — 4 fragments validated.
  • git diff --check origin/main..HEAD
  • Dogfood index rebuilt at HEAD with complete index/reference graph and no warnings or errors.
  • Two independent adversarial review lanes reported no blocking or actionable findings.

Documentation and changelog

Behavioral contracts were documented in the English and Japanese sections of TESTING_GUIDE.md.

Changelog fragments:

  • changelog.d/unreleased/+batch-output-limit-replay.fixed.md
  • changelog.d/unreleased/+csharp-base-list-parser.fixed.md
  • changelog.d/unreleased/+full-scan-extraction-state.fixed.md
  • changelog.d/unreleased/+unused-legacy-schema.fixed.md

Follow-up candidates

No correctness follow-up is required. Further decomposition of transaction-heavy full-scan/readiness tails and baseline per-file persistence carriers is intentionally deferred to separately characterized changes.

@Widthdom
Widthdom marked this pull request as ready for review August 13, 2026 11:53
@Widthdom
Widthdom merged commit 5107f26 into main Aug 13, 2026
11 checks passed
@Widthdom
Widthdom deleted the codex/split-large-types-20260812 branch August 13, 2026 11:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant