Optimize initial full indexing for large codebases

[Widthdom]

Summary

Improve the first full index of a large, empty codebase by removing repeated graph, extraction, worker-protocol, fold-readiness, and persistence work while preserving rebuild, update, retained-graph, plugin, hook, cancellation, and recovery contracts.

This PR contains 16 cohesive commits. Candidates that did not meet their measured acceptance thresholds were reverted and are not included.

What changed

• optimize fresh reference-graph recursion finalization, skip unused dirty-scope bookkeeping, resolve only candidate-bearing references, and refresh SQLite planner statistics after bulk load
• reuse loaded C# prepass configuration and checksum-verified, take-once prepass symbol artifacts without retaining source content
• cache bounded pattern discovery in persistent symbol workers
• gate audited built-in symbol regexes at both file and exact transformed-input levels across supported languages
• consume symbol-worker request frames directly from raw UTF-8 while preserving framing, bounds, cancellation, Unicode, and sanitized-error behavior
• avoid row-count-proportional fold revalidation for authoritative fresh built-in indexes, with fail-closed ownership, producer-generation, external-writer, hook/plugin, and cancellation guards
• validate persisted symbol/reference/container kinds through immutable canonical taxonomies
• document the lifecycle and regression contracts in English and Japanese

Why

Fresh indexing previously repeated work that was useful for incremental updates but redundant for a database proven empty. It also rediscovered configuration, re-extracted or cloned C# prepass results, ran impossible regular expressions, decoded worker input twice, and re-folded every stored name during final readiness.

The changes specialize only the authoritative fresh paths and fall back to the existing full validation/resolution behavior whenever ownership or recovery assumptions are not proven.

Impact

Large first-time indexes perform less SQL graph work, fewer pattern scans, fewer configuration discoveries and UTF-8 conversions, and substantially less row-proportional managed allocation. Existing databases, rebuilds, updates, symbols-only runs, retained/scoped graphs, custom extractors, hooks, and public writer calls keep their established contracts.

Validation

• Release solution build: 0 warnings, 0 errors
• .NET 8: 11,348 passed, 7 skipped, 0 failed
• .NET 9: 10,879 passed, 415 skipped, 0 failed
• dotnet format --verify-no-changes: passed
• changelog validation: 13 fragments passed
• git diff --check origin/main..HEAD: passed
• dogfood index: 1,280 files, 48,232 symbols, 485,573 references; all readiness flags current
• SQLite integrity and foreign-key checks: passed
• workspace health: 2/2 members healthy
• independent adversarial review: No blocking/actionable issues found.

Collaborator checklist

• The change follows the existing code style.
• The change includes tests when behavior changes.
• The change includes a changelog fragment when user-visible behavior changes.
• Documentation has been updated when needed.