Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 83 additions & 7 deletions DEVELOPER_GUIDE.md

Large diffs are not rendered by default.

30 changes: 20 additions & 10 deletions TESTING_GUIDE.md

Large diffs are not rendered by default.

14 changes: 14 additions & 0 deletions changelog.d/unreleased/+compact-lower-rank-candidates.changed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
category: changed
affected:
- src/CodeIndex/Database/DbWriter.References.cs
- src/CodeIndex/Database/DbWriter.ReferenceGraphRefreshScope.cs
---

## English

- **Initial reference-graph finalization now uses a compact lower-rank match set** — all rank-5 fallbacks probe one TEMP row per reference that matched ranks 0–4 instead of repeatedly searching the much larger physical candidate table, while full, scoped, and retained refreshes preserve candidate and ambiguity results across languages.

## 日本語

- **初回reference graph確定がcompactな下位rank一致集合を使うようになりました** — 全rank 5 fallbackは巨大な物理candidate tableを反復検索せず、rank 0〜4で一致したreferenceごとに1行のTEMP集合を参照します。full / scoped / retained refreshのcandidate・ambiguity結果は全言語で維持されます。
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
category: changed
affected:
- src/CodeIndex/Indexer/CSharpStaticInterfacePrepass.cs
---

## English

- **Fresh C# prepass lookup assembly no longer flattens symbol lists** — per-file artifacts are enumerated through a bounded non-owning segmented view until immutable workspace lookups are complete, avoiding duplicate workspace-sized pointer buffers while preserving extraction order, evidence, checksum, and cache ownership contracts.

## 日本語

- **初回C# prepassのlookup構築がsymbol listをflattenしなくなりました** — immutableなworkspace lookupが完成するまでfile単位artifactをboundedなnon-owning segmented viewで列挙し、extraction順、evidence、checksum、cache所有権契約を維持しながらworkspace規模の重複pointer bufferを避けます。
14 changes: 14 additions & 0 deletions changelog.d/unreleased/+csharp-property-receiver-facts.changed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
category: changed
affected:
- src/CodeIndex/Database/DbWriter.References.cs
- src/CodeIndex/Database/DbWriter.ReferenceGraphRefreshScope.cs
---

## English

- **Initial C# property-receiver normalization now seeks compact facts** — full, scoped, and retained graph refreshes drive the two normalization updates from flagged reference IDs and primary-keyed field/property target identities instead of scanning all references and repeatedly probing persistent target symbols.

## 日本語

- **初回C# property-receiver normalizationがcompact factをseekするようになりました** — full / scoped / retained graph refreshは、全referenceをscanして永続target symbolを繰り返しprobeせず、flag済みreference IDとprimary-keyed field / property target identityから2つのnormalization updateを駆動します。
14 changes: 14 additions & 0 deletions changelog.d/unreleased/+csharp-type-family-candidates.changed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
category: changed
affected:
- src/CodeIndex/Database/DbWriter.References.cs
- tests/CodeIndex.Tests/DatabaseTests.cs
---

## English

- **Initial C# type-reference matching now evaluates partial families once** — rank-5 candidate construction groups exact name, arity, and type identity before matching references, then expands successful families back to every physical symbol, preserving candidate and ambiguity behavior without repeating the same compatibility work for each partial declaration.

## 日本語

- **初回C# type-reference照合がpartial familyごとに1回だけ評価されるようになりました** — rank 5 candidate構築はexact name・arity・type identityでgroup化してからreferenceを照合し、一致familyを全物理symbolへ展開するため、candidateとambiguityの動作を維持しながら各partial宣言で同じcompatibility処理を繰り返しません。
13 changes: 13 additions & 0 deletions changelog.d/unreleased/+csharp-workspace-type-lookups.changed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
category: changed
affected:
- src/CodeIndex/Indexer/References/Languages/CSharpReferenceExtractor.Support.cs
---

## English

- **C# workspace pattern lookups no longer construct a discarded known-type set** — enum-shadowing analysis now builds only the raw non-enum conflict names it consumes, reducing fresh-prepass allocation while preserving qualified type and member lookup behavior.

## 日本語

- **C# workspace pattern lookupが未使用のknown-type setを構築しなくなりました** — enum shadowing解析は実際に使うraw non-enum conflict nameだけを構築し、qualified type / member lookupの挙動を維持しながら初回prepassのallocationを削減します。
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
category: changed
affected:
- src/CodeIndex/Database/DbWriter.ReferenceSql.cs
- src/CodeIndex/Database/DbWriter.References.cs
---

## English

- **Fresh full indexes now assign reference sources during insertion** — the authoritative empty-database path resolves each language-neutral reference to its narrowest containing same-file symbol while preserving the 14-parameter batch shape, so graph finalization no longer scans and rewrites every reference solely to add source identity.

## 日本語

- **fresh full indexがreference挿入時にsourceを設定するようになりました** — authoritativeなempty-database経路は14 parameterのbatch shapeを維持しつつ、言語共通の各referenceを同一file内で最も狭く包含するsymbolへ解決するため、graph finalizationはsource identity追加だけのために全referenceをscan・rewriteしません。
16 changes: 16 additions & 0 deletions changelog.d/unreleased/+initial-index-write-batches.changed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
category: changed
affected:
- src/CodeIndex/Database/DbWriter.BatchSql.cs
- src/CodeIndex/Database/DbWriter.ChunkSymbolBatches.cs
- src/CodeIndex/Database/DbWriter.Issues.cs
- src/CodeIndex/Database/DbWriter.References.cs
---

## English

- **Initial full indexing now uses binding-efficient SQLite write batches** — caller-owned file transactions cap named parameters per chunk, symbol, issue, reference-line, and reference statement, avoiding repeated dense parameter-name lookup across every supported language while preserving public writer transaction and SAVEPOINT contracts. Per-statement cancellation checkpoints remain intact. For operations above 500 rows, persistent progress logs are emitted at 500-row boundaries and completion so the smaller statements do not introduce a synchronous log flush per statement.

## 日本語

- **初回フル索引がSQLite binding効率のよいwrite batchを使うようになりました** — caller-owned file transaction内のchunk、symbol、issue、reference-line、reference statementでnamed parameter数を制限し、全対応言語に共通する密なparameter name再探索を避けつつ、public writerのtransaction / SAVEPOINT契約を維持します。statementごとのcancellation checkpointは保ちます。500 rowを超えるoperationでは、永続progress logを500 row境界と完了時だけに出力し、小さなstatementごとの同期log flushを防ぎます。
14 changes: 14 additions & 0 deletions changelog.d/unreleased/+manual-performance-opt-in.changed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
category: changed
affected:
- tests/CodeIndex.Tests/ManualPerformanceFactAttribute.cs
- tests/CodeIndex.Tests/PerformanceTests.cs
---

## English

- **Manual performance tests now have an executable opt-in contract** — setting `CDIDX_RUN_MANUAL_PERFORMANCE_TESTS=1` together with a focused test filter runs the selected production-runtime benchmark, while ordinary CI still skips it. The 1,000-file search fixture is also named for its actual scale.

## 日本語

- **手動performance testに実行可能なopt-in契約を追加しました** — `CDIDX_RUN_MANUAL_PERFORMANCE_TESTS=1` とfocused test filterを併用すると選択したproduction-runtime benchmarkが実行され、通常CIでは引き続きskipされます。1,000-file search fixtureの名前も実際の規模に合わせました。
13 changes: 13 additions & 0 deletions changelog.d/unreleased/+reference-line-window-sizing.changed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
category: changed
affected:
- src/CodeIndex/Database/DbWriter.References.cs
---

## English

- **Initial reference-line persistence no longer hashes every context twice** — language-neutral atomic windows are bounded from worst-case batch rows before the single deduplicating materialization pass, removing a redundant tuple set while preserving window, rollback, and context-identity contracts.

## 日本語

- **初回reference-line永続化で全contextを二重hashしなくなりました** — 言語共通のatomic windowは単一の重複排除materializationより前にbatch rowの最悪ケースからboundされ、window、rollback、context identity契約を維持しながら冗長なtuple setを除去します。
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
category: changed
affected:
- src/CodeIndex/Database/DbWriter.References.cs
- src/CodeIndex/Database/DbWriter.ReferenceGraphRefreshScope.cs
---

## English

- **Initial reference resolution now constructs target-family keys once per symbol** — full, fresh, differential, scoped, and retained graph refreshes resolve candidate rows through primary-keyed TEMP facts instead of rebuilding long language/path/container/name keys for every physical candidate, with unchanged cross-language resolution states.

## 日本語

- **初回reference resolutionがtarget-family keyをsymbolごとに1回だけ構築するようになりました** — full / fresh / differential / scoped / retained graph refreshは、物理candidateごとに長いlanguage / path / container / name keyを再構築せずprimary-keyed TEMP factを通して解決し、言語横断のresolution stateを維持します。
4 changes: 2 additions & 2 deletions docs/test-doc-maintenance-plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ candidate before changing a skip.
|---|---|---|
| Target-framework or platform-specific | `ProductionCliFactAttribute`, `ProductionCliTheoryAttribute`, `ExternalProcessFactAttribute`, `ExternalProcessTheoryAttribute`, and practical-budget guards that run only on the production `net8.0` target. | Keep the shared skip-reason constants and their contract tests beside the attributes. Do not duplicate literal reasons at call sites. |
| External process or toolchain limitation | Published/trimmed CLI and installer paths that can be reported as skipped when SDK/ILLink/runtime availability prevents the test from reaching `cdidx` (#2586, #3571). | Keep the tracking issue in the reason or surrounding guide text, and prefer narrowing the environment guard over disabling broader coverage. |
| Performance-only or manual benchmark | `PerformanceTests` large-scale checks such as `Insert10KFiles` and extractor stress tests with manual `dotnet test --filter ...` instructions. | Keep them skipped by default, keep the command in the reason, and do not treat them as required PR gates. |
| Performance-only or manual benchmark | `PerformanceTests` large-scale checks such as `Insert10KFiles` and extractor stress tests using `ManualPerformanceFactAttribute`. | Keep them skipped by default, require `CDIDX_RUN_MANUAL_PERFORMANCE_TESTS=1` plus a focused `dotnet test --filter ...` invocation, and do not treat them as required PR gates. |
| Temporary investigation skip | Skips with owner, expiration, and `blocked by #NNNN` metadata. | A temporary skip must cite a tracking issue, owner, and expiry date. If that metadata is missing, remove the skip or open the tracking issue before adding it. |
| Intentionally disabled coverage | No standing class of untracked intentional disables should exist. | If coverage must remain disabled, create an issue first and use the temporary-skip format until the replacement coverage lands. |

Expand Down Expand Up @@ -204,7 +204,7 @@ exact-substring query を使い、変更前に候補を 1 件ずつ確認して
|---|---|---|
| 対象フレームワークまたはプラットフォーム固有 | `ProductionCliFactAttribute`、`ProductionCliTheoryAttribute`、`ExternalProcessFactAttribute`、`ExternalProcessTheoryAttribute`、production `net8.0` target だけで走る practical-budget guard。 | 共有 skip-reason constant とその contract test を attribute の近くに置く。call site に literal reason を重複させない。 |
| 外部プロセスまたはツールチェーン制約 | SDK/ILLink/runtime の可用性により `cdidx` に到達する前に skipped として報告されうる published/trimmed CLI と installer 経路(#2586、#3571)。 | reason または周辺 guide text に tracking issue を残し、広い coverage を止めるより environment guard を狭める。 |
| 性能専用または手動ベンチマーク | `PerformanceTests` の `Insert10KFiles` などの大規模チェックと、手動 `dotnet test --filter ...` 指示付き extractor stress test。 | 既定では skipped のままにし、reason に実行コマンドを残し、PR 必須 gate として扱わない。 |
| 性能専用または手動ベンチマーク | `ManualPerformanceFactAttribute` を使う `PerformanceTests` の `Insert10KFiles` などの大規模チェックと extractor stress test。 | 既定では skipped のままにし、`CDIDX_RUN_MANUAL_PERFORMANCE_TESTS=1` と focused `dotnet test --filter ...` の両方を必須とし、PR 必須 gate として扱わない。 |
| 一時調査用 skip | owner、expiration、`blocked by #NNNN` metadata を持つ skip。 | temporary skip は tracking issue、owner、expiry date を必ず持つ。metadata がなければ skip を削除するか、追加前に tracking issue を起票する。 |
| 意図的な無効化 | 未追跡の intentional disable を常設カテゴリとして持たない。 | coverage を無効化したままにする必要がある場合は先に issue を作り、replacement coverage が入るまで temporary-skip format を使う。 |

Expand Down
14 changes: 14 additions & 0 deletions src/CodeIndex/Database/DbWriter.BatchSql.cs
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ namespace CodeIndex.Database;
public partial class DbWriter
{
private const int BatchSize = 500;
// Microsoft.Data.Sqlite resolves every named parameter through SQLite again on
// each execution. Caller-owned transactions let us split dense writes without
// adding transaction scopes, so keep those statements below this binding budget.
private const int MaxCallerTransactionBatchParameters = 32;
private const int MaxFoldedNameCacheEntries = 4096;

private static object FoldedNameDbValue(string? name, Dictionary<string, string?> cache)
Expand Down Expand Up @@ -72,4 +76,14 @@ private static int GetRowsPerInsertStatement(int columnCount)

return Math.Max(1, Math.Min(BatchSize, SqliteDynamicSql.MaxSqlVariables / columnCount));
}

private static int GetRowsPerCallerTransactionInsertStatement(int columnCount)
{
if (columnCount <= 0)
throw new ArgumentOutOfRangeException(nameof(columnCount));

return Math.Max(1, Math.Min(
GetRowsPerInsertStatement(columnCount),
MaxCallerTransactionBatchParameters / columnCount));
}
}
Loading
Loading