[Vector Upsert 1/5] Support allowed-document filtering in exact vector scan

[xiangfu0]

Part 1/5 of the split of #19287 (Fix FULL-upsert vector candidate generation).

Summary

Groundwork for constraining vector candidate generation to the documents a query may
actually see (wired up in part 4/5):

• Extract a shared computeExactMatches helper in ExactVectorScanFilterOperator that
  unifies the exact top-K and threshold scan paths and can restrict scoring to an
  allowed-document bitmap. Only allowed document IDs are read from the forward index, and
  IDs beyond the numDocs watermark are ignored.
• Introduce VectorCandidateScope, an immutable set of document IDs a vector predicate
  is allowed to consider as candidates. Vector top-K is not monotonic: unlike an ordinary
  predicate — which is correct to intersect with the result afterwards — a document set
  that defines what the query may see has to be applied before candidate generation.
  The type carries that contract and deliberately says nothing about where the
  restriction came from; callers decide that. ExactVectorScanFilterOperator accepts one
  through a new constructor overload, and no production caller passes one yet.
• Let the operator record its own exact-scan search and fallback metrics. Exact scans
  were previously invisible to VectorSearchMetrics entirely.
• Report the applied candidate-filter cardinality in explain output, emitted only when a
  scope actually restricts the scan.
• Add filtered-reader coverage for HnswVectorIndexReader.getDocIds(query, k, allowed).

Validation

All commands ran with JDK 25.

• ExactVectorScanFilterOperatorTest: 16 tests passed (6 new allowed-document tests).
• HnswVectorIndexCreatorTest: 6 tests passed (1 new filtered-reader test).
• Spotless, Checkstyle, and license checks passed.

Stack

1. [Vector Upsert 1/5] Support allowed-document filtering in exact vector scan #19297 — exact-scan allowed-document filtering
2. [Vector Upsert 2/5] Harden vector search metric recording and backend param cleanup #19298 — backend param cleanup hardening
3. [Vector Upsert 3/5] Never materialize nested vector subtrees as metadata filters #19299 — nested vector metadata guard
4. [Vector Upsert 4/5] Apply the query's visible-document set before vector candidate generation #19300 — apply the visible-document set before candidate generation (core fix)
5. [Vector Upsert 5/5] Add FULL-upsert vector integration coverage #19301 — integration coverage

[Copilot]

Pull request overview

Adds allowed-document filtering groundwork for exact vector scans, supporting future FULL-upsert candidate scoping.

Changes:

• Adds immutable VectorCandidateScope.
• Unifies filtered top-K and threshold exact scans.
• Adds exact-scan and filtered HNSW tests.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
ExactVectorScanFilterOperator.java	Implements bitmap-restricted exact scanning and explain attributes.
VectorCandidateScope.java	Adds immutable query-scoped document constraints.
ExactVectorScanFilterOperatorTest.java	Tests filtered exact-scan behavior and boundaries.
HnswVectorIndexCreatorTest.java	Tests filtered HNSW retrieval.

Suppressed comments (1)

pinot-core/src/main/java/org/apache/pinot/core/operator/filter/ExactVectorScanFilterOperator.java:293

• This reports the unbounded input bitmap cardinality rather than the effective set actually scanned. For the new {2, 99} / numDocs = 3 case, the operator scores one document but reports two effective allowed IDs. Count only IDs in [0, _numDocs) so explain output matches the watermark behavior.

  private int getEffectiveAllowedDocIdsCardinality() {
    return getRequiredUpsertCandidateCardinality();

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 82.08955% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.20%. Comparing base (e4888b0) to head (0edb57b).

Files with missing lines	Patch %	Lines
...operator/filter/ExactVectorScanFilterOperator.java	83.05%	6 Missing and 4 partials ⚠️
...not/core/operator/filter/VectorCandidateScope.java	75.00%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #19297      +/-   ##
============================================
- Coverage     67.20%   67.20%   -0.01%     
  Complexity     1424     1424              
============================================
  Files          3462     3463       +1     
  Lines        220361   220388      +27     
  Branches      35147    35156       +9     
============================================
+ Hits         148087   148103      +16     
- Misses        60467    60474       +7     
- Partials      11807    11811       +4     

Flag	Coverage Δ
integration	100.00% <ø> (ø)
integration1	100.00% <ø> (ø)
integration2	0.00% <ø> (ø)
java-25	67.20% <82.08%> (-0.01%)	⬇️
lane-a	100.00% <ø> (ø)
lane-b	0.00% <ø> (ø)
temurin	67.20% <82.08%> (-0.01%)	⬇️
unittests	67.19% <82.08%> (-0.01%)	⬇️
unittests1	57.84% <82.08%> (+<0.01%)	⬆️
unittests2	39.21% <0.00%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.