CASSANDRA-18216: Allow sharding of the SAI in-memory index#4885
Open
naseryyash wants to merge 5 commits into
Open
CASSANDRA-18216: Allow sharding of the SAI in-memory index#4885naseryyash wants to merge 5 commits into
naseryyash wants to merge 5 commits into
Conversation
- Refactor MemtableIndex from concrete class to interface
- Add UnshardedMemtableIndex preserving existing behavior
- Add ShardedMemtableIndex with token-range sharded TrieMemoryIndex[]
- Extend ShardBoundaries with precomputed partition ranges and shard-to-key-range intersection
- Add MemtableIndexManager factory routing based on index options
- Sharding is opt-in via CREATE INDEX WITH OPTIONS = {'shards': '<count>'}
- Reject shards option on vector indexes at validation time
- Cross-shard iterator uses MergeIterator with PrimaryKeysMergeReducer for flush path
…terator<PrimaryKey> - Add MemtableIndexFlushBench for low-cardinality sharded flush measurement - Change MemtableIndex.iterator() return type from PrimaryKeys to Iterator<PrimaryKey> - PrimaryKeysMergeReducer chains per-shard iterators lazily via Reducer.Trivial - Update UnshardedMemtableIndex and RowMapping.merge() to match new interface
- Add ShardedMemtableIndexTest and refactor shared helpers from TrieMemoryIndexTest into SAIRandomizedTester - Add RangeIntersectsBoundsTest for Range.intersects() coverage across all bound types - Add DDL test rejecting shards option on vector indexes
naseryyash
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CASSANDRA-18216: Allow sharding of the SAI in-memory index
patch by Yash Nasery; reviewed by Caleb Rackliffe for CASSANDRA-18216