CommandChanges.shouldCleanup short-circuits to NO if there is no data…#4873
Open
belliottsmith wants to merge 1 commit into
Open
CommandChanges.shouldCleanup short-circuits to NO if there is no data…#4873belliottsmith wants to merge 1 commit into
belliottsmith wants to merge 1 commit into
Conversation
Contributor
|
+1 |
ifesdjeen
approved these changes
Jun 11, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
This PR aims to prevent Accord journal cleanup/expunge decisions from being surfaced as NotDefined on the load path (which the PR description notes can cause linearizability violations). The visible changes in this repo primarily add/adjust tests and test utilities to reproduce and guard the behavior.
Changes:
- Add a new regression unit test (
AccordExpungeTest) coveringCleanup.EXPUNGEinteractions withCommandChangesreconstruction andjournal.loadCommand. - Extend
AccordGenerators.commandsBuilder()with an overload to allow customTxnIdgeneration (used by the new test). - Adjust distributed burn-test behavior and minor test cleanup/comment removals.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
test/unit/org/apache/cassandra/utils/AccordGenerators.java |
Adds an overload to build commands with a caller-supplied TxnId generator. |
test/unit/org/apache/cassandra/service/accord/AccordExpungeTest.java |
New regression test exercising EXPUNGE → loadCommand behavior across journal states. |
test/distributed/org/apache/cassandra/service/accord/journal/AccordJournalBurnTest.java |
Sets a global burn-test tuning parameter (CMD_BASE_CHECK_CHANCE) via static initializer. |
test/distributed/org/apache/cassandra/service/accord/AccordJournalCompactionTest.java |
Minor whitespace-only change. |
test/distributed/org/apache/cassandra/distributed/test/accord/AccordLoadTest.java |
Removes commented-out config lines. |
.gitmodules |
Points the modules/accord submodule at a personal fork/branch instead of the canonical Apache repo. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
belliottsmith
force-pushed
the
consistent-expunge
branch
10 times, most recently
from
fec9c89 to
f2b5980
Compare
ifesdjeen
approved these changes
Jun 16, 2026
| { | ||
| if (cell.column().isSimple()) | ||
| return new BTreeRow(clustering, BTree.singleton(cell), minDeletionTime(cell)); | ||
| return new BTreeRow(clustering, primaryKeyLivenessInfo, Deletion.LIVE, BTree.singleton(cell), minDeletionTime(cell)); |
Contributor
There was a problem hiding this comment.
Do we want to do
long minDeletionTime = Math.min(minDeletionTime(primaryKeyLivenessInfo), minDeletionTime(deletion.time()));
as we do in create above here too?
| Future<?> future = toFuture(AsyncResults.reduce(results, Reduce.toNull())); | ||
| long timeoutSeconds = getAccord().execute_waiting_on_start_timeout.toSeconds(); | ||
| if (timeoutSeconds <= 0) future.awaitUninterruptibly().rethrowIfFailed(); | ||
| else if (!future.awaitUninterruptibly(timeoutSeconds, SECONDS)) |
Contributor
There was a problem hiding this comment.
I think we would throw here only on timeout, but not on failure; should we add future.isSuccess() too?
Contributor
Author
There was a problem hiding this comment.
Weird, I thought I already handled that. Thanks!
belliottsmith
force-pushed
the
consistent-expunge
branch
from
0c7d366 to
4f8436c
Compare
CommandChanges.shouldCleanup short-circuits to NO if there is no data, but this is incorrect as Cleanup.EXPUNGE may have dropped the data and the record must receive cleanup EXPUNGE and be reported as ERASED. Also Fix: - Populate CFK system table row markers so we can stop reading sstables as soon as we have data - system_accord_debug.executors has wrong clustering type - RemoteToLocalVirtualTable should lazily allocate the partition collector to avoid LIMIT clause filtering removing it before it's populated - ActiveEpochs.withNewEpochs should handle transition from 0 -> more than 1 - RedundantBefore.minGcBefore should be NONE if empty - Update RangesForEpoch directly, so that we cannot have race conditions where the ownership is unknown - Avoid reentrancy on local callbacks - Ensure ReadCoordinator callbacks are invoked on owning thread - Avoid deadlock when notifying ComplexListener(s) - Release IntrusivePriorityHeap memory from large capacity heaps when empty - Prevent SynchronousRecoverAwait reentrancy when invoking onDone (by exposing and invoking invokeOnDone that first sets isDone) - maybeExecute must invoke either notWaiting or notifyWaiting to ensure tryExecuteListening terminates Also Improve: - Configurable execute_waiting_on_start patch by Benedict; reviewed by Alan Wang and Alex Petrov for CASSANDRA-21440
belliottsmith
force-pushed
the
consistent-expunge
branch
from
4f8436c to
c826edd
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…, but this is incorrect as Cleanup.EXPUNGE may have dropped the data and the record must receive cleanup EXPUNGE and be reported as ERASED. This can lead to lienarizability violations.
Thanks for sending a pull request! Here are some tips if you're new here:
Commit messages should follow the following format:
The Cassandra Jira