fix: ManagedMemoryStore sync methods must respect the path prefix - #258
Closed
d-v-b wants to merge 6 commits into
Closed
fix: ManagedMemoryStore sync methods must respect the path prefix#258d-v-b wants to merge 6 commits into
d-v-b wants to merge 6 commits into
Conversation
d-v-b
force-pushed
the
fix/store-sync-async-parity
branch
from
July 29, 2026 11:56
59f73ad to
8b5c7dd
Compare
d-v-b
force-pushed
the
fix/store-sync-async-parity
branch
from
July 29, 2026 13:29
8b5c7dd to
4ee9723
Compare
Merged
7 tasks
d-v-b
force-pushed
the
fix/store-sync-async-parity
branch
from
July 29, 2026 14:32
4ee9723 to
78228b1
Compare
ManagedMemoryStore inherited get_sync/set_sync/delete_sync from MemoryStore, which use the raw key, while every async method prefixed keys with self.path. Any code taking the sync fast path (e.g. FusedCodecPipeline) wrote/read chunks outside the store's path prefix, so a fresh handle re-reading through the prefix silently got fill values. Override the three sync methods to prefix like their async counterparts. GpuMemoryStore.set_sync gets the same treatment: it now converts its value to a gpu.Buffer like set does, preserving the store's all-values-are-gpu invariant for the sync API. Also fix ManagedMemoryStore.get_partial_values, which applied self.path twice whenever path was non-empty (it pre-prefixed keys, then delegated to MemoryStore.get_partial_values, which itself dispatches through the already-overridden self.get) -- this made it return None for every key. Discovered via the strengthened test fixture below. Add sync/async parity laws to the shared StoreTests suite so every store subclass exercises this invariant: set through one API and read through the other (including byte_range variants), and confirm delete_sync is visible to async get. These are the tests that would have caught the ManagedMemoryStore bug. TestManagedMemoryStore's raw set/get test helpers now respect self.path, and store_kwargs uses a non-empty path, so prefix handling is actually exercised instead of passing vacuously. Add an end-to-end regression with FusedCodecPipeline writing to a ManagedMemoryStore(path=...) sharing a dict with a fresh handle. Add a LocalStore.delete_sync directory-branch test. Assisted-by: ClaudeCode:claude-sonnet-5
d-v-b
force-pushed
the
fix/store-sync-async-parity
branch
from
July 29, 2026 14:35
78228b1 to
1a20c4c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 AI text below 🤖
Fixes the third critical from the PR zarr-developers#3885 audit, and adds the store-suite contract laws that would have caught it.
The bug:
ManagedMemoryStoreoverrides every async method to prefix keys withself.path, but inheritedget_sync/set_sync/delete_syncfromMemoryStore, which use raw keys. The inherited methods satisfy the sync protocols, soFusedCodecPipelinetook the sync fast path and wrote chunks outside the store's namespace: withpath="subdir", metadata landed atsubdir/zarr.jsonbut chunks atc/0; re-reading through a fresh handle silently returned fill values. Two stores sharing a dict with different paths cross-talked. The three sync methods now mirror the async overrides via_join_paths.Bonus bug found by the new fixture:
ManagedMemoryStore.get_partial_valuesdouble-applied the prefix (the base implementation dispatches through the already-prefixingself.get), so it returnedNonefor every key wheneverpathwas non-empty. Fixed by removing the redundant override.Why tests missed both:
TestManagedMemoryStoreused an emptypathand its raw set/get helpers also skipped the prefix — the prefix machinery was never exercised. The fixture now uses a non-empty path and the helpers respect it.Structural fix — sync/async contract laws in
StoreTests, inherited by every store subclass (protocol-based skips for non-sync stores): async-set→get_syncandset_sync→async-get parity on the same instance;delete_syncvisibility to asyncget;get_syncbyte_rangeparity (Range/offset/suffix/past-EOF — previously untested on any store). Plus:GpuMemoryStore.set_syncnow performs the same cpu→gpu conversion as asyncset(gpu-marked test), aLocalStore.delete_syncdirectory-branch test, and an end-to-end fused-pipeline regression asserting no keys escape the prefix.15 new tests fail pre-fix (TDD); full suite 6831 passed / 404 skipped / 4 xfailed; both audit repro scripts run clean.
🤖 Generated with Claude Code