fix: minor correctness and hygiene fixes from the sync-pipeline audit - #259
Closed
d-v-b wants to merge 1 commit into
Closed
fix: minor correctness and hygiene fixes from the sync-pipeline audit#259d-v-b wants to merge 1 commit into
d-v-b wants to merge 1 commit into
Conversation
d-v-b
force-pushed
the
fix/perf-pr-minor-cleanups
branch
from
July 29, 2026 12:04
15224a7 to
457c85c
Compare
Owner
Author
|
🤖 AI text below 🤖 Merge-order note: this branch has small textual conflicts with #254 in |
Owner
Author
|
🤖 AI text below 🤖 Update: after #254 was narrowed (its async-only-inner-codec fix deferred to upstream zarr-developers#4179), the |
d-v-b
force-pushed
the
fix/perf-pr-minor-cleanups
branch
from
July 29, 2026 13:29
457c85c to
ef0c196
Compare
7 tasks
d-v-b
force-pushed
the
fix/perf-pr-minor-cleanups
branch
from
July 29, 2026 14:32
ef0c196 to
6d30b53
Compare
- Codec construction warnings (e.g. sharding's "disables partial reads") fired twice per array open, and on every decode/encode through the fused pipeline's async fallback. Re-constructions of an already-validated codec chain now go through codecs_from_list_unchecked, which validates structure without repeating first-construction advisory warnings; each warning fires exactly once per open under both pipelines. - concurrent_iter returned a lazy generator while its docstring promised eagerly scheduled tasks; it now materializes the task list so awaiting one at a time cannot serialize the batch. - A garbage codec_pipeline.max_workers value (e.g. from the environment) raised ValueError mid-read; it now warns and falls back to the default, consistent with tolerant handling of config input. - The as-completed pipeline helpers abandoned in-flight tasks when one failed, leaving stray background writes and "Task exception was never retrieved" warnings; failures now cancel and drain outstanding tasks. - Benchmarks: seed the data generator for reproducibility; fix a copy-pasted docstring. - Remove dead commented-out test blocks referencing the removed set_range API. Assisted-by: ClaudeCode:claude-sonnet-5
d-v-b
force-pushed
the
fix/perf-pr-minor-cleanups
branch
from
July 30, 2026 07:24
6d30b53 to
cdb0e00
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 AI text below 🤖
Bundles the minor findings from the PR zarr-developers#3885 audit — six independent items, one commit:
codecs_from_list_uncheckedvariant (named function per the two-semantics-two-names convention, no global warning-filter mutation). Regression test asserts exactly one warning per open under both pipelines; directfrom_codecsconstruction still warns once.concurrent_iterlaziness: it returned a generator of yet-to-be-created tasks while its docstring promised eager scheduling — a serialize-on-await trap for future callers. Now materializes the task list, with a flag-based eagerness test.max_workersenv parsing: a garbageZARR_CODEC_PIPELINE__MAX_WORKERSraisedValueErrormid-read; now warns and falls back to the default (tolerant handling for config/env input).set_range_synctest blocks left behind by the pre-merge removal of that API.One review note worth knowing:
BatchedCodecPipeline.evolve_from_array_specnow propagatesself.batch_sizeinstead of re-resolving config — verified safe (all production callers construct viafrom_codecs, which resolves config at that point; the sharding inner-pipeline cache already keys on the config value).Verification: warning test passes for both pipelines (fused failed pre-fix, TDD); 97 + 2042 targeted tests green; full repo suite 6589 passed pre-fix-round with the same items; mypy/ruff clean. Two-round task review (initial review caught the fused double-warn and a thread-safety hazard in the first approach; re-review verified both closed with no new issues).
🤖 Generated with Claude Code