fix(io): arrow.load validates wire-contract row order before reshaping (#199 item 1) → v1.10.1 - #206
Merged
Conversation
…aping (#199 item 1, C-72) → v1.10.1 load reconstructed frames positionally but never checked the 'sample' column save wrote as the order contract — a reordered, truncated, or foreign-rewritten parquet reshaped plausible floats into the wrong sample slots silently. Now fails loud (ValueError) on: row count not a positive multiple of n_samples; sample column deviating from tile(arange(S), N); time/unit not constant within a sample block (cross-cell swaps the tile check alone cannot see). Whole-cell block reorders remain loadable (consistent tables). Implements the check views-postprocessing ADR-013 §4.5(b) made every consumer run themselves. Register C-72; #199 item 2 (mmap/partitioned) stays open. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses #199 item 1 (ordering validation on load). Item 2 (mmap/partitioned reading) deliberately stays open in the issue.
What
arrow.loadreconstructs frames positionally (reshape(n, s)) but never read back thesamplecolumn thatsavewrites as the order contract — so a reordered, truncated, or foreign-rewritten parquet loaded successfully with draws in the wrong sample slots, silently. This is the check views-postprocessing ADR-013 §4.5(b) previously required every consumer to run themselves on a separate raw-table read.loadnow validates the layout first and raisesValueErroron:n_samples.samplecolumn deviates from the writtentile(arange(S), N).time/unitnot constant within a sample block (invisible to the tile check alone).A whole-cell block reorder still loads — identifiers travel with their draws, so it is a consistent table (pinned by a green boundary test).
Governance
Register C-72 (born-resolved, Tier 2 — silent corruption, but only reachable via an out-of-contract intermediary mutating the file; the supported save→load path was always correct); CHANGELOG
[1.10.1]; version → 1.10.1 (fail-loud fix, no API change; banner guard is patch-tolerant). No ADR needed — implements an existing cross-repo contract requirement.Verification
ruff + format clean · mypy strict green (3.10 + 3.11) · pytest 100% line+branch (5 new tests: 3 red raises + truncation + the consistent-reorder green boundary) · validate_docs PASSED (
v1.10 ~ 1.10.1 OK).🤖 Generated with Claude Code