feat(async): chunk-level traversal (achunk-next) + AsyncReverseSeq termination-contract fix#20
Open
whilo wants to merge 1 commit into
Open
feat(async): chunk-level traversal (achunk-next) + AsyncReverseSeq termination-contract fix#20whilo wants to merge 1 commit into
whilo wants to merge 1 commit into
Conversation
PAsyncChunkedSeq/achunk-next on AsyncSeq resolves [keys-array start end next]: consumers run a synchronous inner loop over the leaf's key array and await once per leaf hop. Element-wise anext allocates a continuation per element (~4x slower warm, measured); the chunk shape is the one a query engine's scan kernels want, and its boundary semantics mirror the sync Iter's IChunkedSeq exactly (same-leaf right bound via path-get, next leaf via -next-path + path-lt guard). Parity is pinned chunk-for- chunk against Iter, in-memory and through restored async storage. Also fixes the PAsyncSeq termination contract in AsyncReverseSeq: anext was a literal two-slot vector of `when` forms, so an exhausted reverse seq yielded a truthy [nil nil] instead of nil — one spurious nil element for any consumer written against AsyncSeq's contract (aseq/transduce's if-let, chunk walkers). The guard now encloses the whole tuple; the AsyncRange test helper had the same shape and is fixed alike. The lookup facade gains the opts arity the other facades already have.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First PR of the datahike cljs-async arc (PR A of the grounded workplan).
Chunk-level async traversal
PAsyncChunkedSeq/achunk-nextonAsyncSeq: resolves[keys-array start end next]— the consumer runs a synchronous inner loop over the leaf's key array and awaits once per leaf, not per element. Element-wiseanextallocates a continuation per element (measured ~4x slower on warm data in the datahike engine experiments); the chunk shape is what a query engine's scan kernels consume, and awaiting per 512-element chunk measured within ~8% of a fully synchronous loop.Boundary semantics mirror the sync
Iter'sIChunkedSeqexactly: same-leaf right bound viapath-get, next leaf via-next-pathwith thepath-ltguard. Parity is pinned chunk-for-chunk againstIter(same boundaries, same elements) over full/mid/same-leaf/single/empty ranges plus a leaf-boundary sweep — both in-memory and through a restored async storage (cold node loads).AsyncReverseSeq termination-contract fix
anext's body was a literal two-slot vector ofwhenforms, so an exhausted reverse seq resolved a truthy[nil nil]instead ofnil— one spurious nil element for any consumer written againstAsyncSeq's contract (aseq/transduce'sif-let, chunk walkers). All internal constructors happen to guard emptiness, so this was latent — but external consumers constructing or re-driving reverse seqs would hit it. The guard now encloses the whole tuple; theAsyncRangetest helper had the same shape and is fixed alike; a regression test pinsanext → nilon exhausted seqs of both orientations.Also: the
lookupfacade gains the opts arity the other facades already have (datahike'sIIndex/-lookupopts threading needs it).Tests
node-tests(advanced): 137 tests / 1826 assertions green — includes the two new namespaces (async-exhaustion,async-chunk). cljs-only change; JVM untouched.