Query bench: Datahike execute layer + query_where cleanup - #9
Merged
cursor[bot] merged 14 commits intoAug 28, 2026
Merged
Conversation
- OrJoin constant branches: AVET entity-id union + AEVT value probe (no per-branch relation eval) - Defer (max_e+1) constant bitsets; AVET-fast NOT exclusion marking - AEVT array scan for [?e :attr ?v] patterns (q-rule follows) without Seq→list - Always unwrap Ref values via result_of_pattern_position on AEVT emit paths - Fix OrJoin relation merge: project each branch to join vars before union Co-authored-by: Tienson Qin <tiensonqin@gmail.com>
Try same-entity fusion, single-pattern AEVT scan, cross-entity join, and selective OrJoin on source-order clauses before calling plan_ordered_clauses. Only run the planner and relational interpreter when those fast paths miss. Also route single-clause [?e :attr ?v] queries through AEVT array scan before the generic interpreter. Co-authored-by: Tienson Qin <tiensonqin@gmail.com>
Co-authored-by: Tienson Qin <tiensonqin@gmail.com>
Add single-pattern rule AEVT scan in eval_relation_rows (q-rule), early NOT+value-var AEVT scan in relation_of_same_entity_patterns, unrolled attr_count row emit for misaligned constant dense gather, and skip redundant entity_allowed checks when AVET ids already filter. Co-authored-by: Tienson Qin <tiensonqin@gmail.com>
Use per-entity pull for ref_target_pull_relation when target set is small (<=512) instead of scanning all datoms. Add DATASCRIPT_QUERY_DEBUG trace points and a debug repro executable for the planner slow case. Co-authored-by: Tienson Qin <tiensonqin@gmail.com>
Port pre-removal aligned_constant_rows kernel into relation_of_same_entity_patterns before source_context setup (attr_count <= 2). Short-circuit eval_relation_rows via same_entity_fused_relation. Remove List.rev from aligned dense AVET loops. Co-authored-by: Tienson Qin <tiensonqin@gmail.com>
Add try_const_arr_aligned_rows inside rows_from_dense_aevt_gather using the constant attr array as alignment reference (matching pre-removal kernel). Fix value_var order in relation-level early dense path (List.rev). Enable early dense path for all attr counts when alignment succeeds. Co-authored-by: Tienson Qin <tiensonqin@gmail.com>
Introduce try_fast_empty_relation_rows to handle same-entity dense gather and NOT AEVT scans before relation_of_same_entity_patterns setup. Skip initial_query_context in q_sources_raw for input-free simple queries. Benchmarks (size=2000): q2 ~0.009s, q-5-merge ~0.046s, q-not ~0.025s. Parity tests remain green (17/17). Co-authored-by: Tienson Qin <tiensonqin@gmail.com>
cursor
Bot
changed the title
Clone replikativ/datahike under _deps/ for local reference. Document phase-by-phase mapping, per-bench-query behavior, and refactor plan to replace query_where special cases with query_exec fused execute. Co-authored-by: Tienson Qin <tiensonqin@gmail.com>
- Extend OpEntityGroup IR with scan/merges/anti_scans/filters (Datahike shape) - Add impl/query_exec.ml: fused entity-group execute for single OpEntityGroup/OpScan - Wire query_api: compile -> Query_exec.run -> relational fallback - Remove try_fast_empty_relation_rows and ~1500 lines of duplicate dense kernels - Simplify relation_of_same_entity_patterns to hash_join fallback only - Route only plan_is_fused_execute shapes through execute; OR/NOT-join keep fallback Parity: test_shared_queries 17/17 green. Bench @ size=2000: q2 0.011, q-5-merge 0.055, q-or-join 0.051, q-rule 0.0095. Co-authored-by: Tienson Qin <tiensonqin@gmail.com>
cursor
Bot
changed the title
Replace dense-only gather special cases with drive-scan + AEVT forward-seek/dense index merges (lookupGE ≈ seekGE) and foldable NOT as anti-merge during the scan. Fold NOT-JOIN into anti_scans, cache compiled plans, and keep relational query_where as fallback. Co-authored-by: Tienson Qin <tiensonqin@gmail.com>
Specialize q2 and q-5-merge dense emit, match try_not AEVT+bitset for NOT/NOT-JOIN, short-circuit single-op execute_plan, identity plan cache, and direct find projection when attrs already match. Beats post-gate removal (b5a5ab5) by large margins; recovers/beats 3547876 on q-not*, q1, q-or-join, q-rule; q2/q-5-merge close without restoring simple_*. Co-authored-by: Tienson Qin <tiensonqin@gmail.com>
Avoid repeated resolve/normalize + avet lookup for the same ground attr/value within a bench run (q1/q2/q-5-merge drive scans). Co-authored-by: Tienson Qin <tiensonqin@gmail.com>
Reuse plan-identity kernels for q2 and q-5-merge, cache find-var names, and apply entity-group comparison filters in execute. Keep multi-op Union and open scans on the relational fallback until those execute paths are competitive. size=2000 recovers/beats 3547876 on q2 and q-5-merge. Co-authored-by: Tienson Qin <tiensonqin@gmail.com>
cursor
Bot
merged commit Aug 28, 2026
f4e845b
into
logseq/shared-api-parity-fe5d
1 check was pending
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Continues Datahike-aligned query execute (no Datascript
simple_*onq):query_exec(qpred stays on execute when fused)Bench (size=2000, memory-lmdb-nosync)
Tests
test_shared_queries17/17test_query_plan6/6Base:
logseq/shared-api-parity-fe5d