Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
2180e8e
Add Cloud Agent environment with OCaml 5.5
cursoragent Aug 26, 2026
2de5d74
Add LMDB storage backend and SQLite comparison benchmark
cursoragent Aug 26, 2026
cdc61f4
Document non-PSS LMDB design with Scheme A index types
cursoragent Aug 26, 2026
2f53f7d
Remove file storage and transaction tail from LMDB storage
cursoragent Aug 26, 2026
49de381
Fix LMDB index ordering, storage sync, and restore semantics
cursoragent Aug 26, 2026
1eeda4a
Fix LMDB overlay immutability and storage sync
cursoragent Aug 26, 2026
56c365b
Batch LMDB index writes and cursor range reads (Datalevin-style)
cursoragent Aug 27, 2026
6865015
Optimize LMDB init with bulk overlay indexes and attr caches
cursoragent Aug 27, 2026
ea32d01
Optimize bulk LMDB slices and AVET entity-id query path
cursoragent Aug 27, 2026
635b2e8
Fix query fast path and attr grouping for LMDB benchmarks
cursoragent Aug 27, 2026
90dfbd7
Add dbval-aligned temporal db API (phase 1)
cursoragent Aug 27, 2026
6820cd1
Remove LMDB overlay: append-only index with tx-filter reads
cursoragent Aug 27, 2026
5e2e430
Add session pending_datoms staging for fast transact without storage
cursoragent Aug 27, 2026
3487d7a
Store: append-only delta index sync instead of full rewrite
cursoragent Aug 27, 2026
04d0a3f
Add Datahike-style purge (excise) API for permanent datom removal
cursoragent Aug 27, 2026
ab4c7ec
Expand tx_history tests for dbval/Datahike temporal API coverage
cursoragent Aug 27, 2026
2ebfa6b
Make storage backends pluggable with string kinds and separate packages
cursoragent Aug 27, 2026
2c04b6d
Fix tx_history since+history test to use exclusive since tx0
cursoragent Aug 27, 2026
e1457b5
Remove sync_merged overlay path and fix temporal fold_datoms
cursoragent Aug 27, 2026
745517b
Add Datahike benchmark parity and avet predicate range fast path
cursoragent Aug 27, 2026
da87d76
Add query planner ADR, fix q2 same-entity joins, expand parity tests
cursoragent Aug 27, 2026
e82959c
Fix AVET value-range scans to seek and stop on encoded bounds
cursoragent Aug 27, 2026
ac1a172
Optimize AVET range scans and label benchmark storage backends
cursoragent Aug 27, 2026
f80411a
Align benchmark storage with in-memory nosync LMDB profile
cursoragent Aug 27, 2026
12fbd67
Speed up same-entity join queries and fast benchmark defaults
cursoragent Aug 27, 2026
73f07b3
Add single-query filter to Datahike compare benchmarks
cursoragent Aug 27, 2026
4fc6266
Fix same-entity join fast path for mixed constant/value queries
cursoragent Aug 27, 2026
e8c96a4
Add fast paths for q5, q-or-join, and q-not-join benchmarks
cursoragent Aug 27, 2026
5d291d9
Fix attr-only index slice and fold_slice prefix bounds
cursoragent Aug 27, 2026
70b3504
Add fast path for non-recursive follow rule query (q-rule)
cursoragent Aug 27, 2026
5dc1a05
Document Datahike vs OCaml query gaps; fast-path bind_var equality
cursoragent Aug 27, 2026
c12e8e7
Simplify query paths to align with Datahike structure
cursoragent Aug 27, 2026
49c3356
Use per-entity attr lookup for single-value same-entity queries
cursoragent Aug 27, 2026
5819867
Align multi-value same-entity queries with Datahike entity-group pattern
cursoragent Aug 27, 2026
79fd8c7
Speed up same-entity value lookups with direct Aevt array search
cursoragent Aug 27, 2026
df8d464
Fix LMDB read txn, AVET seek/range, and unify index codec with SQLite
cursoragent Aug 27, 2026
a60563b
Remove legacy PSS Logseq SQLite examples and fix unique checks mid-tx
cursoragent Aug 27, 2026
bf64449
Disable benchmark/perf gates and lein-dependent cross-runtime tests
cursoragent Aug 27, 2026
f5d3a42
Add Datahike category parity tests and fix as_of query visibility
cursoragent Aug 27, 2026
f1a2a2d
Strengthen Datahike result identity and speed qpred2 range scans
cursoragent Aug 27, 2026
1d07749
Speed same-entity and not/not-join query fast paths
cursoragent Aug 27, 2026
c96fbbe
Speed same-entity/not paths and use shared-API naming
cursoragent Aug 27, 2026
83ed53a
Add rules-wide-7x3 golden and dense gather for multi-constant joins
cursoragent Aug 27, 2026
d1e6056
Tighten q-5-merge dense gather and bench result consumption
cursoragent Aug 27, 2026
9071bcd
Add query plan IR with cost-ordered where clauses
cursoragent Aug 27, 2026
9b07011
Apply temporal views to seek/rseek with streaming filter
cursoragent Aug 27, 2026
0d4719f
Stabilize attr caches under append-only history views
cursoragent Aug 27, 2026
cc549a8
Add Instant as-of, history purge-before, and LMDB GC sync
cursoragent Aug 27, 2026
b000d19
Inline non-recursive rules into the relation query path
cursoragent Aug 27, 2026
fe77d25
Do not evaluate or-join via plain-or relation path
cursoragent Aug 27, 2026
3c53419
Harden Cloud Agent OCaml 5.5 env and fix install build
cursoragent Aug 27, 2026
62c1ff2
Fix js_of_ocaml benches to use the jsoo package
cursoragent Aug 27, 2026
34ac9b0
Update query planner docs for bench_ocaml_js artifact path
cursoragent Aug 27, 2026
a682468
Add LMDB vs SQLite compare for full shared query suite
cursoragent Aug 27, 2026
1c432ed
Make cloud-agent-install self-bootstrap opam and OCaml 5.5
cursoragent Aug 27, 2026
4608921
Add plan for SQLite index without LMDB dependency
cursoragent Aug 27, 2026
c924f89
Add SQLite index ported from LMDB index
cursoragent Aug 27, 2026
fbc2236
Implement SQLite Share_index_db without LMDB mirror
cursoragent Aug 27, 2026
5b99ef9
Harden SQLite Share path (Phase 4) and rename index codec
cursoragent Aug 27, 2026
4fcc6d6
Add LMDB vs SQLite operator choice documentation
cursoragent Aug 27, 2026
0c820ce
Make LMDB/SQLite compare benches disk-only with durable flush
cursoragent Aug 27, 2026
841852c
Add LMDB vs SQLite index-scan microbench and LMDB reopen
cursoragent Aug 27, 2026
e7d209b
Fix index-scan seek case to use seek_datoms
cursoragent Aug 27, 2026
626dc8f
Add 1M size to index-scan microbench
cursoragent Aug 27, 2026
3547876
Use 200k and 500k as index-scan default sizes
cursoragent Aug 27, 2026
b5a5ab5
Remove Datascript.q simple_* shape-gate bypasses
cursoragent Aug 27, 2026
9c81255
Align query planner with Datahike logical IR
cursoragent Aug 27, 2026
cef4841
Document Datahike-aligned query planner and fallback
cursoragent Aug 27, 2026
48c3394
Speed same-entity lookups, NotJoin fold, and OrJoin relations
cursoragent Aug 28, 2026
ee99b2f
Speed queries with dense AEVT gather, hash-probe joins, and AVET rows
cursoragent Aug 28, 2026
21e59fe
Tighten dense AEVT gather specialized row emit path
cursoragent Aug 28, 2026
9222bd4
Use AVET entity-id arrays and unrolled dense row emit
cursoragent Aug 28, 2026
891ebfb
Speed bench queries with AVET OrJoin, lazy bitsets, and AEVT scans
cursoragent Aug 28, 2026
e8b5575
Skip Query_plan.compile on fused fast-path hits
cursoragent Aug 28, 2026
556df05
Add AGENTS.md rule: use debug logs to find root cause before fixing
cursoragent Aug 28, 2026
326670c
Speed relation eval with rule, not, and dense gather fast paths
cursoragent Aug 28, 2026
24eb404
Fix wildcard pull slow path and add gated query debug logging
cursoragent Aug 28, 2026
2b699f6
Add early aligned-constant dense gather for q2-shaped queries
cursoragent Aug 28, 2026
1272379
Use const-first AEVT alignment for q-5-merge dense gather
cursoragent Aug 28, 2026
fe8abcc
Add lightweight eval_relation_rows fast path for bench queries
cursoragent Aug 28, 2026
69794cc
Add Datahike vs OCaml query pipeline comparison doc
cursoragent Aug 28, 2026
ea12362
Add query_exec layer and simplify query_where fallback
cursoragent Aug 28, 2026
a571c3e
Align entity-group execute with Datahike sorted-merge
cursoragent Aug 28, 2026
f6875ae
Tighten Datahike entity-group execute hot paths
cursoragent Aug 28, 2026
2f34a8e
Cache last AVET entity-id array in query_exec hot path
cursoragent Aug 28, 2026
f4e845b
Cache resolved entity-group kernels on the Datahike execute path
cursoragent Aug 28, 2026
57b5528
Fix compile_plan crash on function-valued query clauses
cursoragent Aug 28, 2026
2fe723e
Use Platform.now_seconds in query debug_log
cursoragent Aug 28, 2026
c15b0f4
Add Query_exec path probes and fused/fallback parity tests
cursoragent Aug 28, 2026
d0e4bcb
Add storage RSS bench for memory/LMDB/SQLite vs main
cursoragent Aug 28, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .cursor/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
FROM ubuntu:24.04

ENV DEBIAN_FRONTEND=noninteractive
ENV OPAMYES=1

# System packages + opam as root so /usr/bin/opam is available to every shell.
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
ca-certificates \
curl \
git \
build-essential \
pkg-config \
bubblewrap \
opam \
libsqlite3-dev \
liblmdb-dev \
&& rm -rf /var/lib/apt/lists/*

# Node.js 24 for js_of_ocaml / Melange smoke tests.
RUN curl -fsSL https://deb.nodesource.com/setup_24.x | bash - \
&& apt-get install -y --no-install-recommends nodejs \
&& rm -rf /var/lib/apt/lists/*

RUN useradd -m -s /bin/bash ubuntu 2>/dev/null || true

USER ubuntu
WORKDIR /home/ubuntu

# OCaml 5.5 toolchain lives in the image; project opam deps are installed by
# .cursor/cloud-agent-install.sh after the repository checkout is available.
RUN opam init --disable-sandboxing -a -y \
&& opam update -a \
&& opam switch create 5.5 ocaml-base-compiler.5.5.0 -y \
&& eval "$(opam env --switch=5.5)" \
&& opam install dune ocamlfind -y

RUN echo 'test -r ~/.opam/opam-init/init.sh && . ~/.opam/opam-init/init.sh > /dev/null 2> /dev/null || true' >> ~/.bashrc \
&& echo 'eval $(opam env --switch=5.5 2>/dev/null)' >> ~/.bashrc
67 changes: 67 additions & 0 deletions .cursor/cloud-agent-install.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
#!/usr/bin/env bash
# Idempotent Cloud Agent bootstrap.
# Works both when the Dockerfile already provides opam/OCaml 5.5 and when a
# Personal/DB-managed base image does not (install must self-bootstrap).
set -euo pipefail

repo_root="$(git rev-parse --show-toplevel)"
cd "$repo_root"

export DEBIAN_FRONTEND=noninteractive
export OPAMYES=1
export OPAMCOLOR=never

need_sudo() {
if [ "$(id -u)" -eq 0 ]; then
"$@"
elif command -v sudo >/dev/null 2>&1; then
sudo DEBIAN_FRONTEND=noninteractive "$@"
else
echo "Need root or sudo to install system packages: $*" >&2
exit 1
fi
}

ensure_system_packages() {
local missing=0
for pkg in opam pkg-config libsqlite3-dev liblmdb-dev build-essential bubblewrap curl ca-certificates git; do
if ! dpkg -s "$pkg" >/dev/null 2>&1; then
missing=1
break
fi
done
if [ "$missing" -eq 0 ] && command -v opam >/dev/null 2>&1; then
return 0
fi
need_sudo apt-get update
need_sudo apt-get install -y --no-install-recommends \
ca-certificates \
curl \
git \
build-essential \
pkg-config \
bubblewrap \
opam \
libsqlite3-dev \
liblmdb-dev
}

ensure_system_packages

if ! command -v opam >/dev/null 2>&1; then
echo "opam is still missing after apt install" >&2
exit 1
fi

if [ ! -d "${HOME}/.opam" ]; then
opam init --disable-sandboxing -a -y
fi

if ! opam switch list --short 2>/dev/null | grep -qx '5.5'; then
opam switch create 5.5 ocaml-base-compiler.5.5.0
fi

eval "$(opam env --switch=5.5)"

opam install . --deps-only --with-test -y
dune build @install
9 changes: 9 additions & 0 deletions .cursor/environment.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"name": "DataScript OCaml (OCaml 5.5)",
"user": "ubuntu",
"build": {
"dockerfile": "Dockerfile",
"context": ".."
},
"install": ".cursor/cloud-agent-install.sh"
}
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ jobs:
lein: latest

- name: Install system dependencies
run: sudo apt-get update && sudo apt-get install -y libsqlite3-dev pkg-config
run: sudo apt-get update && sudo apt-get install -y libsqlite3-dev liblmdb-dev pkg-config

- name: Install OCaml dependencies
run: opam install . --deps-only --with-test -y
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,5 @@ tmp/
/db.sqlite-shm
/db.sqlite-wal
/_deps/
/vendor/
_bench_data/
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
- Code and comments should be written in English.
- Solve root causes, not workarounds.
- When debugging a problem, do not guess and patch blindly. Add targeted debug logging (or other runtime evidence), identify the root cause, then implement the fix.
- Prefer simple implementations over complex ones.
- All observable behavior should match upstream DataScript.
- Implementation details should match upstream DataScript unless a divergence is explicitly requested and documented.
Expand Down
1 change: 0 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,6 @@ to use a different checkout or compiled JS bundle.
- `type/`: shared public type definitions
- `impl/`: implementation modules
- `test/`: unit, integration, js_of_ocaml, and cross-runtime tests
- `examples/`: small executable examples
- `bench/`: benchmark entry points
- `script/`: parity and benchmark helper scripts

Expand Down
Loading
Loading