bench: windowed aggregation benchmarks + scalability tests (1–16 workers) - #231

Merged

milindsrivastava1997 merged 30 commits into

mainfrom

pr/D-benchmarks-scalability

Apr 6, 2026

Commits on Mar 31, 2026

Add standalone precompute engine to replace Arroyo streaming pipeline

zzylol
and
claude
committed
Wire up DatasketchesKLL in accumulator factory and add E2E test

zzylol
and
claude
committed
Add 1000-sample batch latency test to E2E precompute test

zzylol
and
claude
committed
Add 10M-sample throughput test to E2E precompute test

zzylol
and
claude
committed
update
zzylol
committed
Fix sliding window aggregation: feed samples into all overlapping windows

zzylol
and
claude
committed
fix: remove duplicate [[bin]] entries in Cargo.toml from rebase merge

zzylol
and
claude
committed
style: apply cargo fmt

zzylol
and
claude
committed
fix: use std::io::Error::other per clippy lint

zzylol
and
claude
committed
fix: remove unnecessary cast per clippy lint

zzylol
and
claude
committed
Add standalone precompute engine to replace Arroyo streaming pipeline

zzylol
and
claude
committed
Wire up DatasketchesKLL in accumulator factory and add E2E test

zzylol
and
claude
committed
Add late data handling for closed windows and precompute engine design doc

zzylol
and
claude
committed
Minor wording fix in precompute engine design doc

zzylol
and
claude
committed
Clarify single-machine multi-threaded architecture in design doc

zzylol
and
claude
committed
Add multi-connector ingest support (Prometheus + VictoriaMetrics)

zzylol
and
claude
committed
Run cargo fmt

zzylol
and
claude
committed
Add pane-based incremental sliding window computation

zzylol
and
claude
committed
Add standalone precompute engine to replace Arroyo streaming pipeline

zzylol
and
claude
committed
Wire up DatasketchesKLL in accumulator factory and add E2E test

zzylol
and
claude
committed
Add late data handling for closed windows and precompute engine design doc

zzylol
and
claude
committed
Add multi-connector ingest support (Prometheus + VictoriaMetrics)

zzylol
and
claude
committed
Add windowed aggregation benchmarks and fix build issues

zzylol
and
claude
committed
Add scalability benchmark varying worker count (1-16)

zzylol
and
claude
committed
Make throughput test multi-threaded for data generation and sending

zzylol
and
claude
committed
Increase throughput test drain timeout from 60s to 120s

zzylol
and
claude
committed

Commits on Apr 6, 2026

Resolve merge conflicts with main (pane-based sliding windows)

zz_y
and
claude
committed
fix: align test_e2e_precompute with current API signatures

zz_y
and
claude
committed
fix: add missing value_column arg to AggregationConfig::new

zz_y
and
claude
committed
fix: resolve clippy lints in test_e2e_precompute

zz_y
and
claude
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

bench: windowed aggregation benchmarks + scalability tests (1–16 workers) - #231

bench: windowed aggregation benchmarks + scalability tests (1–16 workers)#231
milindsrivastava1997 merged 30 commits into
mainfrom
pr/D-benchmarks-scalability

Commits on Mar 31, 2026

Add standalone precompute engine to replace Arroyo streaming pipeline

Wire up DatasketchesKLL in accumulator factory and add E2E test

Add 1000-sample batch latency test to E2E precompute test

Add 10M-sample throughput test to E2E precompute test

update

Fix sliding window aggregation: feed samples into all overlapping windows

fix: remove duplicate [[bin]] entries in Cargo.toml from rebase merge

style: apply cargo fmt

fix: use std::io::Error::other per clippy lint

fix: remove unnecessary cast per clippy lint

Add standalone precompute engine to replace Arroyo streaming pipeline

Wire up DatasketchesKLL in accumulator factory and add E2E test

Add late data handling for closed windows and precompute engine design doc

Minor wording fix in precompute engine design doc

Clarify single-machine multi-threaded architecture in design doc

Add multi-connector ingest support (Prometheus + VictoriaMetrics)

Run cargo fmt

Add pane-based incremental sliding window computation

Add standalone precompute engine to replace Arroyo streaming pipeline

Wire up DatasketchesKLL in accumulator factory and add E2E test

Add late data handling for closed windows and precompute engine design doc

Add multi-connector ingest support (Prometheus + VictoriaMetrics)

Add windowed aggregation benchmarks and fix build issues

Add scalability benchmark varying worker count (1-16)

Make throughput test multi-threaded for data generation and sending

Increase throughput test drain timeout from 60s to 120s

Commits on Apr 6, 2026

Resolve merge conflicts with main (pane-based sliding windows)

fix: align test_e2e_precompute with current API signatures

fix: add missing value_column arg to AggregationConfig::new

fix: resolve clippy lints in test_e2e_precompute