feat(bench): RFC 0039 benchmark harness, branch merge as first workload

[azimafroozeh]

What & why

Adds omnigraph-bench, the harness for the end-to-end benchmark (RFC 0039). It owns the general machinery: frozen validated fixtures, append-only schema-validated run records, an A/A noise floor with rule-7 claim labeling, and terminal plus self-contained HTML reporting. Branch merge is the first workload wired through it, measuring wall-clock with 14-phase attribution and storage-call counts per run.

First result from the harness: on the large fixture (8 tables, 100k rows, delta 5000) the three-way row walk is ~90% of merge wall-clock, and merge cost tracks table size, not delta size. The walk had no phase bucket before; the new TableWalk phase records it directly.

Backing issue / RFC

• Is an RFC PR, or implements a merged RFC: docs/rfcs/0039-end-to-end-benchmark.md (RFC PR rfc: add RFC 0039, the end-to-end benchmark #535; this PR lands after it merges)

Checklist

• Change is focused (one logical change: the harness plus the minimal engine seam it reads)
• Tests added/updated for behavior changes (48 unit tests in the crate; engine drift guards extended for the 14th phase)
• Public docs updated if user-facing surface changed (crate README, AGENTS.md crate list and topic map; internal instrument, no public API surface)
• Reviewed against docs/dev/invariants.md — no Hard Invariant weakened, no deny-list item hit (engine change is one observational enum variant plus two timing spans, no-op unless a probe is installed)

Local verification

• cargo fmt --all --check — clean
• cargo clippy -p omnigraph-bench --all-targets -- -D warnings — clean
• cargo clippy -p omnigraph-engine --lib -- -D warnings — clean
• cargo check -p omnigraph-engine --benches — clean
• cargo test -p omnigraph-bench — 48 passed, 0 failed
• bash scripts/check-agents-md.sh — OK (78 links, 72 docs)
• python3 scripts/generate_merge_bench_fixtures.py — builds the release binary and the three v1 fixtures end to end

Notes for reviewers

• Engine footprint: MergeTimingPhase::TableWalk plus two recording spans around stage_streaming_table_merge. Production pays one unset task-local check.
• Rule 6: nothing here gates anything, at any CI stage. Workspace member, publish = false.
• Records carry explicit identity (embedded SUT commit with dirty and stale-build detection, session and invocation ULIDs, machine spec, declared warmth). Env values with secret-looking names are redacted in records and reports.
• Fixtures are generated, never fetched: scripts/generate_merge_bench_fixtures.py is the dataset source of truth (deterministic per builder version, SHA-256 validation stamp; runs refuse unvalidated or modified fixtures).
• Deferrals are listed in the README ("What benchmark v1 defers"): S3/MinIO backend seam, realistic profile, remaining scenarios.

Greptile Summary

Adds the RFC 0039 end-to-end benchmark harness, with validated fixtures, append-only run records, A/A noise-floor comparisons, reporting, and branch-merge timing and storage instrumentation.

• Introduces the omnigraph-bench workspace crate and fixture-generation tooling.
• Adds schema-validated benchmark records, terminal/HTML reports, and per-point A/A comparison support.
• Adds TableWalk timing around the general three-way merge workload.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
crates/omnigraph-bench/src/run.rs	Orchestrates benchmark repetitions, validates every merged repetition, and produces records only after successful checks.
crates/omnigraph-bench/src/diff.rs	Compares records and now correctly withholds point-specific noise-floor labels from cross-point comparisons.
crates/omnigraph-bench/src/record.rs	Defines benchmark identity, specification, result, and noise-floor record structures.
crates/omnigraph-bench/src/report.rs	Produces self-contained benchmark reports from validated run records.
crates/omnigraph/src/exec/merge.rs	Adds observational timing spans around the general three-way table walk without changing merge semantics.
crates/omnigraph/src/instrumentation.rs	Extends merge timing instrumentation with the TableWalk phase used by the benchmark harness.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    F[Validated fixture] --> R[Benchmark repetitions]
    R --> M[Branch merge]
    M --> T[Wall-clock and phase timings]
    M --> S[Storage-call counters]
    T --> V[Per-repetition row validation]
    S --> V
    V --> J[Schema-validated run record]
    J --> D[Diff and A/A noise-floor labels]
    J --> P[Terminal and HTML reports]

Loading

_{Reviews (2): Last reviewed commit: "fix(bench): row-check every repetition, ..." | Re-trigger Greptile}

Context used:

• Knowledge Base — Branching and merge

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.