Fix BEAM benchmark correctness and reproducibility

[ved015]

Stack

This PR is stacked on #44 (BEAM). Review and merge #44 first; this PR contains the correctness and reproducibility hardening developed during its architecture review.

What changed

• Load BEAM from a pinned, prepared public-dataset snapshot and fail closed on revision, schema, count, question-type, rubric, and haystack mismatches.
• Make benchmark behavior explicit through versioned retrieval, answer, evaluation, and aggregation protocols instead of BEAM-specific orchestrator branches.
• Implement the BEAM paper scoring path, including structured nugget judgments and Kendall tau-b event-ordering scoring; keep the Mem0-style nugget-only profile explicit and non-official.
• Normalize provider search results into one text/score/session/timestamp contract and make providers honor benchmark-owned Top-K and answer cutoffs.
• Store ingestion/indexing state once per conversation build, ingest each conversation's dated sessions in deterministic order, and support durable retry/resume without duplicating build state across questions.
• Separate one-time build metrics from per-question query metrics and persist protocol/model/dataset identity through reports, APIs, leaderboards, comparisons, and the UI.
• Add regression coverage and CI for dataset integrity, shared-build ownership, resume behavior, Top-K, provider normalization, scoring, reporting, and malformed-input failures.

Scope remains explicitly BEAM 1M/10M.

Validation

• bun test — 222 tests passed, 851 assertions
• bunx tsc --noEmit
• bunx tsc --noEmit -p ui/tsconfig.json
• bun run format:check
• git diff --check

No benchmark run data or environment files are included.

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	hyparquet-compressors@​1.1.1
	json5@​2.2.3
	hyparquet@​1.27.1

View full report

[socket-security]

Warning

Review the following alerts detected in dependencies.

According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.

Action	Severity	Alert  (click "▶" to expand/collapse)
Warn		Obfuscated code: npm hysnappy is 90.0% likely obfuscated Confidence: 0.90 Location: Package overview From: ? → npm/hyparquet-compressors@1.1.1 → npm/hysnappy@1.0.0 ℹ Read more on: This package | This alert | What is obfuscated code? Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: Packages should not obfuscate their code. Consider not using packages with obfuscated code. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore npm/hysnappy@1.0.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

View full report