Skip to content

VALIDATION.md

Latest commit

 

History

History
66 lines (56 loc) · 3.12 KB

VALIDATION.md

File metadata and controls

66 lines (56 loc) · 3.12 KB
Raw

Validation record: fable-session 0.3.0b1

This is the compact, privacy-safe evidence behind the README's "Why trust it?" section: what was actually verified for 0.3.0b1, how, and — just as important — what it does not show. It deliberately contains no hostnames, usernames, filesystem paths, session ids, credentials, prompt text, or transcript text.

Deterministic release gates

These run offline on every change, locally and in protected-main GitHub CI, and must all pass:

  • python3 -m compileall over sources and tests;
  • the full unit test suite (500+ tests, standard-library unittest; no network, no real tmux sessions, no Claude API);
  • the public-readiness scan of the tracked tree (naming, notices, internal identifiers, credential-shaped literals);
  • the full-history readiness scan — the same policy applied to every tracked blob of every commit reachable from HEAD;
  • whitespace hygiene checked against the empty tree;
  • an offline install smoke: build the wheel, install it into a disposable venv with no index access, and dry-run the installed CLI from outside the checkout;
  • every GitHub Action pinned to exact commit SHAs, with pinned Python tooling versions.

The one real smoke

  • Recorded: 2026-07-14
  • Version: fable-session 0.3.0b1
  • Environment: a Linux host with Claude Code CLI 2.1.208, Python 3.12, and tmux — the verified combination named in the README.
  • Scope: one bounded session end to end — run --launch, one watch --follow, one audit — driving a minimal marker task.

Observed results:

  • the task was delivered to Claude on stdin from a hash-verified, run-scoped payload file; it appeared on no process command line;
  • the launch accepted and passed through --max-budget-usd 0.5 — chosen purely as a cheap smoke-test fixture, not a normal default (a registry with no budget key emits no budget flag at all);
  • the --follow watcher stayed silent while the session ran, emitted one terminal completed event carrying the bounded usage aggregate, and exited by itself;
  • the session returned the exact expected marker FABLE_BETA1_SMOKE_OK;
  • the model audit reported PURE for actor messages — every observed assistant message was served by claude-fable-5 — while the usage aggregate honestly showed auxiliary Haiku activity (helper-model use is reported and does not by itself break purity);
  • total measured cost of the whole session: $0.15400099999999997.

Limitations — what this record does not show

  • It is one smoke on one host with one CLI version (Linux, Claude Code 2.1.208). No other platform or CLI version is verified, and one run is not a performance benchmark, an adoption signal, or a production-maturity claim.
  • It does not show that pauses are avoided: bounding context removes accidental oversized-context triggers, but fable-session can never guarantee a session is not paused, and a genuine pause still stops the run.
  • No release artifact or attestation exists yet. Checksums and provenance attestation are produced only by a successful run of the tag workflow for v0.3.0-beta.1, which has not run.