feat: add shadow-draft exporter and smooth the mutation pipeline#38
Merged
Conversation
Make the propose → challenge → replay-draft → replay → shadow-draft → shadow → install lifecycle passable without hand-authoring contract JSON. - Add `mutations shadow-draft <ID> --suite <PATH> [--context-events N] [--force]`, mirroring `replay-draft`. Extraction lives in autophagy-shadow's new `extract_observation_draft`, deriving a schema-valid `shadow-suite/0.1` from the mutation's own supporting evidence (drafted `intervention_would_help: true`) and counterexamples (drafted `false`). Observation ids (`shd_…`) are deterministic SHA-256 of inputs — no clock, no randomness — and source event ids are unique across the suite. Read-only: it writes one user-named file and performs no state transition. - Harmonize replay's flag: `--suite` is canonical; `--scenarios` stays as a hidden alias. Shadow gains the same `--suite`/`--observations` pair. - Manifest-free deterministic synthesis: the built-in `deterministic` provider falls back to a built-in reference manifest when no `--manifest`/config path is given. Explicit manifests still win and validate strictly; other providers still require one, and that error now points at a complete example manifest. - Actionable replay error: the unreviewed-scenarios failure now truncates the id list (first 3 + "and N more") and appends the exact remedy plus the suite path to edit. - One-line next-step hints to stderr (text mode only, never JSON stdout) after propose, challenge, replay-draft, replay, shadow-draft, and shadow, using real ids and paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Claim
The mutation lifecycle (propose → challenge → replay-draft → annotate → replay → shadow → install) was impassable for a normal user at the shadow step (no Shadow Suite exporter — you had to hand-author
shadow-suite/0.1JSON from the schema) and needlessly rough elsewhere. This PR makes the whole path passable and self-guiding without weakening any zero-permission, determinism, or evidence-traceability invariant.Four fixes:
mutations shadow-draft— mirrorsreplay-draft:mutations shadow-draft <ID> --suite <PATH> [--context-events N] [--force]. Extraction lives in a newextract_observation_draftinautophagy-shadow(the natural downstream home, mirroring replay'sextract_review_draft). It emits a schema-validshadow-suite/0.1: supporting evidence → observation draftedintervention_would_help: true, counterexamples →false, each with a human-readablenote, exact source event ids, and observed trigger selectors. Observation ids (shd_<hex>) are deterministic SHA-256 of inputs (no clock, no randomness), and source event ids are unique across the suite (the contract requires it). Read-only: it reads the DB and writes the one file you name — no state transition.replaygains canonical--suite; the former--scenariosstays accepted as a hidden alias.shadowgets the same--suite/--observationspair.replay-draft/shadow-draftand their evaluators now refer to one file by one name.deterministicand no--manifest/config path is given, synthesis falls back to a built-in reference manifest instead of erroring. An explicit--manifeststill wins and validates strictly. Other providers still require a manifest, and that error now points at a complete example (docs/specs/synthesis/0.3/manifest/valid/claude_cli.json). Validation is not weakened.edit <suite path> and set counterfactual_outcome to "expected_result" or "contradiction" for each listed scenario.Plus one-line next-step hints to stderr (text mode only, never JSON stdout) after
propose,challenge,replay-draft,replay(on pass),shadow-draft, andshadow(on pass), using real ids and paths.Evidence
New tests:
autophagy-shadow→evidence_extraction_produces_stable_schema_valid_shadow_draft: the draft is schema-valid againstdocs/specs/shadow/0.1/suite.schema.json, deterministic across repeated runs and reversed event order, has both would-help and no-op coverage,shd_ids, notes, globally-unique source event ids, and is a legal input toevaluate.autophagy-cli→deterministic_synthesis_needs_no_manifest:mutations synthesizewith no--manifestreportsprovider=deterministic,manifest_path=builtin://deterministic, registers candidates; a non-deterministic provider without a manifest errors with the example-manifest pointer.autophagy-cli→mutation_pipeline_ergonomics_are_smooth_end_to_end: propose/challenge/replay/shadow next-step hints appear on stderr in text mode and never on JSON stdout;replay --suiteworks;shadow-draftemits schema-valid deterministic JSON, refuses to clobber without--force, overwrites identically with--force;shadow --suitepasses.autophagy-cli→ existingmilestone_demo_…updated: the unreviewed-replay error now asserts the scenario ids, the remedy text, and the suite path; the--scenariosalias is exercised.mise run check— green (ci + docs + fmt + lint + test), 216s, all workspace tests pass.Pipeline walked end-to-end on a demo DB (new ergonomics)
JSON mode confirmed clean:
--output json mutations propose→ stdout parses as JSON with nonext:, stderr is 0 bytes.Privacy implications
shadow-draftwrites exactly one local file at the user-named--suitepath. Its contents (AEP event ids and observed trigger selectors) are already stored locally in the user's database; nothing new is collected, nothing leaves the machine, no model or network is used, and no state transition or permission is granted by drafting. Default path stays local-only and offline-capable.🤖 Generated with Claude Code
https://claude.ai/code/session_015LY7xtercMc3NtLeX2z4YV