fix: report selector/index signature-grammar mismatch in efficacy#46
Merged
Conversation
A mutation whose trigger selector uses signature grammar v1 (failure/v1|shell|…) evaluated against a database whose event_signatures index was re-minted to grammar v2 (reindex --index-tool-input) reported "insufficient data · 0 → 0 failures · no prior baseline · needs more observed occurrences". The pre-install failures exist — they are indexed under v2 — so the v1 operation key matches zero rows, and the report was misleading rather than honest. Efficacy now detects when a selector's grammar is older than the newest grammar present in the index and the index holds no rows of that older grammar, and emits the insufficient-data reason selector_grammar_mismatch with the text "selector uses signature grammar v1 but the index is v2 — re-propose from current findings to measure efficacy". A v1 selector is never silently translated to v2 (v2 normalization is not derivable from the v1 string). Additive efficacy/0.1 enum variant; existing fixtures unchanged. Recorded as an addendum to ADR 0015. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Claim
autophagy mutations efficacygave a misleading verdict when an installed mutation's trigger selector uses an older signature grammar than the index it is evaluated against.The mutation
mut_b13f2878…'s selector isfailure/v1|shell|cd zuzoto && go build ./... 2>&1|exit:1, but the showcase database'sevent_signaturesindex was re-minted to grammar v2 byreindex --index-tool-input. The v1 operation key (operation/v1|…) then matches zero rows even though the 8 pre-install failures exist — they are simply indexed under v2. The report read0 → 0 failures · no prior baseline · needs more observed occurrences, as if the failures never happened.The honest statement is that the selector's grammar is older than the index's and the measurement cannot see those events. A v1 selector is not silently translated to v2 — the v2 normalization is not derivable from the v1 string.
Change
insufficient_reasonsvariantselector_grammar_mismatch(efficacy result v0.1, additive enum member). It fires when a selector's grammar version is older than the newest grammar present in the index and the index holds no rows of that older grammar. A partial/skipped reindex that still carries the old grammar does not trip it (the selector can still match those rows).autophagy-efficacy:EfficacyObservationsgainsindex_grammar_versions;evaluateparses each selector's grammar version and compares.autophagy-store: new read-onlyStore::signature_grammar_versions()(no migration).selector uses signature grammar v1 but the index is v2 — re-propose from current findings to measure efficacy, suppressing the consequential "needs …" reasons.result.schema.jsonenum value, a realvalid/selector_grammar_mismatch.jsonfixture (generated from the reindexed showcase DB), spec README, and an addendum to ADR 0015 justifying why an additive enum member is not a version bump.Evidence
Real-data verification on the reindexed showcase database (
AUTOPHAGY_CONFIG_DIR=$(mktemp -d)),--now 2026-11-16T00:00:00Z. The index holds only v2 signatures (34,911 rows, zero v1); the installed mutation's selector is v1.Before
After
insufficient_reasonsin the JSON report is now["selector_grammar_mismatch", "sparse_occurrences"].New unit tests: v1 selector + v2 index → mismatch reason; matching v2 grammars unaffected; an old selector still present in a mixed index is not flagged; store grammar-version enumeration.
mise run checkis fully green.Privacy
None. The report continues to cite exact
event_ids and counts only — never command text or event content. The new store query reads grammar-version tokens from the existing redaction-approvedevent_signaturesprojection; nothing new is persisted and nothing leaves the machine.🤖 Generated with Claude Code