feat(cognitive-threads): the ten reflective threads on the shared Mind (#5)#3142
feat(cognitive-threads): the ten reflective threads on the shared Mind (#5)#3142rysweet wants to merge 8 commits into
Conversation
#5) Mature Simard's single mind with ten new cognitive threads — metacognition, consolidation, reflection, prospection, salience, operator_model, analogy, values_deliberation, interoception, and narrative — scheduled by the existing shared `Mind` alongside OODA. Recipes + prompts + thin deterministic rails over new plumbing: nine threads are thin `CognitiveThread` rails over an agentic recipe invoked through one shared seam; interoception is recipe-free deterministic sensing. All ten are OFF by default behind a double env gate and are fully additive. What lands: - Shared brick `recipe_rail`: the `RecipeInvoker` seam (+ production `RecipeRunnerInvoker`), the no-silent-degradation classification (Json / SemanticMiss / InfraFailure — both misses fail the tick with zero writes), and the security helpers `sanitize_value`, `fence_untrusted`, `secret_scrub` (run-based, redacts punctuation-wrapped tokens), and `validate_concept_key`; plus one capacity-checked goal-proposal helper. - `salience_signal`: the numeric-only, id-validated, atomic Decide-facing signal writer and its fail-closed consumer (presence + size + schema + staleness), and the advisory Decide ordering used by the OODA cycle. - Ten `tick` implementations, each writing durable prefixed facts/metrics/ goals; guarded cadence for reflection and values_deliberation. - Nine recipe + prompt assets under prompt_assets/simard/recipes/, each fencing untrusted memory and emitting a strict-JSON envelope. - Registration of the ten threads behind the master gate; a fail-closed, additive salience→Decide reorder between Orient and Decide (no-op when no fresh signal); the global `SIMARD_THREAD_INTERVAL_SCALE` knob. - Durable docs (catalog reference, salience/Decide concept, RecipeInvoker seam reference, batch how-to) and the tests-first `tests_catalog` contract (39 hermetic unit tests + 10 ignored live-smoke checks) plus a secret-scrub regression suite. No memory-architecture changes (consolidation forgetting is advisory/dry-run via existing APIs), so no amplihack-memory-lib pin bump. No --admin/--no-verify. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
93906ae to
d02c554
Compare
… ContextFile (#5) Step 8b (external-service integration): harden the `recipe-runner-rs` seam so a large recall can never crash the spawn or silently truncate. `RecipeRunnerInvoker::invoke` passed every context var inline as `-c k=v`. Nine recipe-backed threads pass **unbounded** memory there — `fence_untrusted( search_facts(...))` (up to 30 facts of arbitrary size) — with no cap. That is the exact class of bug the codebase already fixed twice: the progress-checker caps inline values at 8000 chars (#2127), and unbounded inputs (journal `day_context`, episode distillation) go through `ContextFile` — payload to a private 0700 temp file, only `<key>_path=<abs>` on argv — to dodge `E2BIG`/"Argument list too long" at `execve` (#2640/#2692/#2622/#2619; `MAX_ARG_STRLEN` is 128 KiB/arg). The new invoker did neither, and `sanitize_value` also flattened the fenced region's newlines. Capping would truncate the memory a thread reasons over = silent degradation, violating this feature's core invariant (I4), so use the file channel. What changes: - `recipe_rail`: `UNTRUSTED_OPEN/CLOSE` consts (reused by `fence_untrusted`), `is_fenced_payload`, and a testable `build_context_args` — a fenced payload is written verbatim to a `ContextFile` and only `<key>_path=<abs>` rides on argv (byte-for-byte, no truncation); a small scalar stays inline as `<key>=<sanitize_value>`. `invoke` uses it and holds the guards across the spawn. Three unit tests pin the transport. - The nine recipe assets declare each fenced var as `<key>_path` and have the prompt read the file at `{{<key>_path}}` (mirrors the journal/distill recipes); small scalars stay inline. - Docs: seam reference documents the transport + adds SR-13 (E2BIG safety); the add-a-thread howto tells future authors to use `{{<key>_path}}` for fenced vars. - Cleanup: correct the now-stale `todo!()`/RED "Status" doc-comments across the module (code has been GREEN since d02c554). The `gh` issue seam (engineer_log_analysis) is already length-bounded (title <=80, body <=4000 chars), so it needs no change. cargo fmt ✓ · clippy --all-targets --all-features ✓ · cognitive_threads tests 76 passed / 0 failed / 10 ignored. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
🧭 Illustrated Guide — PR #3142: The ten reflective cognitive threads
1. Problem statementSimard has one "mind" that today runs a single reasoning loop (OODA: Observe → Orient → Decide → Act). It reacts, but it does not reflect: it never checks whether its own confidence was justified, never consolidates what it learned overnight, never rehearses future scenarios, and never steps back to weigh its values before acting. Issue #5 asks for ten reflective cognitive threads that give the existing mind those capabilities — metacognition, memory consolidation, reflection, prospection (foresight), salience (what deserves attention), an operator model (understanding the human it works for), analogy, values deliberation, interoception (sensing its own internal state), and narrative identity. The hard requirements:
2. Approach overviewThe key insight: these are not new identities and not a new scheduler. Each thread is one more flowchart TD
Mind["Shared Mind (existing scheduler)"] --> OODA["OODA thread (existing)"]
Mind --> T["10 new cognitive threads"]
subgraph T["Ten reflective threads (all OFF by default)"]
MC["🪞 metacognition"]
CO["💤 consolidation"]
RE["🔁 reflection"]
PR["🔮 prospection"]
SA["🎚️ salience"]
OM["👤 operator_model"]
AN["🧩 analogy"]
VD["⚖️ values_deliberation"]
IN["❤️ interoception"]
NA["📖 narrative"]
end
MC & CO & RE & PR & SA & OM & AN & VD & NA -->|"one shared seam"| RI["RecipeInvoker → agentic recipe (strict JSON)"]
IN -->|"deterministic, no recipe"| Sense["reads own metrics"]
SA -.->|"advisory, fresh signal only"| Decide["OODA Decide ordering"]
Every thread is guarded by a double env gate and writes only durable, prefixed facts/metrics/goals. With the master gate unset, nothing registers — zero behavioural change. 3. Detailed walkthrough3a. One shared seam so nine threads don't each reinvent "run a recipe"Problem it solves: nine threads all need the same thing — run an agentic recipe, get back structured JSON, and refuse to act on anything that isn't clean JSON. Duplicating that nine times would be nine chances to leak a silent failure.
pub trait RecipeInvoker: Send {
fn invoke(&self, recipe_name: &str, ctx_vars: &[(&str, String)]) -> InvokeResult;
}
pub enum InvokeResult {
Json(Value), // the ONLY case a rail may write from
SemanticMiss { /* … */ }, // recipe ran, output unusable → fail tick, write nothing
InfraFailure { detail: String }, // recipe couldn't run → fail tick, write nothing
}The critical property: only The same brick also carries the security helpers every rail reuses: 3b. Each thread is a thin rail — the pattern, onceProblem it solves: a thread must assemble read-only context, treat stored memory as untrusted, invoke its recipe, and persist results under its own namespace — without leaking write authority or unfenced text.
//! A thin CognitiveThread rail over the agentic recipe `metacognition-appraise`:
//! assemble read-only context in-thread, fence memory-sourced text as untrusted,
//! invoke the recipe through the shared RecipeInvoker, and persist under the
//! declared `metacog:` prefix only. OFF by default behind the double env gate.
pub const RECIPE: &str = "metacognition-appraise";Each thread owns a fact/metric prefix ( 3c. Salience advises Decide — without ever changing what Simard doesProblem it solves: the salience thread decides what deserves attention, but attention must never silently rewrite the mind's actions. It can reorder priorities, not invent or change them — and only when its input is fresh and trustworthy. The producer, // Presence + size guard: an absent or oversized file is treated as absent.
// Schema guard: a torn/mismatched file yields None, not a guess.
// Staleness guard (2 × interval): a stalled writer cannot pin Decide to an old ranking.
pub fn read_valid_signal(/* … */) -> Option<SalienceSignal> { /* … */ }The OODA seam in // Reorder priorities by a FRESH, numeric-only salience signal so Decide
// prefers salient goals. Fail-closed: an absent/stale/malformed signal (the
// default, since salience is OFF) yields NO reordering — behaves exactly as
// before. Salience ADVISES; it never changes an action's kind.
let salience_order = salience_signal::advisory_priority_order(/* … */);
if !salience_order.is_empty() {
// stable sort: salient goals move to the front; everything else preserved
}Because the default (salience OFF) yields an empty order, this branch is a no-op in production — the loop is byte-for-byte the prior behaviour. 3d. Registration behind the double env gateProblem it solves: switching capabilities on must be deliberate and blast-radius-limited; an operator wants to enable one thread at a time.
flowchart LR
A["SIMARD_COGNITIVE_THREADS_ENABLED<br/>(master gate)"] -->|unset| Z["nothing registers · zero behaviour change"]
A -->|set| B["per-thread gate<br/>SIMARD_THREAD_<NAME>_ENABLED"]
B -->|set| C["thread registered on the shared Mind"]
B -->|unset| Z
A global 3e. Recipes + prompts as assets, not codeProblem it solves: the reasoning itself belongs in reviewable, iterable recipe/prompt assets — not compiled Rust. Nine YAML recipes under 4. Key decisions & trade-offs
5. Testing
Generated illustrated guide. Deep-link anchors are best-effort; if an anchor doesn't resolve, use the Files tab. |
🛠️ Crusty-old-engineer review — PR #3142Reviewed the actual code at head Short framingThis PR adds ten reflective threads whose entire premise is that they run in the background, subordinate to the object-level OODA loop. The wiring does the opposite: it runs them inline on the main OODA loop thread, so enabling one stalls the very loop it is supposed to reflect on. 🔴 Finding 1 — BLOCKER: reflective ticks block the object-level OODA loop
The This contradicts three things:
Metacognition lens (as requested). Metacognition is the monitoring and control of one's own cognition — and a healthy metacognitive architecture keeps reflection subordinate to object-level cognition; the monitor must not starve the process it monitors. Encoding the reflective loop so it blocks the primary reasoning loop is a textbook metacognitive-regulation failure: the "thinking about thinking" halts the thinking. Getting the scheduling relationship right (reflection yields to action) is as much a part of "consuming the metacognition concept correctly" as the appraisal logic inside the thread. The appraisal content (calibration: stated confidence vs actual outcome, error/bias signatures, ≤1 recalibration goal) is well aligned with the concept; the control-loop placement is not. Mitigation today: threads are OFF by default, so no production regression right now — but the feature is unusable-as-designed the moment it is switched on, which is the point of shipping it. Fix: run the non-critical scheduler pass on the existing 🟡 Finding 2 — minor:
|
Run cognitive-thread scheduler passes on a background thread with an overlap guard so blocking recipe-backed reflective threads cannot delay the next authoritative OODA cycle. Keep Overseer health snapshots non-blocking with try_lock. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
PR #3142 evaluation plan — metrics, canary install, and merge criteriaThis is not a vibes test. Ten reflective threads are supposed to improve Simard's cognition without degrading the authoritative OODA loop. Measure both halves. If the core loop slows down or starts making worse decisions, the feature failed even if the new thread logs look interesting. 1. What to measureUse the SRE framing: define service-level indicators first, then objectives. For this PR the "service" is Simard's live cognitive loop. A. Non-negotiable safety / no-regression metricsThese decide whether the experiment is allowed to continue.
B. Effectiveness metrics by cognitive functionMetacognition is monitoring + control of cognition, not just a prompt named "metacognition." So measure whether monitoring changes later behavior.
2. Trial designDo not test this by flipping all ten switches on production and admiring the dashboard. That is how slow incidents are born. Phase 0 — install path proofBecause the operator requirement is now that Simard and Crocutus install and run from the installer, the PR trial must use the installer path. If the installer cannot install a specific PR/commit into an isolated state root, that is not a test inconvenience; it is a deployment blocker. Required before trial:
Phase 1 — baseline on mainRun current main with cognitive threads disabled.
This is the control. Without it, we are just comparing PR logs to optimism. Phase 2 — PR #3142 canary, narrow enablementInstall PR #3142 head from the installer into an isolated canary state root copied from a fresh memory backup. Start with the safest, highest-signal subset:
Run for 6 hours. If safety metrics hold, run for another 12–24 hours and add reflection/prospection. Do not enable all recipe-backed threads in the first pass. Phase 3 — comparisonCompare PR canary against main baseline:
Use both quantitative counters and sampled human review. There is no honest fully automated metric for "better cognition" yet. Pretending otherwise would be cargo cult. 3. Merge criteriaMerge only if all are true:
4. Adjust instead of merge if
References
Crusty bottom lineThe PR is promising, but promising is not a merge criterion. Treat it as a canaryable control-loop change. The first thing it must prove is restraint. |
Collapse the cognitive-runtime/overlap-guard condition per clippy while preserving the background-pass behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Crusty re-review after
|
Avoid making the installer contract depend on dashboard-audit's headless browser codegen. The dashboard jobs cover that path; install-real validates the runtime daemon install path with signal enabled. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Resolve install_real conflict by keeping canonical installer contracts and preserving runtime-feature cargo install coverage.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep the install-real cargo-install regression at module scope and preserve the canonical installer contract after merging main.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
📊 Coverage Summary
Coverage data from CI run. Test files matching |
PR #3142 canary result — do not merge as-isI ran the requested installer-path canary on ia2 after resolving conflicts and getting CI green. What was tested
Safety notesThe first canary attempt was aborted and rolled back because an old PR binary tried to open the live v42 store as storage v41 and quarantined/rebuilt an empty store. Main was restored from the original quarantined v42 store and then the PR binary was clean-rebuilt. The clean PR binary was tested against an isolated copied store ( Canary resultThe clean PR canary ran without storage-version mismatch and interoception itself succeeded: But the OODA loop regressed badly: This fails the posted merge criteria: p95 OODA duration is nowhere near +10% of baseline, and action success collapsed to 1/20. The feature is not merge-ready as-is. RollbackRolled back via installer to main VerdictDo not merge. Adjust before another canary. The most obvious next investigation is why enabling the cognitive-thread runtime changed effective OODA fanout to 20 priorities/actions and why action success dropped to 1/20 even with only interoception enabled. |
Summary
Matures Simard's single mind with ten new cognitive threads (issue #5),
scheduled by the existing shared
Mindalongside OODA. These are not newidentities or a new scheduler — each is one more
CognitiveThreadon theexisting brain. Recipes + prompts + thin deterministic rails over new code:
nine threads are thin rails over an agentic recipe invoked through one shared
seam; interoception is recipe-free deterministic sensing. All ten are OFF
by default behind a double env gate and are fully additive.
Threads (issue's canonical list; red-team #5 and attention/executive #6 excluded):
🪞 metacognition · 💤 consolidation · 🔁 reflection · 🔮 prospection ·
🎚️ salience · 👤 operator_model · 🧩 analogy · ⚖️ values_deliberation ·
❤️ interoception · 📖 narrative.
Design → build
The design/plan phase produced durable docs and a tests-first RED contract
(
src/cognitive_threads/tests_catalog.rs, 39 hermetic unit tests + 10 ignoredlive-smoke checks). This PR implements them GREEN plus the recipes, registration,
and the salience→Decide seam. Durable docs only — no point-in-time reports.
docs/reference/cognitive-threads-catalog.mddocs/concepts/salience-and-decide.mddocs/reference/recipe-invoker-seam.mddocs/howto/configure-cognitive-thread-batch.mdWhat lands
recipe_rail: theRecipeInvokerseam + productionRecipeRunnerInvoker; the no-silent-degradation classification (only a cleanJsonresult writes — bothSemanticMissandInfraFailurefail the tickwith zero durable writes); security helpers
sanitize_value,fence_untrusted,secret_scrub(run-based; redacts punctuation-wrappedtokens),
validate_concept_key; one capacity-checked goal-proposal helper.salience_signal: numeric-only, id-validated, atomic Decide-facing signalwriter + fail-closed consumer (presence + size + schema + staleness) + the
advisory Decide ordering.
tickrails, each writing durable prefixed facts/metrics/goals;guarded cadence for reflection and values_deliberation.
prompt_assets/simard/recipes/, eachfencing untrusted memory and emitting a strict-JSON envelope.
additive salience→Decide reorder between Orient and Decide (a no-op when no
fresh signal exists); the global
SIMARD_THREAD_INTERVAL_SCALEknob.Abstraction-gap note
Simard's brain abstraction held:
ThreadKindis pure telemetry, so ten threadsneeded only eight new enum variants + ten thin rails + one shared seam + one
scoped OODA seam. No new "thread framework". No memory-architecture changes
(consolidation forgetting is advisory/dry-run via existing memory APIs), so no
amplihack-memory-lib pin bump was required.
Review / quality
secret_scrubleakingpunctuation-wrapped tokens) fixed with run-based redaction + a regression
test.
cargo fmt --all -- --check✓ ·cargo clippy --all-targets --all-features --locked -- -D warnings✓ (debug + release) · all cognitive_threads tests ✓.--admin, no--no-verify; pre-commit + pre-push gates green.Please leave this PR open for human review. It is intentionally left
unmerged — the final merge decision is the operator's.