docs: event-sourced memory architecture notes

docs/design/memory-event-sourcing-notes.md

@@ -0,0 +1,71 @@
+# Memory Architecture Notes — Event-Sourced Memory with Projections
+
+Ported design study from Autohand Code CLI's memory subsystem
+(`src/memory/`, Apache-2.0, Copyright 2025 Autohand AI LLC) mapped onto
+yaad's existing engine. This is a design document, not an implementation.
+
+## What code-cli does
+
+Four files carry the whole design:
+
+| Component | File | Responsibility |
+|---|---|---|
+| Event log | `MemoryEventLog.ts` | Append-only log of raw memory events (add, update, delete, access). Source of truth. |
+| Projection | `MemoryProjection.ts` | Deterministic fold of the event log into the current memory view. Rebuildable at any time; never mutated directly. |
+| Summary tree | `MemorySummaryTree.ts` | Hierarchical summarization: leaf memories are summarized into parent nodes so a bounded prompt can carry unbounded history. |
+| Session extraction | `extractSessionMemories.ts` | End-of-session pass that mines the transcript for durable facts and emits events into the log. |
+| Safety | `MemoryPathSafety.ts` | Path validation for memory file operations. |
+
+## Why it matters
+
+The event-log/projection split gives three properties yaad's current
+direct-write storage does not:
+
+1. **Auditable history** — every memory change is an immutable event with
+   provenance (which session, which tool call). Today `storage/` keeps only
+   latest state.
+2. **Deterministic replay** — the projection is a pure fold; corruption or
+   schema migration becomes "replay from event N" instead of data surgery.
+3. **Cheap undo/branching** — time-travel is reading the log at offset K.
+   This composes naturally with yaad's temporal decay work (`temporal/`,
+   `engine/decay.go`): decay becomes a projection-time view, not a mutation.
+
+## Mapping onto yaad
+
+| code-cli concept | yaad home | Notes |
+|---|---|---|
+| MemoryEventLog | new `events/` package, SQLite table `memory_events(seq INTEGER PRIMARY KEY, ns, type, payload JSON, session_id, ts)` | Append-only; namespace-scoped like the rest of `engine/namespace.go`. |
+| MemoryProjection | fold inside `engine/`; replaces direct fact writes from `ingest/extract.go` | Keep the current graph tables as *the* projection target so MCP tools are unchanged. |
+| MemorySummaryTree | extends `compact/` | Leaf = individual facts; parents = cluster summaries. yaad's Louvain communities (`graph/community.go`) are natural tree groupings. |
+| extractSessionMemories | hook in `ingest/` triggered on session end / compaction boundary | Emits events; extraction quality stays in one place. |
+| MemoryPathSafety | mostly N/A | yaad stores in SQLite, not files; keep the idea as parameterized query validation in `internal/server`. |
+
+## Interaction with existing subsystems
+
+- **Decay/GC** (`temporal/`, `engine/decay.go`): decay should mark events
+  suppressed in the projection rather than deleting rows. Deletion loses
+  the ability to re-rank later; suppression is reversible and auditable.
+- **Dedup** (`dedup/`): dedup moves from write-time gate to
+  projection-time merge — two near-duplicate events can still be folded
+  differently by future projections without losing either input.
+- **Graph engine** (`graph/`): PageRank/community detection run on the
+  projected state exactly as today; nothing upstream changes.
+- **Export** (`exportimport/`): `.yaadpack` gains a new section — the raw
+  event slice — enabling signed team packs to be merged by replay instead
+  of record-level conflict resolution (`conflict/`).
+
+## Adoption path
+
+1. **Shadow mode**: write events alongside current writes; projection runs
+   but is not authoritative. Compare snapshots in tests.
+2. **Cutover**: make ingest emit events only; projection builds state.
+   MCP tool surface unchanged.
+3. **Summaries**: add the summary tree over projected clusters; wire into
+   context assembly for bounded prompts.
+
+Step 1 is low-risk and independently valuable (audit trail).
+
+## References
+
+- Autohand Code CLI `src/memory/*` — https://github.com/autohandai/code-cli
+- yaad `ARCHITECTURE.md` — engine, graph, storage layers this plugs into