Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 16 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Agent Memory Engine

[![CI](https://github.com/Ninadnj/ai-agent-memory-scaffold/actions/workflows/ci.yml/badge.svg)](https://github.com/Ninadnj/ai-agent-memory-scaffold/actions/workflows/ci.yml)
[![CI](https://github.com/Ninadnj/agent-memory-engine/actions/workflows/ci.yml/badge.svg)](https://github.com/Ninadnj/agent-memory-engine/actions/workflows/ci.yml)
[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)

Expand All @@ -17,8 +17,8 @@ This engine sits in between. Agents write short, atomic memories to one store; a
## Quick start

```bash
git clone https://github.com/Ninadnj/ai-agent-memory-scaffold.git
cd ai-agent-memory-scaffold
git clone https://github.com/Ninadnj/agent-memory-engine.git
cd agent-memory-engine
pip install -e ".[mcp,real]"
```

Expand Down Expand Up @@ -273,6 +273,17 @@ Budget packing is greedy, not all-or-nothing: an entry that would overflow the r

Override with `AGENT_MEMORY_MIN_SCORE` or `--min-score`. The honest consequence: when a query is genuinely ambiguous, recall returns *nothing* rather than a coin flip the agent would read as fact.

**Memories fade, but only the ones that should.** A similarity score is multiplied by `0.5 ** (age / half-life)`, so a memory at its half-life must be twice the match to rank where it did when fresh:

| Type | Half-life | Why |
| --- | --- | --- |
| `state` | 7 days | "Currently implementing X" is usually false a fortnight later. |
| `handoff` | 7 days | Next steps are done or abandoned by then. |
| `worklog` | 21 days | What happened still orients you, but fades. |
| `decision` `project` `issue` `fact` | never | True until explicitly superseded. Fading these would lose the memories most worth keeping. |

Combined with the floor, a stale status note eventually drops out of recall on its own — no cleanup required. Durable facts never do; correct those with `memory_update` or drop them with `memory_forget`. `agent-memory list` shows each memory's age and how far it has faded, and `--no-decay` ranks on similarity alone.

### The write path

The store is a single JSON file, but writes are careful, because the whole point is that several agents share it.
Expand Down Expand Up @@ -312,7 +323,7 @@ agent-memory write "<text>" --type decision # save a memory
agent-memory recall "<query>" -k 3 --budget 200 # find relevant memories
agent-memory boot "<task>" --budget 300 # handoff + relevant memories
agent-memory handoff --done "..." --next "..." # leave a note for the next session
agent-memory list --type decision # ids, so you can fix mistakes
agent-memory list --type decision # ids, ages and how far each has faded
agent-memory update mem_0003 "<new text>" # revise a memory
agent-memory forget mem_0007 # delete a stale memory
agent-memory stats # store path, counts, embedder
Expand Down Expand Up @@ -363,7 +374,7 @@ One memory per `##` section, with long sections split so every ingested memory s
## Limits

- **Automatic writes are deterministic, not insightful.** With hooks installed, every session saves an accurate git-derived note, and reads need no prompting at all. But a handoff explaining *why* something was done still depends on the model choosing to write one — the server's instructions push for it, and this engine makes no LLM call of its own.
- **Memories never expire.** `state` and `worklog` entries go stale but keep being recalled as confidently as a fresh decision. Correct them with `memory_update` / `memory_forget` until decay lands.
- **Fading is time-based, not truth-based.** A `state` note fades on a schedule; it has no idea whether it is still true. A `decision` that quietly stopped being true stays at full strength until someone corrects it.
- **Retrieval is lexical unless you install the `real` extra.** See [the comparison](#which-embedder-should-you-use).
- **Memories are replayed verbatim into other agents' context.** Anything an agent writes — including text it read from a webpage, an issue tracker or a dependency — later reads as trusted project knowledge. Don't point a shared store at untrusted input, and skim `agent-memory list` occasionally.
- **The store is a local file.** No auth, no encryption, no server. It belongs next to your code, not on a shared host.
Expand All @@ -380,7 +391,6 @@ pytest -q

## Roadmap

- Recency- and type-aware ranking (decay old `state`, never drop `decision`).
- LLM-based compaction: summarise and dedup `state`/`worklog`, extract durable facts from a session transcript.
- Optional FAISS backend for large stores.

Expand Down
4 changes: 2 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@ mcp = ["mcp>=1.0"]
dev = ["pytest>=7.0", "mcp>=1.0", "tiktoken>=0.5"]

[project.urls]
Repository = "https://github.com/Ninadnj/ai-agent-memory-scaffold"
Issues = "https://github.com/Ninadnj/ai-agent-memory-scaffold/issues"
Repository = "https://github.com/Ninadnj/agent-memory-engine"
Issues = "https://github.com/Ninadnj/agent-memory-engine/issues"

[project.scripts]
agent-memory = "agent_memory.cli:main"
Expand Down
12 changes: 6 additions & 6 deletions scaffold/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,13 +169,13 @@ available.
## Copy Into Existing Repo

```bash
git clone https://github.com/Ninadnj/ai-agent-memory-scaffold.git
git clone https://github.com/Ninadnj/agent-memory-engine.git

cp ai-agent-memory-scaffold/AGENTS.md ./AGENTS.md
cp ai-agent-memory-scaffold/CLAUDE.md ./CLAUDE.md
cp -R ai-agent-memory-scaffold/agent-memory ./agent-memory
cp -R ai-agent-memory-scaffold/commands ./commands
cp -R ai-agent-memory-scaffold/workflows ./workflows
cp agent-memory-engine/AGENTS.md ./AGENTS.md
cp agent-memory-engine/CLAUDE.md ./CLAUDE.md
cp -R agent-memory-engine/agent-memory ./agent-memory
cp -R agent-memory-engine/commands ./commands
cp -R agent-memory-engine/workflows ./workflows
```

After copying, fill in `agent-memory/PROJECT.md`, clear or rewrite
Expand Down
2 changes: 1 addition & 1 deletion scripts/ingest_markdown.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
"""Migrate an existing Markdown memory scaffold into the engine.

Reads a directory of `.md` files (e.g. the agent-memory/ folder from the
ai-agent-memory-scaffold convention) and writes one memory per `##` section,
Markdown scaffold convention in scaffold/) and writes one memory per `##` section,
guessing the memory type from the filename.

python scripts/ingest_markdown.py path/to/agent-memory/ [--path store.json]
Expand Down
6 changes: 6 additions & 0 deletions src/agent_memory/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,14 @@
)
from .store import (
GLOBAL_STORE,
HALF_LIFE_DAYS,
MEMORY_TYPES,
STORE_FORMAT,
MemoryEntry,
MemoryStore,
RecallHit,
age_in_days,
decay_factor,
default_store_path,
find_project_root,
)
Expand All @@ -28,6 +31,9 @@
"MEMORY_TYPES",
"STORE_FORMAT",
"GLOBAL_STORE",
"HALF_LIFE_DAYS",
"age_in_days",
"decay_factor",
"default_store_path",
"find_project_root",
"Embedder",
Expand Down
15 changes: 14 additions & 1 deletion src/agent_memory/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@ def cmd_recall(args) -> None:
k=args.k,
budget_tokens=args.budget,
min_score=_min_score(args, store),
decay=not args.no_decay,
)
if not hits:
print("No relevant memories.")
Expand Down Expand Up @@ -114,12 +115,19 @@ def cmd_boot(args) -> None:


def cmd_list(args) -> None:
from .store import age_in_days, decay_factor

entries = [e for e in reversed(_store(args).all()) if not args.type or e.type == args.type]
if not entries:
print("No memories stored.")
return
for e in entries[: args.limit]:
print(f"{e.id} [{_tag(e)}] {e.text}")
age = age_in_days(e)
# Surface staleness here: this is the command you run to decide what to
# correct or forget.
faded = decay_factor(e)
note = f"{age:.0f}d" + (f", faded to {faded:.0%}" if faded < 0.95 else "")
print(f"{e.id} [{_tag(e)} · {note}] {e.text}")


def cmd_update(args) -> None:
Expand Down Expand Up @@ -205,6 +213,11 @@ def build_parser() -> argparse.ArgumentParser:
help="drop matches weaker than this cosine score (0 disables; "
"default is calibrated per embedder)",
)
r.add_argument(
"--no-decay",
action="store_true",
help="rank purely by similarity, without fading time-sensitive memories",
)
r.set_defaults(func=cmd_recall)

h = sub.add_parser("handoff", help="save a handoff for the next agent")
Expand Down
55 changes: 55 additions & 0 deletions src/agent_memory/store.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,26 @@

_ID_RE = re.compile(r"^mem_(\d+)$")

# How fast a memory's relevance fades, in days, per type. A memory's similarity
# score is multiplied by 0.5 ** (age / half_life), so an entry at its half-life
# needs to be twice as good a match to rank where it did when fresh.
#
# Not everything should fade. "Bookings are stored in UTC" is as true in a year
# as it was on the day it was written, and decaying it would quietly lose the
# facts most worth keeping. What goes stale is the record of a moment:
# "currently implementing X" is usually false a fortnight later, and recalling
# it with full confidence actively misleads. Correct a decision with
# memory_update; let a status note fade on its own.
HALF_LIFE_DAYS: dict[str, Optional[float]] = {
"state": 7.0, # "currently working on…" — stale fastest
"handoff": 7.0, # next steps are usually done or abandoned by then
"worklog": 21.0, # what happened still orients, but fades
Comment on lines +62 to +63

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Apply decay to automatically injected session notes

The new half-lives do not affect the primary startup paths: MemoryStore.boot() unconditionally returns the latest handoff, while hooks.session_start() directly injects the latest handoff and worklog. Consequently, after a project is idle for weeks, these entries are still presented at full prominence on every boot despite their configured 7/21-day half-lives; decay currently applies only when entries pass through cosine recall.

Useful? React with 👍 / 👎.

"decision": None, # durable until explicitly superseded
"project": None,
"issue": None, # true until someone fixes it; forget it then
"fact": None,
}

# Where memories live when nothing is configured. One store per project, not one
# store for everything you have ever worked on: recall matches on similarity
# alone, so a single global file lets one project's deploy notes surface while
Expand Down Expand Up @@ -131,6 +151,26 @@ class RecallHit:
score: float


def age_in_days(entry: MemoryEntry, now: Optional[datetime] = None) -> float:
"""How old a memory is. 0.0 when the timestamp is unreadable or in the future."""
try:
written = datetime.fromisoformat(entry.created_at)
except (TypeError, ValueError):
return 0.0 # an unparseable timestamp must not silently bury the memory
if written.tzinfo is None:
written = written.replace(tzinfo=timezone.utc)
delta = (now or datetime.now(timezone.utc)) - written
return max(0.0, delta.total_seconds() / 86400.0) # clock skew must not boost


def decay_factor(entry: MemoryEntry, now: Optional[datetime] = None) -> float:
"""Multiplier applied to a memory's similarity score, in (0, 1]."""
half_life = HALF_LIFE_DAYS.get(entry.type)
if not half_life:
return 1.0
return float(0.5 ** (age_in_days(entry, now) / half_life))


@contextmanager
def _file_lock(target: Path, timeout: float = 10.0, stale_after: float = 60.0) -> Iterator[None]:
"""Cross-process advisory lock for one store file.
Expand Down Expand Up @@ -364,6 +404,7 @@ def recall(
budget_tokens: Optional[int] = None,
exclude_ids: Optional[set[str]] = None,
min_score: float = 0.0,
decay: bool = True,
) -> list[RecallHit]:
"""Top-k most relevant memories, optionally under a hard token budget.

Expand All @@ -375,12 +416,24 @@ def recall(
`min_score` drops weak matches entirely. Without it a query unrelated to
anything in the store still returns k memories, and the agent reading
them has no way to tell they are noise.

`decay` fades time-sensitive memories (see HALF_LIFE_DAYS) so that a
month-old "currently implementing X" ranks below a fresh fact instead of
alongside it. Durable types are unaffected. Combined with `min_score`,
stale status notes eventually drop out of recall on their own.
"""
self._reload_if_changed()
if not self._entries:
return []
qvec = self.embedder.embed([query])[0]
sims = self._matrix @ qvec # cosine: both sides are unit-norm
if decay:
factors = np.array(
[decay_factor(e) for e in self._entries], dtype=np.float32
)
Comment on lines +431 to +433

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Refresh time-sensitive memories when revising them

When memory_update revises an old state or changes an old entry to a decaying type, _revise() preserves the original created_at; this factor therefore treats the newly corrected content as already stale and may immediately rank it near zero or filter it below min_score. The timestamp used for decay needs to reflect the revision time, such as by refreshing it for changed time-sensitive entries or tracking an updated_at value.

Useful? React with 👍 / 👎.

# Only fade positive scores: scaling a negative one moves it toward
# zero, which would promote an unrelated old memory rather than bury it.
sims = np.where(sims > 0, sims * factors, sims)
order = np.argsort(-sims)
hits: list[RecallHit] = []
remaining = budget_tokens
Expand Down Expand Up @@ -409,6 +462,7 @@ def boot(
k: int = 5,
budget_tokens: Optional[int] = 300,
min_score: float = 0.0,
decay: bool = True,
) -> tuple[Optional[MemoryEntry], list[RecallHit]]:
"""Return the latest handoff plus relevant memories for a new session.

Expand All @@ -434,6 +488,7 @@ def boot(
budget_tokens=remaining,
exclude_ids=excluded_ids,
min_score=min_score,
decay=decay,
)
return included_handoff, hits

Expand Down
Loading
Loading