Migrate the gateway to a single global daemon - #3
Merged
Conversation
The gateway ran one process per repo, keeping its durable inbox and singleton lock under <repo>/.memcode. That is wrong on two counts: channel bot tokens are single-consumer (Telegram getUpdates, Slack socket, Discord gateway allow one connection per token), so two per-repo daemons on the same token steal each other's messages; and gateway operational state is machine-global, not project state. Move the gateway's OWN state to the global config dir (~/.config/memcode): - durable inbox + singleton lock -> gwconfig.Dir() (was <root>/.memcode) - event log -> a global gateway-events.db (was the project's event store) The singleton lock being global means a second 'memcode gateway' from any repo is now refused, so one daemon owns the shared tokens. The coding engine, the CLI, and jobs.go are untouched: jobs still spawn against the default project root and their artifacts still live in that project's .memcode. Only gateway-operational ownership moved, classified by who owns the information.
A remote message must never turn into execution against an arbitrary filesystem
path. Introduce a registry of projects the gateway may run in:
- config: Project{path, enabled} + projects map + default_project.
- ResolveProject(id): only a registered + ENABLED project resolves, and the
returned root is the path's CANONICAL form (symlinks/.. resolved at use
time) so the resolved dir is the execution authority; registration cannot
be tricked by a later symlink swap into running elsewhere.
- "memcode project add <path>" and "memcode project list".
- the gateway executes against default_project's canonical root, falling back
to the current repo when none is registered.
Registration (is this path runnable at all) is deliberately distinct from
authorization (may this principal run against it); the initial trust model is
that every allow-listed principal may run against every enabled project, with a
per-principal policy left as a later primitive. The coding-engine interface is
still untouched: jobs.Spawn just receives an authorized root.
…ge 3)
Introduce durable personas (internally Persona; product term "agent") and the
one additive change to the coding engine: it can receive caller-supplied
supplemental context, and knows nothing of where it came from.
Engine (internal/agent/runtime):
- ContextItem{Kind, Content, Source} with GENERIC kinds (instruction, memory,
reference, history) — never orchestration concepts. Session.SetContext lets a
caller supply items; injected every turn after project/user context, in a
FIXED Kind precedence (deterministic, channel-independent). Empty context =>
no block => byte-for-byte the CLI's behavior.
- Tests: empty/all-blank => no block; deterministic order; and a
dependency-direction test that the engine imports no gateway/channel package.
Gateway (the layer above): personaContext composes a bound persona's own
MEMCODE.md/memory.md (from ~/.memcode/agents/<id>) into ContextItems. User-global
and project tiers are left to the engine, not duplicated. The composed context is
written to a global, session-keyed file; the spawned child self-discovers it by
session id, so jobs.Spawn stays unchanged. channels.<name>.agent binds a channel
to a persona.
The coding CLI never sets --session, so it never loads a context file: its runs
are unchanged, as required.
A conversation now remembers which persona and project it is pointed at, and a
task snapshots that selection at receipt so it is immutable for that task's life.
- state: a durable conversations table (channel, conversation -> agent,
project) with Conversation/SetConversationAgent/SetConversationProject; and
agent/project snapshot columns on the inbox item.
- Deliver: /agent <name> and /project <name> are control messages (handled
after authorization, never enqueued) that re-point the conversation for its
SUBSEQUENT tasks; an unknown/unregistered id is rejected. A normal message
snapshots the conversation's current selection (or the channel/gateway
defaults) onto the inbox item.
- runJob: resolves the snapshotted project id to its canonical root (registry
is the authority; an id that no longer resolves falls back to the default)
and composes the snapshotted persona's context.
So "/project adrenal" after "fix CI" changes only the next task; the queued
"fix CI" still runs against the project it was received under. /agent answers
"who am I working with", /project answers "what are we working on" — distinct
primitives, and both are chat commands, never the `memcode agent` CLI verb.
…ault Stage 4 resolves each task's project to a canonical root and spawns the job there, but waitForJob still polled r.root (the gateway default). Since a job's bookkeeping lives under <spawn-root>/.memcode/jobs/<id>, any task pointed at a non-default project would have its meta written under the resolved root while the poller looked under the default one, so every such task reported "Lost track of the job" instead of its real result. Poll under the same root the job spawned into. Add a jobs test asserting Get is root-scoped (found under the spawn root, absent under another), which is the invariant that makes the poller's root matter. Found in review (Kimi).
timothyerwin
marked this pull request as ready for review
August 14, 2026 15:49
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Migrate the gateway from one process per repo to a single global daemon, and establish the layered architecture underneath it. One PR, one commit per stage.
Why
Per-repo gateways are broken (channel bot tokens are single-consumer — two per-repo daemons on the same Telegram/Slack/Discord token steal each other's messages) and mis-shaped (memcode has grown past "codebase engineer": agents, chat, cron, builder, datahub; channels/automations are global; a repo is not an agent identity).
Architectural law (held throughout)
Hard invariant (tested):
cd ~/github/memcode && memcodebehaves exactly as today, forever — even if the entire agent/gateway layer were deleted. We refactor gateway ownership, notroot;jobs.gostaying unchanged is the signal the seam is right.Stages (one commit each)
jobs.gountouched. Fixes the shared-token bug. ✅projects:+memcode project add; id → canonicalized root →jobs.Spawn(root); one configured default project. Engine still untouched.Invocation{Root, Prompt, Context[]};Personahomes; user/persona context composed intoContext.currentProject; immutable task snapshot;/agent,/project.Naming law:
memcode agent <task>stays the coding CLI; the durable-persona type is internallyPersona, never the CLI verb.Full plan tracked in the branch. Draft until all stages land.