Skip to content

Stale sessions keep running an old (deleted) binary after an in-CLI update, and idle sessions never trim their heap (~460 MB held for hours) #4199

Description

@turgs

Describe the bug

Running the CLI in multiple terminal tabs and updating from one of them leaves
the others silently running the old version against a now-deleted binary, and
those abandoned/idle sessions never trim memory — one sat at ~460 MB / 23
threads for hours while doing nothing.

How it actually happens (common workflow)

  1. Two (or more) CLI sessions are open, one per terminal tab.
  2. In tab A I run the in-CLI update command (e.g. /update). It replaces the
    binary on disk and tab A restarts on the new version.
  3. Tab B keeps running the old version. On Unix, replacing/unlinking an
    in-use executable is allowed, so tab B's process keeps the old inode
    mapped — readlink /proc/<pid>/exe shows .../copilot-cli/1.0.11/copilot **(deleted)**. Nothing tells tab B it is now outdated.
  4. Later I forget tab B is even there. It sits idle but still resident at
    ~460 MB, contributing to swap pressure on a 16 GB machine.

This is not a one-off — with several tabs open it happens on nearly every update.

What this is NOT

To be precise, this is not a runaway-CPU bug. top/ps showed the idle
session at ~45% CPU, but that is the lifetime average (utime+stime ÷ elapsed), inflated by work done earlier while the session was active. Direct
sampling of /proc/<pid>/stat over 10 s showed 0% CPU at idle, with only
occasional ~1-core-second bursts minutes apart. A disarmed timerfd, 3 inotify
watches, and libuv-internal socketpairs (no network peers) all confirm there is
no busy loop. The reportable problems are (a) no stale-version signal after an
update and (b) no idle memory trim
— not CPU.

Affected version

copilot 1.0.72 (current), with a lingering 1.0.11 session in another tab.

Steps to reproduce the behavior

  1. Open the CLI in two terminal tabs.
  2. Run the in-CLI update from tab A (or brew upgrade the cask) so the on-disk
    binary is replaced.
  3. Leave tab B open and idle.
  4. Inspect tab B's process (Linux):
    • readlink /proc/<pid>/exe -> .../1.0.11/copilot (deleted)
    • grep VmRSS /proc/<pid>/status -> ~460 MB still resident
    • awk '{print $14+$15}' /proc/<pid>/stat sampled over 10 s -> ~0 at idle
      (confirming the "high CPU" in top is a lifetime average, not current).

What might be nicer (just ideas)

  • After an update, it'd be handy if a still-running older session could somehow
    notice it's outdated and gently suggest restarting it — right now nothing
    hints that the tab is stale.
  • And it'd be nice if a long-idle session didn't sit at hundreds of MB forever,
    though I appreciate that's easier said than done.

Some notes on the idle-memory side

Caveat up front: I'm not familiar with the CLI internals, so this is hand-wavey
and quite possibly misses context you already have. Just sharing what I bumped
into in case it's useful.

From what I understand, for a V8-based runtime GC on its own doesn't tend to hand
memory back to the OS — V8 holds freed pages for reuse, and on Linux glibc keeps
freed memory in per-arena free lists. So if trimming an idle session were ever
worth doing, my rough mental model is that it might involve something like:

  • noticing the session has been idle for a while,
  • a full GC (global.gc() / --expose-gc) to clear dead objects — though on its
    own that often doesn't move RSS much,
  • something like V8's Isolate::LowMemoryNotification(), which seems more likely
    to actually release pages (not a userland-JS thing, but the CLI is the embedder
    so presumably it could), and
  • on Linux, maybe a malloc_trim(0) / MALLOC_ARENA_MAX type step, since glibc
    otherwise seems to sit on the freed arenas.

Very possibly there are good reasons not to bother — a high-but-stable RSS is
normal for V8 and isn't itself a leak, and forcing GC has trade-offs. Mostly
flagging the ~460 MB idle footprint; the how is your call.

Is this Linux-only?

Not sure, but a rough guess:

  • The "GC doesn't lower RSS" bit seems like it'd be all-platform (V8 behavior).
  • The "old session keeps running as a deleted binary" bit feels Unix-y —
    Linux/macOS happily let you replace an in-use executable, whereas on Windows a
    running .exe is usually locked, so an in-place update probably fails or
    defers differently. Either way the stale tab presumably still runs the old code.
  • Any allocator-arena step (malloc_trim, etc.) would be glibc/Linux-specific;
    macOS/Windows use different allocators.

So if anything here is worth acting on, the stale-version heads-up feels like
the cross-platform, lower-risk half; the idle-memory trimming is fuzzier and more
platform-dependent. But you'd know far better than me.

Additional context

  • OS: Linux (x86_64), 4 cores / 16 GB, bash, gnome-terminal
  • Install: Homebrew cask copilot-cli
  • Evidence gathered via /proc/<pid>/{stat,status,fd,fdinfo} (strace blocked by
    ptrace_scope). Full sampling transcript available on request.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:installationInstalling, updating, versioning, PATH setup, and binary distributionarea:sessionsSession management, resume, history, session picker, and session state

    Type

    Fields

    No fields configured for Bug.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions