Skip to content

Keep no catalog on the box, and let cached icons expire - #420

Merged
onel merged 2 commits into
devfrom
catalog/no-store-copy
Aug 17, 2026
Merged

Keep no catalog on the box, and let cached icons expire#420
onel merged 2 commits into
devfrom
catalog/no-store-copy

Conversation

@onel

@onel onel commented Aug 17, 2026

Copy link
Copy Markdown
Collaborator

What & why

The brain no longer writes the catalog snapshot to disk. It holds one in memory, re-fetches it, and shows an empty store when it cannot reach the endpoint. Browsing a stale catalog was never the same as installing from it, since installing pulls images over the same network, and a box pinned to an old copy can offer a manifest the store no longer publishes.

  • remote.go: loadCache / cachePath / cacheFileName and the write-through in syncOnce are gone. RemoteOptions.CacheDir became AssetCacheDir, which is now what it says: proxied icons and screenshots only.
  • New MALMO_CATALOG_FILE seeds the store from a local snapshot for the dev and test lanes that have no control plane (make dev-app, both QEMU bootstraps). Read once, never written back; unset on a real box.
  • Assets keep their disk cache but expire after 24h. Filenames are stable per app, so without a TTL the first icon a box fetched was the icon it served forever. An expired asset whose refetch fails is served stale rather than broken.
  • testdata/snapshot.json is now synthetic: three hand-written fake apps covering the full key surface, replacing a copy of the published catalog. App artifacts are authored in malmoos/store and reach a box only through the published snapshot. A -update flag re-stamps the fixture digest after a shape edit.

Spec(s) touched

DECISIONS.md 2026-08-17 (flips part of 2026-07-02), APP_STORE.md, architecture.md, NEXT.md, plus CLAUDE.md, contributing.md, running-locally.md, authoring-apps-with-an-agent.md and progress/catalog-no-box-side-copy.md.

What was tested

make test-nopam, fmt-check, vet, openapi-check green locally. On CI, CI / Go and CI / Cloud image are both green on the branch — the cloud lane boots a real VM, so it exercises MALMO_CATALOG_FILE on a booted box. New unit tests: TestRemoteKeepsNoSnapshotOnDisk, TestRemoteSnapshotFileSeed, TestRemoteSnapshotFileInvalid, TestRemoteAssetExpires, TestRemoteExpiredAssetSurvivesFailedRefresh.

Known gaps & deviations

  • TestNoUnmodeledFields can no longer detect that the published shape moved, because both sides of the comparison now live in this repo. Its doc comment and APP_STORE.md say so; that check belongs on the publish side.
  • A box that cannot reach the catalog endpoint now shows an empty store instead of a last-good copy. That is the intended behavior change, not a regression.
  • Two tests are deleted because the behavior they covered is gone: TestRemoteLastGoodFallback (the on-disk last-good cache) and TestVerifyRealSnapshot (the real published snapshot fixture, now synthetic).
  • No issue tracked this work, so the Closes line below has no number yet.

Definition of done

  • Behavior works in the inner loop (make dev), and integration-tested against the real system if it touches one.
  • Tests added at the right layer; make check green (frontend changes: make check-web too).
  • Two tests are deleted on purpose (see Known gaps): TestRemoteLastGoodFallback tested the on-disk fallback this PR removes, and TestVerifyRealSnapshot verified a copy of the published catalog that no longer lives here. No test is newly skipped, and no new test is environment-gated.
  • Progress entry written (docs/progress/catalog-no-box-side-copy.md); indexes in both READMEs updated; Up-next re-ordered if needed.
  • Catalog change: if this touches the box's catalog wire (internal/catalog/wire.go) or the store views, said whether malmo's other store surface needs the same change (docs/specs/APP_STORE.md # What the box models, and what it drops).
  • Catalog change, per-app field: no new per-app field is added on the wire, so no deployed box is affected.
  • Spec doc updated if behavior realized/diverged; DECISIONS.md entry if a locked decision flipped.
  • No section-sign symbol (write # instead), no hard-wrapped markdown, log/slog only, conventions per CLAUDE.md.
  • Branch off dev, PR into dev.

Closes #

The brain no longer writes the catalog snapshot to disk. It holds one in
memory, re-fetches it, and shows an empty store when it cannot reach the
endpoint. Browsing a stale catalog was never the same as installing from
it, since installing pulls images over the same network, and a box pinned
to an old copy can offer a manifest the store no longer publishes.

- remote.go: loadCache/cachePath/cacheFileName and the write-through in
  syncOnce are gone. RemoteOptions.CacheDir became AssetCacheDir, which
  is now what it says: proxied icons and screenshots only.
- New MALMO_CATALOG_FILE seeds the store from a local snapshot for the
  dev and test lanes that have no control plane (make dev-app, both QEMU
  bootstraps). Read once, never written back; unset on a real box.
- Assets keep their disk cache but expire after 24h. Filenames are stable
  per app, so without a TTL the first icon a box fetched was the icon it
  served forever. An expired asset whose refetch fails is served stale
  rather than broken.
- testdata/snapshot.json is now synthetic: three hand-written fake apps
  covering the full key surface, replacing a copy of the published
  catalog. App artifacts are authored in malmoos/store and reach a box
  only through the published snapshot. A -update flag re-stamps the
  fixture digest after a shape edit.

TestNoUnmodeledFields can no longer detect that the published shape
moved, since both sides of the comparison now live here. Its doc comment
and APP_STORE.md say so; that check belongs on the publish side.

Docs: DECISIONS.md 2026-08-17 (flips part of 2026-07-02), APP_STORE.md,
architecture.md, NEXT.md, CLAUDE.md, contributing.md, running-locally.md,
authoring-apps-with-an-agent.md, progress/catalog-no-box-side-copy.md.
@greptile-apps

greptile-apps Bot commented Aug 17, 2026

Copy link
Copy Markdown

Confidence Score: 5/5

The PR appears safe to merge because no blocking failure remains in the eligible follow-up scope.

No blocking failure remains.

Important Files Changed

Filename Overview
internal/catalog/remote.go Removes persistent snapshot caching, adds optional read-only seed loading, preserves last-good in-memory state across refresh failures, and introduces expiring asset caching with stale fallback.
cmd/brain/main.go Renames catalog cache configuration around asset-only storage and wires the optional snapshot seed into the remote catalog.
internal/hostagent/brainlaunch/brainlaunch.go Conditionally forwards the local catalog seed path into the brain container while retaining the same-path data-directory mount.
internal/catalog/remote_test.go Covers seed loading, empty startup, refresh preservation, asset expiration, stale fallback, and concurrent fetch collapsing.
dev/test-qemu/bootstrap.sh Replaces the air-gapped lane’s writable catalog cache seed with a read-only snapshot staged under the brain’s mounted data directory.
dev/cloud/test/bootstrap.sh Updates the hosted boot-proof lane to stage and forward a read-only catalog snapshot for its offline app-install scenario.
internal/catalog/wire.go Updates fixture-related wire validation support while retaining snapshot schema and digest verification.

Reviews (2): Last reviewed commit: "Say which lanes actually seed a snapshot" | Re-trigger Greptile

@onel
onel merged commit 2f89952 into dev Aug 17, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant