Skip to content

fix(ocap-kernel): quarantine unrestorable vats on kernel boot#977

Open
grypez wants to merge 2 commits into
mainfrom
grypez/issue-964-restore-missing-bundle
Open

fix(ocap-kernel): quarantine unrestorable vats on kernel boot#977
grypez wants to merge 2 commits into
mainfrom
grypez/issue-964-restore-missing-bundle

Conversation

@grypez

@grypez grypez commented Jul 15, 2026

Copy link
Copy Markdown
Member

Explanation

On startup the daemon replays persisted subclusters and restarts their vats, fetching each vat's code from the bundleSpec recorded in the kernel store (the #963 restore path). If a single referenced bundle no longer exists on disk (moved, deleted, or otherwise unresolvable), the fetch throws ENOENT. Because VatManager.initializeAllVats restored every vat inside one Promise.all, that single rejection aborted the entire kernel boot — one orphaned bundle reference made the whole kernel unbootable.

This PR makes restore fault-tolerant. VatManager.initializeAllVats now restores each persisted vat in isolation. A vat whose launch rejects is quarantined — skipped, with a structured warning naming the vat, its subcluster, and its bundle spec — and the rest of the kernel boots normally. The persisted record is retained, so the vat is restored automatically on a later boot if its bundle returns (no silent deletion of persisted state).

Scope

The linked issue (#964) also called for making the daemon's fatal exit observable (the real error, rendered to daemon.log, instead of [object Object] on a discarded stderr). That observability work landed separately in #966, which installs file-scoped fatal handlers in daemon-entry.ts. This PR contains only the @metamask/ocap-kernel correctness fix and is rebased on main (now including #966).

With this fix, a missing bundle no longer aborts makeKernel, so the daemon reaches its normal init path and boots successfully rather than leaving a wedged process.

Test plan

Closes #964

🤖 Generated with Claude Code


Note

Medium Risk
Touches the kernel startup path in VatManager.initializeAllVats, but only adds per-vat error isolation on restore; explicit vat launch still fails fast.

Overview
Kernel boot no longer fails when one persisted vat cannot be restored (e.g. missing bundleSpec / ENOENT). VatManager.initializeAllVats now restores each vat through private #restoreVat, which catches launch failures, logs a structured warning (vat id, subcluster, source via new describeVatSource), and skips that vat while other restores still complete.

Persisted vat records are not removed on quarantine, so a vat can come back on a later boot if its bundle is available again. launchVat / runVat behavior for explicit launches is unchanged—only the bulk restore path is fault-tolerant.

Tests cover mixed healthy/unhealthy restores and warning content; CHANGELOG documents the fix.

Reviewed by Cursor Bugbot for commit d65f902. Bugbot is set up for automated code reviews on this repo. Configure here.

grypez added a commit that referenced this pull request Jul 15, 2026
#977)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 71.31%
⬇️ -0.02%
8880 / 12451
🔵 Statements 71.14%
⬇️ -0.01%
9030 / 12692
🔵 Functions 72.48%
⬆️ +0.02%
2142 / 2955
🔵 Branches 64.78%
⬇️ -0.06%
3588 / 5538
File Coverage
File Stmts Branches Functions Lines Uncovered Lines
Changed Files
packages/ocap-kernel/src/vats/VatManager.ts 94.44%
⬇️ -5.56%
77.77%
⬇️ -22.23%
100%
🟰 ±0%
94.44%
⬇️ -5.56%
400-406
Generated in workflow #4544 for commit d65f902 by the Vitest Coverage Report Action

@grypez
grypez force-pushed the grypez/issue-964-restore-missing-bundle branch from 01d3c8f to 78f951d Compare July 15, 2026 16:48
@grypez
grypez changed the base branch from main to chip/daemon-entry-fatal-logging July 15, 2026 16:49
@grypez grypez changed the title fix(ocap-kernel,kernel-cli): tolerate missing vat bundles on daemon restore fix(ocap-kernel): quarantine unrestorable vats on kernel boot Jul 15, 2026
Base automatically changed from chip/daemon-entry-fatal-logging to main July 15, 2026 18:12
@grypez
grypez force-pushed the grypez/issue-964-restore-missing-bundle branch from 78f951d to 21b3816 Compare July 15, 2026 19:02
grypez and others added 2 commits July 16, 2026 10:53
#964)

Failing tests for VatManager.initializeAllVats: when a single persisted
vat's launch rejects (e.g. its bundle is gone), the current Promise.all
aborts the entire restore instead of quarantining the unrestorable vat and
restoring the rest.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
VatManager.initializeAllVats now restores each persisted vat in isolation.
A vat whose bundle is missing or unresolvable (fetch rejects with e.g.
ENOENT) is quarantined — skipped with a structured warning naming the vat,
its subcluster, and its bundle — and the rest of the kernel boots. The
persisted record is retained, so the vat is restored automatically on a
later boot if its bundle returns. Previously one orphaned bundle reference
made the whole kernel unbootable.

Daemon-side observability of fatal exits (rendering the real error to
daemon.log under `stdio: 'ignore'`) is handled separately by #966.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@grypez
grypez force-pushed the grypez/issue-964-restore-missing-bundle branch from 21b3816 to d65f902 Compare July 16, 2026 14:53
@grypez
grypez marked this pull request as ready for review July 16, 2026 15:46
@grypez
grypez requested a review from a team as a code owner July 16, 2026 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Daemon startup aborts entirely when a persisted subcluster references a missing vat bundle

1 participant