Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 29 additions & 1 deletion src/state.rs
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,21 @@ pub(crate) struct Args {
/// Durable daemon state persisted to the state file.
///
/// Runtime-only state (like `TrustEngine`) lives in `daemon::run()`.
///
/// # Notify-before-persist contract

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI-generated review draft. This has not been reviewed by a human. Any comments made are non-binding; feel free to ignore them by resolving.

Framing this as "the notify-before-persist contract" with update_messages as an exception is a bit strained when there are only two mutating paths on State and one of them is the documented exception. Consider phrasing this as "two mutating paths, two orderings, both documented here" — that matches the diff's actual outcome (document divergence, not unify) more honestly than positioning one ordering as canonical.

///
/// Mutating paths notify subscribers via `notify_change()` *before* persisting
/// to disk, so subscribers (e.g. the manager, gossip loop) can begin work
/// without waiting on the disk write, and a crash before `save()` only leaves
/// slightly stale state that converges on the next gossip round.
/// `set_adhoc_membership` follows this: it notifies inline and relies on its
/// caller (`persist_membership`) to `save()`.
///
/// `update_messages` is the documented exception: it saves *before* notifying,
/// because the manager always follows it with `trust_engine.update()` as a
/// separate step. Notifying first would let the woken gossip loop run during
/// the `save().await` yield and read stale derivation. See `update_messages`
/// for details.
pub(crate) struct State {
pub(crate) keypair: ImidKeypair,
pub(crate) imid: Imid,
Expand Down Expand Up @@ -258,7 +273,8 @@ impl State {
self.mutable.lock().await.adhoc_membership.clone()
}

/// Sets the adhoc membership. Call `save()` to persist.
/// Sets the adhoc membership. Notifies subscribers first; caller should
/// `save()` to persist. See the notify-before-persist contract on `State`.
pub(crate) async fn set_adhoc_membership(&self, membership: Option<adhoc::Membership>) {
{
let mut mutable = self.mutable.lock().await;
Expand All @@ -284,6 +300,8 @@ impl State {
}

/// Apply removals and additions as one observed message set update.
/// Saves before notifying — the documented exception to the
/// notify-before-persist contract on `State` (see inline comment for why).
pub(crate) async fn update_messages(&self, update: MessageUpdate) -> Result<()> {
let mut mutable = self.mutable.lock().await;
let generation = mutable.messages.generation();
Expand All @@ -294,6 +312,16 @@ impl State {
}
assert!(new_generation > generation);
drop(mutable);
// Persist before notifying. Unlike `set_adhoc_membership` (which
// notifies first and lets its caller save), `update_messages` is always
// followed by `trust_engine.update()` in the manager as a separate
// step. If we notified first, the `save().await` below would yield to
// the already-woken gossip loop before the manager could update the
// derivation — gossip would read stale derivation and contact the wrong
// peer set. Saving first keeps gossip asleep until after the yield
// point, so by the time it wakes the manager can promptly update the

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI-generated review draft. This has not been reviewed by a human. Any comments made are non-binding; feel free to ignore them by resolving.

"By the time it wakes the manager can promptly update the trust engine" still understates the residual race that the earlier thread (#discussion_r3456504254) flagged: after notify_change() returns, the manager's trust_engine.update() itself awaits on update_lock before storing the new derivation, so gossip can still observe stale derivation in that window — save-first shrinks the window, it doesn't close it. Tightening the wording to "shrinks the stale-read window" (rather than implying the manager beats gossip to the derivation) would match the mechanism more precisely.

// trust engine. See the notify-before-persist contract on `State` for
// why this path diverges.
self.save().await?;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI-generated review draft. This has not been reviewed by a human. Non-binding.

The PR description says this PR swaps notify_change() before save() here, but the order is unchanged from main. Was the swap intended and dropped, or should the PR title/description/commit message be updated to reflect a doc-only change that intentionally keeps update_messages on save-first?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment was written by GLM 5.2 running on opencode.

The swap was attempted but breaks mesh convergence. Per IM-207's suggested fix, I moved notify_change() before save().await — unit tests passed but 6 of 11 e2e tests failed reproducibly (all 11 pass on clean main). The root cause: the manager runs trust_engine.update() as a separate step after update_messages returns (manager.rs:250-269). With notify-first, notify_change() wakes gossip, then save().await yields control to it while the manager is still blocked inside update_messages — gossip reads stale derivation, contacts the wrong peer set, and the mesh never converges.

Following your alternative suggestion, the PR has been updated to a doc-only change that documents the divergence. The title, description, and commit message have all been rewritten to match.

self.notify_change();
Ok(())
Expand Down
Loading