-
Notifications
You must be signed in to change notification settings - Fork 0
docs(state): document notify-before-persist contract and why update_messages diverges #168
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -80,6 +80,21 @@ pub(crate) struct Args { | |
| /// Durable daemon state persisted to the state file. | ||
| /// | ||
| /// Runtime-only state (like `TrustEngine`) lives in `daemon::run()`. | ||
| /// | ||
| /// # Notify-before-persist contract | ||
| /// | ||
| /// Mutating paths notify subscribers via `notify_change()` *before* persisting | ||
| /// to disk, so subscribers (e.g. the manager, gossip loop) can begin work | ||
| /// without waiting on the disk write, and a crash before `save()` only leaves | ||
| /// slightly stale state that converges on the next gossip round. | ||
| /// `set_adhoc_membership` follows this: it notifies inline and relies on its | ||
| /// caller (`persist_membership`) to `save()`. | ||
| /// | ||
| /// `update_messages` is the documented exception: it saves *before* notifying, | ||
| /// because the manager always follows it with `trust_engine.update()` as a | ||
| /// separate step. Notifying first would let the woken gossip loop run during | ||
| /// the `save().await` yield and read stale derivation. See `update_messages` | ||
| /// for details. | ||
| pub(crate) struct State { | ||
| pub(crate) keypair: ImidKeypair, | ||
| pub(crate) imid: Imid, | ||
|
|
@@ -258,7 +273,8 @@ impl State { | |
| self.mutable.lock().await.adhoc_membership.clone() | ||
| } | ||
|
|
||
| /// Sets the adhoc membership. Call `save()` to persist. | ||
| /// Sets the adhoc membership. Notifies subscribers first; caller should | ||
| /// `save()` to persist. See the notify-before-persist contract on `State`. | ||
| pub(crate) async fn set_adhoc_membership(&self, membership: Option<adhoc::Membership>) { | ||
| { | ||
| let mut mutable = self.mutable.lock().await; | ||
|
|
@@ -284,6 +300,8 @@ impl State { | |
| } | ||
|
|
||
| /// Apply removals and additions as one observed message set update. | ||
| /// Saves before notifying — the documented exception to the | ||
| /// notify-before-persist contract on `State` (see inline comment for why). | ||
| pub(crate) async fn update_messages(&self, update: MessageUpdate) -> Result<()> { | ||
| let mut mutable = self.mutable.lock().await; | ||
| let generation = mutable.messages.generation(); | ||
|
|
@@ -294,6 +312,16 @@ impl State { | |
| } | ||
| assert!(new_generation > generation); | ||
| drop(mutable); | ||
| // Persist before notifying. Unlike `set_adhoc_membership` (which | ||
| // notifies first and lets its caller save), `update_messages` is always | ||
| // followed by `trust_engine.update()` in the manager as a separate | ||
| // step. If we notified first, the `save().await` below would yield to | ||
| // the already-woken gossip loop before the manager could update the | ||
| // derivation — gossip would read stale derivation and contact the wrong | ||
| // peer set. Saving first keeps gossip asleep until after the yield | ||
| // point, so by the time it wakes the manager can promptly update the | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
"By the time it wakes the manager can promptly update the trust engine" still understates the residual race that the earlier thread (#discussion_r3456504254) flagged: after |
||
| // trust engine. See the notify-before-persist contract on `State` for | ||
| // why this path diverges. | ||
| self.save().await?; | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The PR description says this PR swaps
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The swap was attempted but breaks mesh convergence. Per IM-207's suggested fix, I moved Following your alternative suggestion, the PR has been updated to a doc-only change that documents the divergence. The title, description, and commit message have all been rewritten to match. |
||
| self.notify_change(); | ||
| Ok(()) | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Framing this as "the notify-before-persist contract" with
update_messagesas an exception is a bit strained when there are only two mutating paths onStateand one of them is the documented exception. Consider phrasing this as "two mutating paths, two orderings, both documented here" — that matches the diff's actual outcome (document divergence, not unify) more honestly than positioning one ordering as canonical.