port upstream #7875: downstream origin marker to stop silent pull drops - #21
port upstream #7875: downstream origin marker to stop silent pull drops#21artemlitch wants to merge 2 commits into
Conversation
…se conflict skip The downstream skips a pulled document when the fork state differs from the assumed master state, on the assumption that a local write is waiting for the upstream to resolve it. A lost meta write or a crash between the fork write and the meta write produces the same difference without any local write, so the document is skipped on every later pull while the checkpoint keeps advancing. The downstream now records the origin of its own fork writes in _meta.o (identifier hash plus the revision height it is about to write). When fork and assumed master differ but the marker matches the current fork revision, the difference came from the downstream itself, so the skip is bypassed and the pulled document is applied. A local write bumps the revision height and voids the marker, so real local writes are still protected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Follow-up on the "one wedge class remains open" paragraph: the class is real at the protocol level but turns out to be unreachable for Bookwise tracked_books, proven by a vitest repro against this branch (readwiseio/rekindled@c7deac0c4c, The reason is an accidental shape mismatch in the app's sync config. The push modifier sends wire-shaped docs (booleans, The rekindled test is written as a tripwire: if the push shapes are ever aligned, it fails and the wedge class opens up. The unported conflicts.ts half of upstream pubkey#7875 is effectively a no-op for Bookwise as long as that mismatch stands. |
Problem
When a pulled document's fork state differs from its assumed-master snapshot, the downstream assumes a local write is pending and skips applying the pull. The checkpoint still advances. If the divergence was actually caused by a crash between the fork write and the meta write (or a lost meta write), no push ever comes: the document is wedged, and every future server update for it is dropped silently. This is upstream bug pubkey#7804, fixed in 17.0.0 by pubkey#7875. Our fork is 15.27.0, so we port the fix.
Fix
When the downstream writes a pulled document into the fork, it stamps
_meta.o = { _rev, hash }in the same atomic write: the revision height this write produces, plus the replication identifier hash. A later local edit bumps the revision height and voids the marker. The field name and shape match upstream 17.x exactly, so databases stay wire-compatible on this field.At the skip site, a valid marker (height matches the doc's current revision, hash matches this replication) proves the fork copy is untouched server output, so the divergence can only be a lost meta write. The skip is bypassed: the pull applies and the meta repairs. A voided or absent marker keeps today's behavior, so pending local edits stay protected. Upstream's
skipStoringPullMetaclause is dropped; that option does not exist in 15.x.Not ported: the conflicts.ts half
Upstream pubkey#7875 also removes the
isEqualshort-circuit inresolveConflictError, so an equal-conflict still resolves and writes the assumed-master meta. That is expressible in 17.x because the conflict handler has separateisEqual()andresolve()methods. In 15.x the handler is one function whose equal output carries nodocumentData, so there is nothing to write; any port must invent the resolution. The obvious adaptation (substituterealMasterStateon equality) works but costs one extra master write per equal-conflict and breaks the protocol test asserting the master stays at revision1-when both sides insert identical documents. Upstream's own version of that test passes only because their throwing test-handler suppresses all writes, not because the invariant holds.Consequence: one wedge class remains open in this fork. A document present with identical content on both sides before replication starts never gets an assumed-master row: the downstream skips it (no assumed master), the push classifies it as a conflict, the handler reports equality, and no meta is ever written. The marker cannot help because the fork copy was never written by the downstream. This class needs a follow-up (15.x-native fix or the adaptation with the protocol test updated), tracked with the retroactive-repair work.
Verification
'FirstUpdate'served instead of'SecondUpdate') and passes with the port; test 2 is a guard for the still-skipped case and passes on both.bail: trueconfig hid it).One shared-with-upstream caveat: with
keepMetaand a master-supplied_rev, the predicted marker height can be wrong; the marker then never matches and behavior degrades to the pre-port skip. Fails safe; we do not usekeepMeta.Second commit is the regenerated dist (consumers install prebuilt output from GitHub).