Skip to content

examples: make the at-least-once duplicate observable at a sink - #26

Merged
cardmagic merged 2 commits into
mainfrom
feat/at-least-once-demo
Aug 24, 2026
Merged

examples: make the at-least-once duplicate observable at a sink#26
cardmagic merged 2 commits into
mainfrom
feat/at-least-once-demo

Conversation

@cardmagic

Copy link
Copy Markdown
Owner

Closes #23.

a contract clause nobody can observe firing is decoration

This PR makes the at-least-once clause observable, and the documented remedy observable absorbing it.

The demo

examples/at-least-once (run with pnpm run test:at-least-once) stages one actor turn whose effect writes to an external sink file, then runs the crash choreography twice:

  1. Deduplication off: the first effect worker writes to the sink and crashes before acknowledgement (process.exit between the external write and completeEffect). A second worker reclaims the stale effect past the liveness threshold and delivers again. The sink reads 2 — and both deliveries carry the same stable effect id, at attempts 1 and 2, which is exactly the handle the remedy needs.
  2. Deduplication on: the same choreography with a guard on that effect id. The sink reads 1.

In both runs the actor state commits exactly once — the sharpest line of the proof: the state machine keeps its exactly-once story while the outside world sees two, unless the sink holds the key.

Demo output:

{
  "duplicate": { "stateCommits": 1, "sinkDeliveries": 2, "sameEffectId": true, "attempts": [1, 2] },
  "remedy": { "stateCommits": 1, "sinkDeliveries": 1 }
}

Wiring

  • pnpm run test:at-least-once script; CI runs it beside test:recovery in both the quality and floor jobs, so the clause is proven on every push and on the Node floor.
  • docs/correctness.md links the demo from the at-least-once limitation it makes observable.
  • The sink module has unit coverage for both guard modes (applies every delivery with dedup off; applies a replayed effect id once with dedup on; empty-sink read).
  • Demo asserted stable across four consecutive runs; the choreography mirrors the failure-recovery demo's conventions (forked children, short lease and liveness settings, executable assertions).

Implements #23. The delivery contract promises at-least-once and the
docs tell external systems to hold stable idempotency keys, but no
artifact showed the duplicate arriving anywhere with deduplication
deliberately off. A contract clause nobody can observe firing is
decoration; this demo lets anyone watch the clause fire and watch
the documented remedy absorb it.

examples/at-least-once stages one actor turn whose effect writes to
an external sink file. The first effect worker crashes between the
sink write and the acknowledgement; a second worker reclaims the
stale effect after the liveness threshold and delivers again. With
deduplication off the sink reads 2, both deliveries carrying the
same stable effect id at attempts 1 and 2. With a guard on that id
the sink reads 1. The actor state commits exactly once in both runs,
which is the sharpest line of the proof: the state machine kept its
exactly-once story while the outside world saw two.

pnpm run test:at-least-once runs it; CI runs it beside the recovery
demo in both the quality and floor jobs; docs/correctness.md links
it from the at-least-once limitation it makes observable. The sink
module carries unit coverage for both guard modes.
@greptile-apps

greptile-apps Bot commented Aug 24, 2026

Copy link
Copy Markdown

Greptile Summary

This PR adds an executable demonstration of at-least-once effect delivery and sink-side deduplication.

  • Adds a crash-and-recovery example showing one actor-state commit but two external deliveries.
  • Demonstrates deduplication using the stable effect ID.
  • Adds sink unit tests, CI coverage, documentation, and changelog entries.
  • Preserves non-missing sink errors and fails recovery when no effect becomes claimable.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
examples/at-least-once/demo.ts Orchestrates both crash-recovery scenarios and asserts the expected actor-state and sink-delivery counts.
examples/at-least-once/effect-worker.ts Runs one effect attempt, intentionally crashes after the sink write when requested, and rejects recovery when no effect is processed.
examples/at-least-once/sink.ts Implements the demonstration sink, preserves non-ENOENT read failures, and optionally deduplicates stable effect IDs.
test/at-least-once-sink.test.ts Covers duplicate delivery, deduplication, missing files, malformed data, and unreadable sink paths.
.github/workflows/ci.yml Runs the at-least-once demonstration in both the quality and minimum-Node-version jobs.

Sequence Diagram

sequenceDiagram
    participant A as Actor worker
    participant DB as Durable database
    participant W1 as Crashing effect worker
    participant S as External sink
    participant W2 as Recovery effect worker
    A->>DB: Commit actor state and effect
    W1->>DB: Claim effect (attempt 1)
    W1->>S: Record delivery
    W1--xDB: Crash before acknowledgement
    W2->>DB: Reclaim stale effect (attempt 2)
    W2->>S: Record or deduplicate by effect ID
    W2->>DB: Complete effect
Loading

Reviews (2): Last reviewed commit: "fix: stop the demo sink from hiding its ..." | Re-trigger Greptile

Comment thread examples/at-least-once/sink.ts
Comment thread examples/at-least-once/effect-worker.ts
@cardmagic

Copy link
Copy Markdown
Owner Author

Ruby counterpart is open: cardmagic/solid-objects-ruby#51. Same proof against the gem: bundle exec rake at_least_once crashes an effect worker between the sink write and the acknowledgement, and the sink reads 2 with deduplication off and 1 with a guard on context.id. The state commit is 1 in both runs, matching this PR's numbers.

Greptile flagged two ways the at-least-once proof could report success
without proving anything.

readSink turned every read failure into an empty sink, so a damaged or
unreadable file looked identical to a first run. The deduplication phase
would then forget the effect id it needs, append the replay as if it were
the first delivery, and still satisfy the assertion that the sink holds
one delivery. Only ENOENT means an empty sink now; every other error
propagates. Two tests cover it: a truncated JSON file and a path that is
a directory.

The effect worker exited 0 when its 200 claim attempts all came back
empty, and the parent reads that exit status as proof that the delivery
completed and was acknowledged. It now fails when nothing was processed,
so the recovery assertion means what it claims.

The Ruby counterpart already had both properties, so this brings the two
runtimes back to the same shape rather than moving them apart.
@cardmagic

Copy link
Copy Markdown
Owner Author

Both findings were valid and are fixed in af82d86.

Preserve non-missing sink errors. readSink now treats only ENOENT as an empty sink and rethrows everything else. This one had teeth: a damaged sink read as empty made the deduplication phase forget the effect id, append the replay as a first delivery, and still satisfy sinkDeliveries === 1. The proof would have reported success while proving nothing. Two tests cover it, a truncated JSON file and a path that is a directory, and both fail against the old catch-all.

Recovery can exit without processing. The effect worker now throws no effect became claimable when its claim loop comes back empty, so the parent's recoveryExit === 0 assertion means what it says.

Checked the Ruby counterpart in cardmagic/solid-objects-ruby#51 for the same two defects. Neither is present: AtLeastOnceSink.read already rescues only Errno::ENOENT, and effect_worker.rb already raises no effect became claimable. Both properties were incidental there, so that PR now carries the matching tests to make them deliberate.

pnpm test 342 pass, pnpm run check clean, and pnpm run test:at-least-once still prints the same proof.

@cardmagic

Copy link
Copy Markdown
Owner Author

@greptileai review

@cardmagic
cardmagic merged commit ae60f8c into main Aug 24, 2026
27 of 28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

examples: make the at-least-once duplicate observable at a sink with dedup off

1 participant