Skip to content

fix: retry contended SQLite writes outside deadlines - #10

Merged
cardmagic merged 2 commits into
mainfrom
agent/enqueue-lock-retry-and-benchmarks
Aug 9, 2026
Merged

fix: retry contended SQLite writes outside deadlines#10
cardmagic merged 2 commits into
mainfrom
agent/enqueue-lock-retry-and-benchmarks

Conversation

@cardmagic

Copy link
Copy Markdown
Owner

Closes the three items left open after 0.7.0.

1. The intermittent EnqueueTest failure

with_lock_retry is a no-op unless a SyncDeadline is active, so an asynchronous enqueue had
no Ruby-level retry budget and depended entirely on SQLite's busy handler. Once concurrent
writers exhaust that handler, the caller gets SQLite3::BusyException with nothing to catch it.
This is the failure that forced a re-run on three separate pull requests.

DatabaseAdapters::Sqlite#transaction now retries busy errors outside a deadline, with
exponential backoff capped at 250ms and a bounded attempt count from the new
lock_retry_attempts setting (default 10). Behaviour inside a synchronous deadline is unchanged:
that path still uses with_lock_retry and its deadline budget.

On the evidence. SQLite's own busy handler masks the gap until exhausted, so a test that
merely holds a lock proves nothing on fast hardware. The three new tests disable the busy handler
so that only a Ruby-level retry can succeed:

  • a contended enqueue retries instead of failing immediately
  • six concurrent enqueues all succeed and allocate unique sequences
  • a permanently locked database still raises rather than hanging, honouring the attempt bound

All three fail on main with SQLite3::BusyException and pass here.

I could not reproduce the original EnqueueTest flake locally even with the busy handler
disabled entirely: this machine never contends enough, which matches earlier attempts. The
targeted tests are the evidence for the mechanism; the CI flake staying gone is the evidence for
the outcome, and that can only accumulate over time.

2. Every GitHub Actions reference pinned

All of actions/checkout, ruby/setup-ruby, actions/setup-node, and
softprops/action-gh-release are now pinned to commit SHAs with the tag kept as a trailing
comment. rubygems/configure-rubygems-credentials was already pinned. Zero mutable references
remain, which matters because every one of these jobs gates the release job.

Note this does not resolve the Node 20 deprecation warning on action-gh-release; that needs an
upstream release targeting Node 24, and pinning does not change the runtime it requests.

3. Benchmark evidence for the delivery paths

benchmark/component_delivery.rb, for one mutation that changes three components:

Delivery path Browser requests Server render time
Individual component refreshes 3 0.535 ms
Batched refresh 1 0.249 ms
State payload broadcast 0 0.100 ms

The request column is the point. Server render time is small in every path, so the win is not
faster rendering, it is fewer round trips: each individual refresh is a full HTTP request through
the Rails middleware stack.

These are server-side numbers. They exclude network latency, Action Cable delivery, and
browser rendering, which dominate wall-clock time in a real deployment and make the request-count
difference matter more than these figures suggest. End-to-end latency against a deployed
application is still unmeasured, and I have said so in docs/benchmarks.md rather than implying
the benchmark covers it.

Compatibility

No database change, no migration, no initializer regeneration, no token format change.
lock_retry_attempts is additive with a default. The retry only engages on lock contention that
previously raised, so no currently-succeeding call behaves differently.

Validation

bundle exec rake   # 271 runs, 1055 assertions, 0 failures
npm test           # 23 pass, 0 fail
COUNT=200 bundle exec ruby -Ilib benchmark/component_delivery.rb

Standard Ruby, RuboCop, RBS, Steep, and Brakeman clean.

Asynchronous enqueue had no Ruby-level retry budget, so a contended write depended entirely on SQLite's busy handler and surfaced SQLite3::BusyException once concurrent writers exhausted it. This is the intermittent EnqueueTest failure that has cost re-runs on three pull requests. Retry busy errors with bounded exponential backoff, capped by lock_retry_attempts. Also pin every GitHub Actions reference to a commit SHA, and add a benchmark for the three reactive delivery paths.
@greptile-apps

greptile-apps Bot commented Aug 9, 2026

Copy link
Copy Markdown

Greptile Summary

The PR adds bounded Ruby-level retries for contended SQLite transactions outside synchronous deadlines, pins GitHub Actions to immutable revisions, and documents a benchmark of reactive delivery paths.

  • Adds configurable exponential backoff for SQLite busy errors while retaining deadline-based retry behavior.
  • Adds focused contention tests covering eventual success, concurrent sequence allocation, and retry exhaustion.
  • Pins workflow actions, advances the release version to 0.7.1, and adds delivery-path benchmark support and results.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
lib/solid_objects/database_adapters/sqlite.rb Routes non-deadline transactions through bounded busy-error retries while preserving the existing synchronous-deadline path.
lib/solid_objects/configuration.rb Adds the lock_retry_attempts configuration setting with a default of ten retries.
test/integration/enqueue_lock_retry_test.rb Exercises transient contention, concurrent enqueue sequencing, and permanent-lock exhaustion with SQLite’s native busy wait disabled.
.github/workflows/ci.yml Replaces mutable action tags with immutable commit revisions.
benchmark/support.rb Adds benchmark fixtures and measurements for individual refresh, batched refresh, and state-payload delivery.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[SQLite transaction] --> B{Sync deadline active?}
  B -->|Yes| C[Use deadline-based lock retry]
  B -->|No| D[Execute transaction]
  D --> E{Busy error?}
  E -->|No| F[Return result or propagate other error]
  E -->|Yes| G{Retry budget exhausted?}
  G -->|Yes| H[Re-raise busy error]
  G -->|No| I[Wait with capped exponential backoff]
  I --> D
Loading

Reviews (2): Last reviewed commit: "chore: prepare 0.7.1 release" | Re-trigger Greptile

@cardmagic

Copy link
Copy Markdown
Owner Author

@greptileai review

@cardmagic
cardmagic merged commit e890b7f into main Aug 9, 2026
13 checks passed
@cardmagic
cardmagic deleted the agent/enqueue-lock-retry-and-benchmarks branch August 10, 2026 13:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant