perf: cut database round trips per sync call - #7
Merged
Conversation
benchmark/support.rb applied only the initial migration, so the state_revision column added in 0.4.0 was missing. Every message failed at commit with ActiveModel::UnknownAttributeError, retried with backoff, and the synchronous benchmarks timed out reporting waiting_on=not_yet_available. Apply both migrations, as the test helper already does.
Read the database clock once per transaction rather than once per step, and resolve the SQLite busy wait from the pool configuration rather than querying the connection for it. A synchronous call now issues 49 queries instead of 66, which matters most on PostgreSQL and MySQL where every query is a network round trip. Suspension still writes PRAGMA busy_timeout, because clearing the handler in Ruby instead changes where contention fails: BEGIN IMMEDIATE fails outright rather than the statement inside it.
Greptile SummaryThe PR reduces synchronous database round trips by sharing one database-clock reading per transaction and resolving SQLite busy-wait restoration from connection configuration. It also updates the benchmark schema setup, benchmark documentation, release metadata, generated signatures, and integration coverage. Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Begin adapter transaction] --> B[Create transaction clock scope]
B --> C[First database_now call]
C --> D[Read database clock]
D --> E[Cache transaction timestamp]
E --> F[Later database_now calls]
F --> G[Reuse cached timestamp]
G --> H[Commit or rollback]
H --> I[Clear clock scope and cache]
Reviews (2): Last reviewed commit: "refactor: inline the busy wait suspensio..." | Re-trigger Greptile |
Owner
Author
|
@greptileai review |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reduces the database work in a synchronous call, and fixes the benchmark harness that made
any of this impossible to measure.
The benchmark harness was broken
benchmark/support.rbapplied only the initial migration, so thestate_revisioncolumnadded in 0.4.0 was missing. Every message failed at commit with
ActiveModel::UnknownAttributeError, retried with backoff, and the synchronous benchmarkstimed out reporting
waiting_on=not_yet_available. No benchmark in the repo could run.Fixed by applying both migrations, as
test/database_test_helper.rbalready does.What the measurements showed
With the harness working, a warm synchronous call issues 66 queries but spends only
0.6-1.4ms in SQL, so 92-95% of wall time is Ruby, not the database. Two overheads
dominated the query count:
PRAGMA busy_timeoutread + write around every deadline-bound transactionSELECT STRFTIMEto read the database clockWhat this changes
The database clock is read once per transaction, not once per step. Callers inside one
transaction now share a single reading, which is also the more correct semantic for lease
and fencing comparisons. Clock reads per synchronous call drop from 7 to 4, which is exactly
one per transaction: enqueue, claim, and two fenced transactions.
The busy wait is resolved from the pool configuration instead of being queried.
configured_busy_handler_timeoutis pure Ruby, so checking it first removes thePRAGMA busy_timeoutread entirely. The pragma read now happens only when no timeout isconfigured, which is the case where there is no Ruby busy handler to restore.
Net: 66 queries to 49 per synchronous call, a 26% reduction. On SQLite that barely moves
p50, because SQL is 5% of the time. On PostgreSQL or MySQL, where every query is a network
round trip, query count is the latency:
docs/benchmarks.mdrecords a 60ms median MySQLwrite, which is what 66 round trips costs.
What this deliberately does not change
The remaining 14
PRAGMA busy_timeout = 0writes stay. Replacing them with a Ruby-levelbusy_handler(nil)is measurably better but changes behaviour: with the pragma,BEGIN IMMEDIATEsucceeds under contention and the statement inside it fails; with thehandler cleared,
BEGIN IMMEDIATEfails outright. Two existing contention tests assertretry attempts against the processes table and fail under the second behaviour.
The prize is real and measured, so it is worth doing separately with proper scrutiny:
The p99 tail belongs entirely to those writes. This PR does not fix it.
Effects
now compare against a consistent instant within a transaction, and each transaction still
reads a fresh one. Suspension and restoration of the busy wait are unchanged.
only. If both a
timeout:and an explicitpragmas: {busy_timeout:}are configured, theconfigured handler now wins on restore. That combination was already ambiguous.
Validation
bundle exec rake245 runs, 979 assertions, 0 failures. Standard Ruby, RuboCop, RBS, Steep, and Brakeman clean.
PostgreSQL and MySQL suites were not run locally because neither server is available; CI covers them.
Tests
Four tests, each written failing first: