feat: add a Redis cross-process wake-up - #20
Conversation
Greptile SummaryThe PR adds an optional Redis Pub/Sub wake-up adapter that uses one background subscription to notify all runtime roles in a process.
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains. Important Files Changed
Sequence DiagramsequenceDiagram
participant Role as Runtime role
participant Adapter as Redis adapter
participant Subscriber as Subscriber thread
participant Redis
Role->>Adapter: wait(timeout)
Adapter->>Subscriber: start subscription if needed
Subscriber->>Redis: SUBSCRIBE channel
Redis-->>Subscriber: subscription confirmed
Subscriber-->>Adapter: ready
Role->>Adapter: wait on condition
participant Publisher as Another process
Publisher->>Redis: PUBLISH channel
Redis-->>Subscriber: message
Subscriber->>Adapter: increment counter and broadcast
Adapter-->>Role: wake early
Reviews (2): Last reviewed commit: "fix: do not absorb a signal while subscr..." | Re-trigger Greptile |
| return paced_failure(timeout) unless listen | ||
|
|
||
| mutex.synchronize do | ||
| signalled = @signalled | ||
| condition.wait(mutex, timeout.to_f) | ||
| @signalled != signalled |
There was a problem hiding this comment.
Signal counter absorbs wake-ups
When a Redis notification arrives after listen releases the mutex but before wait snapshots @signalled, the increment becomes the new baseline and the condition broadcast occurs before the waiter is registered, causing the runtime role to sleep until polling_interval expires.
Prompt To Fix With AI
This is a comment left during a code review.
Path: lib/solid_objects/wake_up_adapters/redis.rb
Line: 54-59
Comment:
**Signal counter absorbs wake-ups**
When a Redis notification arrives after `listen` releases the mutex but before `wait` snapshots `@signalled`, the increment becomes the new baseline and the condition broadcast occurs before the waiter is registered, causing the runtime role to sleep until `polling_interval` expires.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.There was a problem hiding this comment.
Valid, and it is the same class of bug as the one my own test caught earlier in this adapter: a window where a signal is delivered to nobody.
A notification arriving after listen released the mutex but before wait snapshotted @signalled became the new baseline, and its broadcast reached no registered waiter, so the role slept out its polling interval.
The counter is now snapshotted before subscribing and re-checked before blocking, so a signal delivered while a waiter was still getting ready is observed rather than swallowed. A test publishes from inside listen to model exactly that interleaving; it fails against the previous code and passes now.
|
@greptileai review |
MySQL has no notification primitive, so applications that cannot use PostgreSQL notifications had no way to remove the cross-process polling delay. Measured latency drops from 103.8ms to 5.7ms at p50. One background subscription per process fans out to every waiting role in memory: Redis delivers only to a subscribed connection, and a subscribed connection cannot serve other callers, so a connection per thread would multiply connections without benefit. Subscribing eagerly closes the window where a signal sent during startup was missed, which an earlier per-thread draft did not, and its test caught. The redis gem stays outside this gem's dependencies and WakeUpAdapters.for does not select it.
A notification arriving after listen released the mutex but before wait snapshotted the counter became the new baseline, and its broadcast reached no registered waiter, so the role slept until its polling interval expired. Snapshot before subscribing and re-check before blocking, so a signal delivered while a waiter was getting ready is observed rather than swallowed.
910337b to
bc2576c
Compare
Roadmap milestone 1, the remaining cross-process option.
Why
MySQL has no notification primitive, so MySQL applications had no way to remove the cross-process polling delay that #14 fixed for PostgreSQL. Measured on Redis 7.4:
WakeUpWakeUpAdapters::RedisOne subscription per process, not per thread
The PostgreSQL adapter opens a listening connection per waiting thread, because
LISTENstate lives on the connection. Redis is different: a subscribed connection cannot issue other commands, so a connection per thread would multiply connections for no benefit. Instead one background thread holds the subscription and fans out to every waiting role through an in-process condition variable.My first draft did use per-thread connections, and its shared-adapter test caught a genuine flaw:
listencreated a client but never actually subscribed, so a signal published beforewaitwas missed and that waiter sat until its timeout. Subscribing eagerly in the background thread closes that window, andlistenblocks until the subscription is confirmed.Staying optional
The
redisgem is not a dependency of this gem; applications add it themselves, and a clearArgumentErrorexplains it if missing.WakeUpAdapters.fordeliberately does not select Redis: it is infrastructure this gem otherwise does not require, so choosing it should be explicit rather than inferred from aREDIS_URLhappening to be set.Polling remains the upper bound, and neither signalling nor waiting raises into its caller.
Tests
Nine integration tests against a real Redis: a signal from another client wakes a waiter, waiting still times out when nothing signals, every waiter on one shared adapter wakes on one signal, one subscription serves the process, signalling and waiting never raise and the failed wait still paces, stopping releases the subscription, the wake-up contract holds, and a bad client is rejected clearly.
A
redisCI job runs them against Redis 7, and the release job depends on it. They skip when no Redis is reachable.Roadmap
Milestone 1 removed, recorded under "Implemented and tested" with the measured numbers, and the note that MySQL keeps polling now says "unless they configure the Redis adapter". Remaining milestones renumbered, per #15.