Skip to content

feat: add a Redis cross-process wake-up - #20

Merged
cardmagic merged 2 commits into
mainfrom
agent/redis-wake-up
Aug 10, 2026
Merged

feat: add a Redis cross-process wake-up#20
cardmagic merged 2 commits into
mainfrom
agent/redis-wake-up

Conversation

@cardmagic

Copy link
Copy Markdown
Owner

Roadmap milestone 1, the remaining cross-process option.

Why

MySQL has no notification primitive, so MySQL applications had no way to remove the cross-process polling delay that #14 fixed for PostgreSQL. Measured on Redis 7.4:

Wake-up strategy p50 p95
In-process WakeUp 103.8 ms 105.1 ms
WakeUpAdapters::Redis 5.7 ms 8.9 ms

One subscription per process, not per thread

The PostgreSQL adapter opens a listening connection per waiting thread, because LISTEN state lives on the connection. Redis is different: a subscribed connection cannot issue other commands, so a connection per thread would multiply connections for no benefit. Instead one background thread holds the subscription and fans out to every waiting role through an in-process condition variable.

My first draft did use per-thread connections, and its shared-adapter test caught a genuine flaw: listen created a client but never actually subscribed, so a signal published before wait was missed and that waiter sat until its timeout. Subscribing eagerly in the background thread closes that window, and listen blocks until the subscription is confirmed.

Staying optional

The redis gem is not a dependency of this gem; applications add it themselves, and a clear ArgumentError explains it if missing. WakeUpAdapters.for deliberately does not select Redis: it is infrastructure this gem otherwise does not require, so choosing it should be explicit rather than inferred from a REDIS_URL happening to be set.

Polling remains the upper bound, and neither signalling nor waiting raises into its caller.

Tests

Nine integration tests against a real Redis: a signal from another client wakes a waiter, waiting still times out when nothing signals, every waiter on one shared adapter wakes on one signal, one subscription serves the process, signalling and waiting never raise and the failed wait still paces, stopping releases the subscription, the wake-up contract holds, and a bad client is rejected clearly.

A redis CI job runs them against Redis 7, and the release job depends on it. They skip when no Redis is reachable.

Roadmap

Milestone 1 removed, recorded under "Implemented and tested" with the measured numbers, and the note that MySQL keeps polling now says "unless they configure the Redis adapter". Remaining milestones renumbered, per #15.

bundle exec rake   # 321 runs, 0 failures

@greptile-apps

greptile-apps Bot commented Aug 10, 2026

Copy link
Copy Markdown

Greptile Summary

The PR adds an optional Redis Pub/Sub wake-up adapter that uses one background subscription to notify all runtime roles in a process.

  • Adds Redis subscription lifecycle, publication, failure pacing, and generated RBS.
  • Adds Redis integration coverage and a dedicated Redis CI service.
  • Documents explicit Redis adoption and updates the roadmap and changelog.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
lib/solid_objects/wake_up_adapters/redis.rb Implements a shared Redis Pub/Sub subscription, local waiter fan-out, lifecycle cleanup, and non-raising failure behavior.
test/integration/redis_wake_up_test.rb Exercises cross-client signaling, concurrent waiters, repaired startup timing, failure pacing, and shutdown.
.github/workflows/ci.yml Adds a Redis-backed integration job and requires it before release.
docs/roadmap.md Records the Redis milestone and renumbers the remaining roadmap items.

Sequence Diagram

sequenceDiagram
    participant Role as Runtime role
    participant Adapter as Redis adapter
    participant Subscriber as Subscriber thread
    participant Redis
    Role->>Adapter: wait(timeout)
    Adapter->>Subscriber: start subscription if needed
    Subscriber->>Redis: SUBSCRIBE channel
    Redis-->>Subscriber: subscription confirmed
    Subscriber-->>Adapter: ready
    Role->>Adapter: wait on condition
    participant Publisher as Another process
    Publisher->>Redis: PUBLISH channel
    Redis-->>Subscriber: message
    Subscriber->>Adapter: increment counter and broadcast
    Adapter-->>Role: wake early
Loading

Reviews (2): Last reviewed commit: "fix: do not absorb a signal while subscr..." | Re-trigger Greptile

Comment on lines +54 to +59
return paced_failure(timeout) unless listen

mutex.synchronize do
signalled = @signalled
condition.wait(mutex, timeout.to_f)
@signalled != signalled

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Signal counter absorbs wake-ups

When a Redis notification arrives after listen releases the mutex but before wait snapshots @signalled, the increment becomes the new baseline and the condition broadcast occurs before the waiter is registered, causing the runtime role to sleep until polling_interval expires.

Prompt To Fix With AI
This is a comment left during a code review.
Path: lib/solid_objects/wake_up_adapters/redis.rb
Line: 54-59

Comment:
**Signal counter absorbs wake-ups**

When a Redis notification arrives after `listen` releases the mutex but before `wait` snapshots `@signalled`, the increment becomes the new baseline and the condition broadcast occurs before the waiter is registered, causing the runtime role to sleep until `polling_interval` expires.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid, and it is the same class of bug as the one my own test caught earlier in this adapter: a window where a signal is delivered to nobody.

A notification arriving after listen released the mutex but before wait snapshotted @signalled became the new baseline, and its broadcast reached no registered waiter, so the role slept out its polling interval.

The counter is now snapshotted before subscribing and re-checked before blocking, so a signal delivered while a waiter was still getting ready is observed rather than swallowed. A test publishes from inside listen to model exactly that interleaving; it fails against the previous code and passes now.

@cardmagic

Copy link
Copy Markdown
Owner Author

@greptileai review

MySQL has no notification primitive, so applications that cannot use PostgreSQL notifications had no way to remove the cross-process polling delay. Measured latency drops from 103.8ms to 5.7ms at p50. One background subscription per process fans out to every waiting role in memory: Redis delivers only to a subscribed connection, and a subscribed connection cannot serve other callers, so a connection per thread would multiply connections without benefit. Subscribing eagerly closes the window where a signal sent during startup was missed, which an earlier per-thread draft did not, and its test caught. The redis gem stays outside this gem's dependencies and WakeUpAdapters.for does not select it.
A notification arriving after listen released the mutex but before wait snapshotted the counter became the new baseline, and its broadcast reached no registered waiter, so the role slept until its polling interval expired. Snapshot before subscribing and re-check before blocking, so a signal delivered while a waiter was getting ready is observed rather than swallowed.
@cardmagic
cardmagic force-pushed the agent/redis-wake-up branch from 910337b to bc2576c Compare August 10, 2026 14:28
@cardmagic
cardmagic merged commit 68956a9 into main Aug 10, 2026
20 checks passed
@cardmagic
cardmagic deleted the agent/redis-wake-up branch August 10, 2026 14:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant