Merge pull request #6 from cardmagic/agent/doctor-cleanup-and-component-instrumentation

fix: scope doctor probe cleanup and instrument component refreshes

Author: cardmagic

CHANGELOG.md

@@ -1,5 +1,29 @@
 # Changelog
 
+## 0.5.1 - 2026-08-07
+
+- Restore the SQLite busy wait that a synchronous invocation suspends for its
+  deadline. Rails installs the busy wait as a Ruby busy handler through the
+  sqlite3 `timeout` configuration, which `PRAGMA busy_timeout` reports as zero
+  and silently replaces, so the previous save and restore left pooled
+  connections with no busy handler at all. Every later writer on that
+  connection, inside or outside Solid Objects, then failed immediately with
+  `SQLite3::BusyException` instead of waiting for the lock. Suspend the busy
+  wait only when the adapter can identify how to restore it, so an Active
+  Record release that stops exposing the configured timeout loosens
+  synchronous deadline bounds instead of stripping lock waiting from a shared
+  pooled connection.
+
+- Run the doctor round-trip probe on a dedicated caller process, and accept an
+  explicit process registry in `SynchronousInvocation`, so the probe can no
+  longer stop and delete a shared application caller process, release its
+  activations, and unclaim its messages.
+- Report doctor probe cleanup failures as a failed or warned check instead of
+  raising a database lock error out of the command and leaking the probe
+  caller process.
+- Instrument component refreshes with actor identity, component name, key,
+  dependencies, refresh method, revision, and outcome, excluding locals.
+
 ## 0.5.0 - 2026-08-07
 
 - Add repeatable reactive components with signed string or integer keys and

Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    solid_objects (0.5.0)
+    solid_objects (0.5.1)
       actioncable (>= 8.0)
       actionpack (>= 8.0)
       actionview (>= 8.0)
@@ -373,7 +373,7 @@ CHECKSUMS
   rubocop-rails-omakase (1.1.0) sha256=2af73ac8ee5852de2919abbd2618af9c15c19b512c4cfc1f9a5d3b6ef009109d
   ruby-progressbar (1.13.0) sha256=80fc9c47a9b640d6834e0dc7b3c94c9df37f08cb072b7761e4a71e22cff29b33
   securerandom (0.4.1) sha256=cc5193d414a4341b6e225f0cb4446aceca8e50d5e1888743fac16987638ea0b1
-  solid_objects (0.5.0)
+  solid_objects (0.5.1)
   sqlite3 (2.9.5-aarch64-linux-gnu) sha256=78075b6337d3d182c6d2b4691049ed45cd220826160c9ea18946bf6a1de200dc
   sqlite3 (2.9.5-aarch64-linux-musl) sha256=18c801185deb4adc01ddb281e8f672a39e3d1729979ca91e39439cd3eac0402d
   sqlite3 (2.9.5-arm-linux-gnu) sha256=1bdfca0c7d63998c60b0f4a8e3c8df2d33800ccc4abd2d612eddbbbc92a4c48b

app/controllers/solid_objects/components_controller.rb

@@ -8,12 +8,25 @@ class ComponentsController < ActionController::Base
 
     # @rbs () -> void
     def show
+      SolidObjects.instrument(:"component.refreshed") { |payload| refresh(payload) }
+    end
+
+    private
+
+    # @rbs (Hash[Symbol, untyped]) -> void
+    def refresh(payload)
       registration = ComponentRegistration.from_token(
         params.require(:token)
       )
+      payload.merge!(registration_payload(registration))
       requested_revision = requested_revision_key
       snapshot = ActorSnapshot.new(registration.reference)
-      return head :conflict if newer_than_snapshot?(requested_revision, snapshot)
+      payload[:instance_id] = snapshot.instance_id
+      payload[:revision] = snapshot.revision
+      if newer_than_snapshot?(requested_revision, snapshot)
+        payload[:outcome] = "conflict"
+        return head :conflict
+      end
 
       authorization_context = SolidObjects
         .configuration
@@ -26,18 +39,32 @@ def show
         authorization_context:
       ).call
       response.headers["Cache-Control"] = "private, no-store"
+      payload[:outcome] = "rendered"
       render html: component_frame(registration, snapshot, rendered)
     rescue Unauthorized
+      payload[:outcome] = "unauthorized"
       head :forbidden
     rescue UnknownComponent
+      payload[:outcome] = "unknown_component"
       head :not_found
     rescue ActionController::ParameterMissing,
       ArgumentError,
       InvalidComponentToken
+      payload[:outcome] = "invalid_token"
       head :bad_request
     end
 
-    private
+    # @rbs (ComponentRegistration) -> Hash[Symbol, untyped]
+    def registration_payload(registration)
+      {
+        actor_type: registration.reference.actor_type,
+        actor_id: registration.reference.actor_id,
+        component_name: registration.component_name,
+        component_key: registration.component_key,
+        dependencies: registration.dependencies,
+        refresh_method: registration.refresh_method
+      }
+    end
 
     # @rbs () -> Array[Integer]
     def requested_revision_key

docs/correctness.md

@@ -167,7 +167,21 @@ result. Adapter lock/query deadlines cover the durable enqueue, caller-process
 registration and heartbeat, activation coordination, and result observation.
 SQLite retries busy coordination operations only within the original call
 deadline and reports `waiting_on=database_contention` when the database cannot
-be inspected at timeout. If enqueue cannot commit, `SyncEnqueueTimeout` is
+be inspected at timeout. To keep those retries in Ruby, the SQLite adapter
+suspends the connection's busy wait for the duration of each deadline-bound
+transaction and restores it afterwards. Restoration reinstalls the Ruby busy
+handler Rails configures from the sqlite3 `timeout` setting, which
+`PRAGMA busy_timeout` neither reports nor preserves, so a synchronous call
+leaves the connection's lock waiting behaviour exactly as it found it for
+later writers inside and outside Solid Objects.
+
+The adapter suspends the busy wait only when it can identify how to restore
+it. When a future Active Record release stops exposing the configured
+timeout, the adapter leaves the connection untouched: synchronous deadlines
+lose their tight bound and wait as long as the configured busy wait allows,
+rather than stripping lock waiting from a pooled connection the rest of the
+application shares. A test asserts the timeout stays discoverable so the
+looser bound cannot be adopted silently. If enqueue cannot commit, `SyncEnqueueTimeout` is
 raised and no message reference exists. MySQL lock waits have one-second InnoDB
 granularity. Ruby handlers that already started are not preempted.
 

docs/operations.md

@@ -15,6 +15,14 @@ so the schema check compares the required shape instead of a fixed timestamp.
 Warnings such as an all-deny neutral policy do not fail the command because a
 context-aware production policy may correctly deny the probe.
 
+The round-trip probe runs on its own dedicated caller process rather than the
+shared application caller process, and removes that record together with its
+temporary actor when it finishes. Running the doctor inside a process that
+already serves synchronous calls therefore leaves the application caller
+process, its activations, and its claimed messages untouched, including when an
+application call overlaps the probe. A database busy enough to block cleanup
+reports a failed or warned check rather than raising out of the command.
+
 ## Runtime
 
 Start all configured roles:
@@ -147,10 +155,20 @@ transaction rejection, commit-action start/completion/failure, effect and
 broadcast enqueue/completion, reminder enqueue, actor destruction/expiration,
 retention pruning, process cleanup, and supervisor lifecycle.
 
+`solid_objects.component.refreshed` covers every authorized component refresh
+request. Its payload carries the actor identity, `component_name`,
+`component_key`, declared `dependencies`, `refresh_method`, the rendered
+`instance_id` and `revision`, and an `outcome` of `rendered`, `conflict`,
+`unauthorized`, `unknown_component`, or `invalid_token`. Use it to watch
+refresh rate per key, authorization denials, superseded requests, and render
+duration. A rejected token reports only the outcome, since no signed identity
+was recovered.
+
 Payloads contain stable runtime identifiers, actor identity, sequence,
 attempts, ownership generations, and safe exception summaries where relevant.
-Arguments, actor state, results, and outbox payloads are excluded. The bundled
-log subscriber turns the same notifications into structured logger hashes.
+Arguments, component locals, actor state, results, and outbox payloads are
+excluded. The bundled log subscriber turns the same notifications into
+structured logger hashes.
 
 ## Retention and backups
 

lib/solid_objects/database_adapters/sqlite.rb

@@ -67,11 +67,49 @@ def with_lock_probe
       def with_transaction_deadline(connection)
         return yield unless SyncDeadline.active?
 
-        previous_timeout = connection.select_value("PRAGMA busy_timeout").to_i
-        connection.execute("PRAGMA busy_timeout = 0")
-        yield
-      ensure
-        connection.execute("PRAGMA busy_timeout = #{previous_timeout}") if previous_timeout
+        busy_wait = restorable_busy_wait(connection)
+        return yield unless busy_wait
+
+        begin
+          connection.execute("PRAGMA busy_timeout = 0")
+          yield
+        ensure
+          restore_busy_wait(connection, busy_wait)
+        end
+      end
+
+      # @rbs (untyped) -> Hash[Symbol, untyped]?
+      def restorable_busy_wait(connection)
+        pragma_timeout = connection.select_value("PRAGMA busy_timeout").to_i
+        return { pragma_timeout: } if pragma_timeout.positive?
+
+        handler_timeout = configured_busy_handler_timeout(connection)
+        return nil unless handler_timeout
+
+        { pragma_timeout:, handler_timeout: }
+      end
+
+      # @rbs (untyped, Hash[Symbol, untyped]) -> void
+      def restore_busy_wait(connection, busy_wait)
+        handler_timeout = busy_wait[:handler_timeout]
+        if handler_timeout
+          connection.raw_connection.busy_handler_timeout = handler_timeout
+          return
+        end
+
+        connection.execute("PRAGMA busy_timeout = #{busy_wait.fetch(:pragma_timeout)}")
+      end
+
+      # @rbs (untyped) -> Integer?
+      def configured_busy_handler_timeout(connection)
+        return nil unless connection.respond_to?(:raw_connection)
+        return nil unless connection.raw_connection.respond_to?(:busy_handler_timeout=)
+
+        pool = connection.respond_to?(:pool) ? connection.pool : nil
+        return nil unless pool.respond_to?(:db_config)
+
+        timeout = pool.db_config.configuration_hash[:timeout]
+        timeout&.to_i
       end
 
       # @rbs (Exception) -> bool

lib/solid_objects/doctor.rb

@@ -215,25 +215,63 @@ def check_runtime
     # @rbs () -> Check
     def check_sync_round_trip
       actor_id = SecureRandom.uuid
+      probe_registry = ProcessRegistry.new
+      check = run_sync_probe(actor_id, probe_registry)
+      leftovers = remove_probe_records(actor_id:, probe_registry:)
+      return check if leftovers.empty? || check.failed?
+
+      warn_check(
+        :sync_round_trip,
+        "#{check.message}; could not remove the #{leftovers.join(" and ")}"
+      )
+    end
+
+    # @rbs (String, ProcessRegistry) -> Check
+    def run_sync_probe(actor_id, probe_registry)
+      probe_registry.register(kind: "caller", metadata: { execution: "doctor" })
       value = SecureRandom.hex(8)
-      process_registry = SolidObjects.caller_process.process_registry
-      reference = ProbeActor.ref(actor_id)
       message_reference = Mailbox.new.enqueue(
-        reference,
+        ProbeActor.ref(actor_id),
         :ping,
         { value: },
         kind: "sync"
       )
-      result = SynchronousInvocation.new.call(message_reference, timeout: 5.seconds)
+      result = SynchronousInvocation
+        .new(process_registry: probe_registry)
+        .call(message_reference, timeout: 5.seconds)
       raise Error, "unexpected round-trip result" unless result == value
 
       pass(:sync_round_trip, "durable synchronous actor call completed without a worker")
     rescue => error
       fail_check(:sync_round_trip, "#{error.class}: #{error.message}")
-    ensure
-      Instance.where(actor_type: ProbeActor.actor_type, actor_id:).delete_all if actor_id
-      process_registry&.stop
-      process_registry&.process_record&.delete
+    end
+
+    # @rbs (actor_id: String, probe_registry: ProcessRegistry) -> Array[String]
+    def remove_probe_records(actor_id:, probe_registry:)
+      leftovers = []
+      leftovers << "probe actor" unless delete_probe_actor(actor_id)
+      leftovers << "probe caller process" unless delete_probe_caller_process(probe_registry)
+      leftovers
+    end
+
+    # @rbs (String) -> bool
+    def delete_probe_actor(actor_id)
+      Instance.where(actor_type: ProbeActor.actor_type, actor_id:).delete_all
+      true
+    rescue
+      false
+    end
+
+    # @rbs (ProcessRegistry) -> bool
+    def delete_probe_caller_process(probe_registry)
+      process_record = probe_registry.process_record
+      return true unless process_record
+
+      probe_registry.stop
+      process_record.delete
+      true
+    rescue
+      false
     end
 
     # @rbs (Check, Check) -> bool

lib/solid_objects/synchronous_invocation.rb

@@ -6,6 +6,13 @@
 
 module SolidObjects
   class SynchronousInvocation
+    # @rbs @dedicated_process_registry: ProcessRegistry?
+
+    # @rbs (?process_registry: ProcessRegistry?) -> void
+    def initialize(process_registry: nil)
+      @dedicated_process_registry = process_registry
+    end
+
     # @rbs (MessageReference, timeout: Numeric) -> untyped
     def call(message_reference, timeout:)
       return call_before_deadline(message_reference, timeout:) if SyncDeadline.active?
@@ -92,9 +99,17 @@ def raise_rejection(message)
       )
     end
 
+    # @rbs () -> ProcessRegistry
+    def process_registry
+      dedicated_registry = @dedicated_process_registry
+      return SolidObjects.caller_process.process_registry unless dedicated_registry
+
+      dedicated_registry.tap(&:heartbeat)
+    end
+
     # @rbs (Message, deadline: Float) -> Integer
     def assist(message, deadline:)
-      process_registry = SolidObjects.caller_process.process_registry
+      process_registry = self.process_registry
       activation = ActivationManager
         .new(owner_id: process_registry.process_record.id)
         .claim(instance_id: message.instance_id)

lib/solid_objects/version.rb

@@ -1,5 +1,5 @@
 # rbs_inline: enabled
 
 module SolidObjects
-  VERSION = "0.5.0"
+  VERSION = "0.5.1"
 end

sig/generated/controllers/solid_objects/components_controller.rbs

@@ -7,6 +7,12 @@ module SolidObjects
 
     private
 
+    # @rbs (Hash[Symbol, untyped]) -> void
+    def refresh: (Hash[Symbol, untyped]) -> void
+
+    # @rbs (ComponentRegistration) -> Hash[Symbol, untyped]
+    def registration_payload: (ComponentRegistration) -> Hash[Symbol, untyped]
+
     # @rbs () -> Array[Integer]
     def requested_revision_key: () -> Array[Integer]