Add Periscope: a typed, hierarchical observability framework#69
Open
kyleve wants to merge 59 commits into
Open
Add Periscope: a typed, hierarchical observability framework#69kyleve wants to merge 59 commits into
kyleve wants to merge 59 commits into
Conversation
Adds the three PeriscopeCore/PeriscopeUI/PeriscopeTools SPM library targets under Shared/Periscope/ with per-module README.md/AGENTS.md, placeholder sources, and hosted test bundles wired into Project.swift (unitTests helper, per-bundle schemes, Stuff-iOS-Tests CI scheme). Pure groundwork — no behavior yet; the framework's API lands in subsequent commits. Closes plan step: Scaffold PeriscopeCore/UI/Tools targets, docs, test bundles, and scheme wiring.
LogEvent is the Codable+Sendable protocol for structured events (stable eventName + eventVersion for persisted payloads, default .info level, rendered message). Message is the built-in freeform event the level conveniences will emit. LogLevel is an extensible struct (name + severity, OTel-style) so apps can define custom levels; OSLogType maps by severity band, with warning intentionally on .default like LogKit. Closes plan step: Core event model: LogEvent protocol, Message event, extensible LogLevel.
Log<Event> is a Sendable value struct over a deterministic scope tree: ScopeID derives from parent + name (SHA-256, namespaced), so the same path is the same scope in any process or launch. Deriving an event type creates a typed child scope; deriving for an identifier keys a child scope to an entity; + / linked(with:) merge scope sets so one event can reference both a model context and a UI context. Freeform level conveniences emit the built-in Message event on any typed logger. Records flow through the LogRecorder protocol (the Periscope system lands next; tests use an in-memory recorder). Closes plan step: Log<Event> value type: scope tree, id scoping, links via +.
Periscope is the process-wide recorder Logs emit into. Emitting never blocks: records and scope definitions append to one ordered, lock-based pending queue and a background drain task delivers batches to each LogSink (scope definitions always precede the records that reference them; late-added sinks get the scope registry replayed, idempotent per scope ID). A bounded recent-records buffer plus a per-record liveRecords() stream feed live UI. flush() awaits full delivery, then each sink's own flush. OSLogSink mirrors records to os.Logger with the root scope as category and the sub-path prefixed on the message; Periscope.shared ships with one under the main bundle ID. Log gains init(system: .shared) to match the planned Log<MyRoot>() spelling. Closes plan step: Periscope system: LogSink pipeline, non-blocking recorder, OSLog sink.
Level floors: Periscope.minimumLevel plus per-subtree overrides (nearest overridden ancestor wins; linked records pass when any of their scopes admits them), checked at emit before other work — Log's freeform conveniences consult shouldRecord first so filtered logging skips message rendering entirely. Flush policy: records at flushThreshold (default .error) trigger an automatic flush so high-severity context reaches sinks promptly. Drop policy: the pending queue is bounded; on overflow the oldest records drop (scope definitions never drop) and a synthetic DroppedEvents record under the Periscope system scope reports the gap. Redaction: an optional configuration hook transforms or suppresses every record before it is buffered or delivered anywhere. Closes plan step: Pipeline policies: level floors, flush/drop policy, redaction hook.
Conforming classes get a .log derived from type + instance identity: a root scope named after the type with numbered child scopes (#1, #2, …) per instance, cached by the system so an instance keeps its identity — no logger threading, no SwiftUI dependency, and a type's subtree finds every instance's events. LogEventType (default Message) types the logger; logSystem (default .shared) picks the system. Also hardens the scope-ordering test to index-based assertions now that systems define their own Periscope scope at init. Closes plan step: LogContextProviding protocol in Core for models and controllers.
log.withContext { … } (async and sync forms; async preserves caller
isolation via #isolation) binds the log's context to a @TaskLocal, so
Log.current — typed to any event, Log<Message>.current for freeform —
carries the full scope set anywhere in the async call tree, including
structured child tasks, without threading loggers through signatures.
Nested contexts link (inner log primary, duplicates collapse); with no
ambient context, Log.current falls back to a root logger on .shared.
Closes plan step: Task-local context propagation: withContext and
Log.current.
PeriscopeStore is the durable LogSink: a @Modelactor over SDLogEvent / SDLogScope / SDLogSession with #Index on the hot query fields. Events persist with their full scope hierarchy (event↔scope many-to-many plus the ordered scope list for display), a store-assigned monotonic sequence for stable same-millisecond ordering, a JSON payload keyed by eventName+eventVersion (decode(_:) recovers the type; tooling degrades to raw JSON), and the session that produced them — one SDLogSession per launch carrying app version, build, OS, and device model. Query API: events(matching:) filters by time range, level floor, event name, session, exact scope, scope subtree, and message search, newest first with limit/offset paging; scopes/sessions/event(id:) round out reads. Retention: pruneEvents(olderThan:)/(keepingNewest:), deleteAllEvents, and a changes() stream that pings after commits. Sink failures log to OSLog and count in an @_spi(Testing) counter — never silently dropped. Closes plan step: PeriscopeStore @Modelactor: schema, session metadata, indexes, retention, query API.
Tags are orthogonal to the scope hierarchy: log.tagged(key, value)
returns a context that stamps every event it emits (typed LogTagKey per
the identifier convention). Tags flow down derivations, merge across
links and nested ambient contexts (primary/inner side wins key
conflicts), mirror into OSLog messages as a sorted {k=v} suffix, and
persist via shared SDLogTag rows with an indexed key+value pair column.
LogQuery gains tag filtering, and the exact/subtree scope filters
collapse into one ScopeFilter enum (which also keeps the fetch
predicate within type-checker limits).
Closes plan step: Tag system: tagged contexts stamping events,
tag-based queries.
log.measure(token) { … } (sync and async, isolation-preserving) emits
paired SpanBegan/SpanEnded events sharing a SpanID, with the end event
emitted even on throw and durations measured by ContinuousClock —
tokens type-check against Event.SpanName (default String), so typed
events measure with leading-dot enums. log.begin(for:)/end(for:) open
and close spans keyed by primary scope + identifier via the recorder,
so a rebuilt logger on the same path closes the pair; mismatched calls
warn instead of silently dropping. Spans mirror to OSSignposter at
emission time (real durations in Instruments, not sink-delivery time).
The store persists an indexed spanID column with events(inSpan:) kept
separate from the general query predicate.
Closes plan step: Spans: measure closures, begin/end groups,
os_signpost mirroring.
LogAttachment (name, MIME content type, Data) attaches arbitrary payloads to any event: log(attachments:) for structured events and an attachments: parameter on the freeform level methods. Conveniences cover the common payloads — .error (description/domain/code as JSON), .json for any Encodable, and .image (PNG, UIKit-gated) — plus raw Data via the base init. Persistence uses @Attribute(.externalStorage) rows cascade-deleted with their event; queried events carry lightweight LogAttachmentInfo metadata and bytes load on demand through PeriscopeStore.attachments(forEvent:), so list fetches never pull blobs. Closes plan step: Attachments: LogAttachment API and external-storage persistence.
AmbientEventSource is the extensible seam for environmental context: Periscope.startAmbientSource(_:) retains a source for the process lifetime and hands it a logger under the shared ambient scope. Sources emit the standard AmbientEvent (typed AmbientKind + value, level raisable). Built-ins — started together via startDefaultAmbientSources() — cover app lifecycle transitions and memory warnings (UIKit-gated), network path changes via NWPathMonitor (connectivity and interface changes), thermal state (serious/critical at .warning), and Low Power Mode. Closes plan step: Ambient event sources: protocol plus built-in system observers.
PeriscopeCore's feature set is complete, so replace the scaffold-status README with the real overview (quick start, public API map, pipeline behavior, contracts) and expand the module AGENTS invariants (deterministic scope IDs, sink failure visibility, versioned JSON payloads). Docs-only change. Part of plan step: Ambient event sources (final Core docs pass).
View.logContext(_:) contributes a logger's scopes and tags to its descendants, linking onto whatever enclosing modifiers already contributed (nearest primary, tags merged — Log.linked(with:) semantics); an overload takes any LogContextProviding model directly, so .logContext(model.photo) + .logContext(screenLog) stack. Views read @Environment(\.logContext) — a Log<Message> that logs freeform immediately or derives typed loggers — with a Periscope.shared root fallback outside any modifier, mirroring Log.current. Core gains Log.retyped(to:), the context-preserving type change the environment accessor is built on. Module README/AGENTS updated to the real API. Closes plan step: PeriscopeUI: logContext modifier and environment accessor.
PeriscopeViewer renders a PeriscopeStore newest-first: searchable, filterable by level (standard ladder plus custom levels present), event type, scope subtree, and session, paged 200 at a time, with live refresh off the store's changes() signal and honest loading/failed/ empty states. Rows show a severity badge, event type, timestamp, and scope path; the detail screen shows tags, the pretty-printed payload JSON, and attachments loaded on demand. Export renders the active filter's full result set as NDJSON (deterministic sorted keys, oldest first, payload embedded as nested JSON) into a share sheet. Closes plan step: PeriscopeTools: latest-logs viewer with search, filters, NDJSON export.
LogTraceView walks backward from an origin event: the trail merges earlier events from the subtrees of all the origin's scopes (linked model + UI contexts both trace), events logged directly at ancestor scopes up each root chain (siblings excluded), and the origin's span pair — deduplicated and ordered newest first with the store's insertion sequence as tiebreak (StoredLogEvent now exposes sequence for exactly that). Every event detail gains a Trace button, and trail rows open their own detail so tracing can continue further back. Closes plan step: PeriscopeTools: log tracer across time and hierarchy.
PeriscopeAlerter watches a Periscope system's live records and routes everything at its threshold or above to a PeriscopeAlertHandler — the hookable seam, so apps with their own toast/notification stack conform instead of overriding UI. The built-in LocalNotificationAlertHandler posts each alerted record as a local notification under provisional authorization (quiet delivery, no permission prompt), logging post failures to OSLog rather than alerting recursively. start()/stop() manage the watch; only records emitted after start alert, and double start is a no-op. Closes plan step: PeriscopeTools: hookable debug toast for warning+ events.
View.logInspectable(_:) — taking a Log or a LogContextProviding model — badges the wrapped view while log view mode is on; tapping the badge presents every stored event in that context's scope subtrees (merged newest first across linked scopes, live-refreshing), each row linking into the standard event detail and from there the tracer. The mode's source of truth is a new Periscope.isInspectModeEnabled flag in Core; PeriscopeInspector is its observable SwiftUI mirror, injected once at the root via View.periscopeInspector(_:) and toggled from an app's developer settings. Finishes the PeriscopeTools docs (README quick start + AGENTS invariants). Closes plan step: PeriscopeTools: log view mode modifier for per-view event inspection.
A failed save previously left the ModelContext dirty: the poisoned batch's inserts stayed staged, every subsequent save re-attempted them, and a persistent failure silently lost all events from then on (a failed prune could even commit its staged deletions with the next unrelated write). Failure paths now run recoverFromFailedWrite(): rollback the transaction, drop the scope/tag row caches (they may hold rolled-back rows), and clear the session-row reference — the store now remembers its session identity as a value, and ensureActiveSession refetches or reinserts the same session on the next write, so recovery never forks the launch attribution. Throwing APIs (startSession, the prune/delete family) roll back and rethrow. Adds a DEBUG-gated @_spi(Testing) injectNextWriteFailure seam at the save funnels and covers each path: poisoned batch then healthy write, stale cache recovery with tags/placeholder scopes, session identity across recovery, scope-definition retry, and prune rollback. Addresses code-review finding 1 (store failure leaves context dirty).
InstanceScopeRegistry keyed instances by bare ObjectIdentifier, so a deallocated object's recycled pointer handed its cached identity — and its scope — to whatever object landed at that address next, even one of a different type. (The new InstanceIDTests demonstrated the reuse live: two back-to-back temporaries compared equal by address.) Two changes: InstanceID is the new cache key — pointer identity plus dynamic type, with a debugDescription that names the type — so a recycled address can never cross types; and each tracked instance now carries a retained deallocation tracker (ObjC associated object, keyed per registry so one object logged into two systems tracks both) whose deinit evicts the cache entry before the allocator can recycle the address, closing the same-type case too. Trackers hold the registry weakly so long-lived objects don't pin short-lived test registries. Instance numbers stay monotonic — #3 always means one specific instance within a run — and the registry no longer grows unboundedly. Addresses code-review finding 2 (instance identity via recycled pointers).
PeriscopeViewerModel.run() and LogInspectorModel.run() loaded first and subscribed to store.changes() after, so a commit landing between the two never triggered a refresh — a live viewer could sit stale until the next unrelated write. Both now acquire the stream before loading; a mid-load commit buffers in the stream and refreshes right after. Adds live-refresh tests for both models (previously untested), racing a write against the initial load deliberately. Addresses code-review finding 3 (subscribe-after-load gap).
Every record at the flush threshold spawned its own Task { flush() },
each looping on the drain until the system went quiet — an error storm
piled up one task per record for no added durability. Records now
request the flush through a coalescing gate: a single auto-flush task
runs at a time, and qualifying records that land mid-flush set a
pending flag that grants exactly one follow-up flush (covering their
durability) before the task retires. Deterministic storm test holds
the drain with a gated sink while 50 errors land and asserts at most
two sink flushes; a second test covers re-arming after settling.
Addresses code-review finding 4 (auto-flush task pileup).
liveRecords() streams used AsyncStream's default unbounded buffering, so a slow or stuck consumer (a paused toast handler, a backgrounded inspector) accumulated every emitted record in memory indefinitely. Streams now buffer with .bufferingNewest under the new Configuration.liveBufferCapacity (default 256): a consumer that falls behind loses the oldest buffered records — live surfaces want the newest activity, and durable history is the store's job. Test covers the drop-oldest behavior and normal flow after catching up. Addresses code-review finding 5 (unbounded live-stream buffering).
Spans could leak forever: a lost end(for:) left its entry (and its signpost interval) in memory permanently, and permanently locked out re-begins for that key. Spans now have full lifecycle semantics: - SpanLifetime (explicit on begin): .bounded(budget:) spans are closed as .expired by a deadline-driven watchdog when they outlive their budget (generation-tagged task, earliest-deadline sleep; the sweep is clock-injectable via @_spi so tests never sleep); .indefinite is the conscious opt-in for open-ended flows; .scoped marks measure spans. - SpanExit on every SpanEnded: success/failure/cancelled (with optional reasons) plus system modes superseded/expired/orphaned. measure now derives exits automatically — thrown errors record as .failure and CancellationError as .cancelled instead of masquerading as success. Abnormal exits log at .warning; messages describe the exit. - Re-begins supersede: beginning an already-open key closes the prior span as .superseded (attributed with its begin-time scopes and tags, which OpenSpan now carries) instead of refusing — no lockout. - SpanRelaunchPolicy, recorded on the SpanBegan payload: at startSession the store closes unmatched endsWithProcess spans from earlier sessions as .orphaned (duration nil — unknowable across process death); survivesRelaunch spans stay open, with resume mechanics staged in Shared/Periscope/TODOs.md. SpanBegan/SpanEnded bump to eventVersion 2 (first real exercise of the versioned-payload contract). Adds Shared/Periscope/TODOs.md, seeded with the staged span work and the remaining code-review findings. Addresses code-review finding 6 (open-span and signpost leaks).
Periscope.chunked(_:) grouped pending items by rewriting the last chunk with array + [element], copying the accumulated run on every iteration — quadratic in batch size, so a full 5,000-record backlog did ~12.5M element copies inside the drain. Runs now accumulate in mutable buffers and close when the item kind flips: O(n). Adds a gated-drain regression test asserting an interleaved backlog (records, scope definitions, records, …) reaches the sink as ordered runs. Addresses code-review finding 7 (quadratic chunking).
eventValue maps every fetched row's tags and attachments relationships into the returned value, so each row of a 200-event viewer page faulted both relationships in their own round trips (N+1). Event-read descriptors (events(matching:), events(inSpan:), fetchEventRow, and the orphan sweep's began fetch, which reuses row tags) now share a readDescriptor helper that sets relationshipKeyPathsForPrefetching for both. Attachment blobs still load lazily — only the metadata rows prefetch. Behavior unchanged; existing read-path tests cover it. Addresses code-review finding 8 (N+1 relationship faulting).
The new seeded fuzz test (fixed SplitMix64 seeds, four concurrent tasks interleaving emit/derive/flush/add-sink) immediately caught a real ordering bug: add(sink:) appended its scope replay to the END of the pending queue, so records already pending reached the new sink before the definitions of the scopes they reference. The replay is now prepended, and the drop report slots after the leading scope run rather than being written first, preserving the same guarantee. The fuzz invariants assert no record loss or duplication, per-emitter emission order, and scopes-before-records on every sink including late-added ones. Also covers the drop policy's scope-definitions-never-drop promise (gated overflow with interleaved definitions and records), and makes the alerter lifecycle tests deterministic: a new @_spi(Testing) Periscope.liveObserverCount asserts the subscription count directly instead of racing duplicate deliveries against a Task.yield(), and the stop test verifies unsubscription before emitting rather than relying on a sentinel from a second alerter. Addresses the code review's missing-tests items.
Periscope.record ran the redaction hook on every record before the floor check, so user redaction code executed — touching PII — for records the floor was about to discard (every filtered debug message under a .warning floor). Floors now gate first; redaction runs only for admitted records. Contract documented on Configuration.redact: floors apply to the record as emitted — redaction is content scrubbing, not routing. Test asserts the hook never fires for floor-filtered records (freeform or structured) and exactly once for admitted ones. Addresses code-review low-severity item: redaction-before-floor ordering.
Resolves the AGENTS.md targets-section conflict by combining main's Foreman removal with this branch's Periscope additions. Package.swift and Project.swift auto-merged: the package is now iOS-only (main dropped the .macOS platform with Foreman), and the Foreman-macOS-Tests scheme is gone.
Foreman's removal on main dropped the .macOS platform from Package.swift, so PeriscopeUI/PeriscopeTools no longer advertise a platform they can't compile for — the latent break is gone from the other side. Core keeps its canImport(UIKit) gating as per-SDK hygiene. Docs-only change (TODOs.md). Addresses code-review low-severity item: macOS compilation of the UI modules.
log.measure(.saveEvent, budget: .seconds(1)) { … } (sync and async)
emits a structured SpanOverdue warning while the closure is still
running past its budget — the signal arrives mid-hang, when it's
actionable, not after the fact. Each budgeted call spawns one
short-lived sentinel task (sleep the budget, emit unless cancelled),
cancelled when the closure finishes; the shared watchdog is
deliberately not involved, since its per-key tracking would make
concurrent same-token measures supersede each other. The span itself
stays .scoped with derived exits; SpanOverdue carries the SpanID (so
events(inSpan:) and the tracer link it) and the budget. The four
measure variants now share the timedSpan core.
Addresses the review follow-up: budget signal for closure spans.
NetworkPathAmbientSource.start replaced its boxed NWPathMonitor without cancelling the old one, which kept running and logging forever. The swap now returns the previous monitor and cancels it. AmbientEventSource.start documents its called-exactly-once contract — the notification-based built-ins deliberately stay unguarded rather than growing lock boxes for a call pattern nothing produces. Addresses code-review low-severity item: ambient source double-start.
PeriscopeViewer, LogInspectorView, LogTraceView, and LogEventDetailView captured their models via State(initialValue:), so a parent reconstructing them in place with a different store, origin, or scope set silently kept the first inputs forever. Each view's .task(id:) is now keyed on the store's actor identity plus its other identity inputs and rebuilds the model when they no longer match — re-keying also cancels the old task, tearing down the old model's changes() subscription. Verified by a hosting test that swaps stores under a live viewer and asserts, via a new @_spi(Testing) changeObserverCount on PeriscopeStore, that the viewer binds to the new store and releases the old stream. Tools test support gains an async showHosted helper and an async-predicate waitUntil; the rebinding rule is recorded as a PeriscopeTools AGENTS invariant. Addresses code-review low-severity item: State-from-init input capture.
PeriscopeInspector read Periscope.isInspectModeEnabled once at init and only wrote through afterwards, so direct writes to the Core flag — the documented source of truth — left the SwiftUI mirror stale (wrong toggle state, badges not reacting). Periscope now publishes the flag via inspectModeChanges() (current value, then one per change; the setter gains the standard no-change guard so redundant writes don't yield; bufferingNewest(1) since only the latest state matters), and the inspector subscribes for its lifetime, mirroring changes back into isEnabled. No-change guards on both sides keep the loop stable — covered by a mixed-writers convergence test plus direct-write reflection tests, and the AGENTS invariant now reads 'either side may write; they converge.' Addresses code-review low-severity item: inspector one-way sync.
Periscope.openSpans() exposes a snapshot of every span currently open via begin(for:), longest running first. OpenSpansView renders it with ticking ages (TimelineView re-snapshots once a second — no change-stream plumbing for live state that only needs a heartbeat), each row showing the span's name, age, lifetime/budget, and scope path. It reads the system rather than the store — open spans are live state, not history — and pushes from a developer menu inside an ambient NavigationStack. Snapshot lifecycle covered in Core tests (begin/end/expiry all reflected); hosting smoke tests cover populated and empty states. Addresses the review follow-up: open-spans developer surface.
SDLogEvent gains an indexed spanExitMode column (persisted from SpanEnded events via the new LogRecord.spanExit accessor), surfaced on StoredLogEvent and queryable via LogQuery.spanExitMode — 'everything that failed/expired/orphaned' is now one indexed query instead of payload archaeology. The viewer adds a span-exit filter, rows show a tinted exit chip beside the level badge, event detail renders Exit + reason (reason decoded from the payload; the mode is columnar), and NDJSON export includes the mode. The ninth predicate condition pushed the #Predicate macro past the type-checker's budget (and the ClosedRange workaround compiles but crashes SwiftData's SQL translation at runtime), so events(matching:) now builds one of two predicate variants: the exit variant replaces the event-name condition — an exit filter only matches span-ended events, so a conflicting name filter provably returns nothing, guarded explicitly. Addresses the review follow-up: surface span exits beyond message text.
log(PhotoLogs.self) { PhotoLogs(photoID: id) } read naturally but
failed to compile: Swift resolves a value call's arguments and
trailing closure as a single callAsFunction application, unlike type
callees (SwiftUI's Layouts) which get an implicit init-then-call
split — verified by experiment. New overloads give the natural
spelling a real resolution with the obvious semantics: derive the
typed (or entity-keyed) child scope and emit the event into it, one
expression. Derive-only forms keep their meaning via distinct arity;
attachment variants stay two-step. The PeriscopeUI typed probe now
uses the one-expression spelling as cross-module proof.
Addresses code-review low-severity item: trailing-closure footgun
(upgraded from docs-only to API after the Layout comparison).
LocalNotificationAlertHandler asked UNUserNotificationCenter for authorization before every post — one daemon round-trip per alerted record, during exactly the error storms the alerter exists for. The outcome now caches in a lock box: granted goes straight to add, denied goes quiet without re-asking, and a *failed* request stays unknown so transient failures retry on the next alert. The center sits behind an AlertNotificationCenter seam modeled on Where's NotificationReminderCenter (production adapter @unchecked Sendable — UNUserNotificationCenter is documented thread-safe), with the posting path nonisolated so the non-Sendable UNNotificationRequest is built in the sending region. Tests cover one-request-across-N-posts, quiet denial, and transient-failure retry via a scriptable fake center. Addresses code-review low-severity item: per-alert authorization requests. Closes out the review's low-priority list.
closeOrphanedSpans runs inside startSession — the launch path — and materialized every span-ended row ever retained (payload included) just to build a set of IDs, plus full rows for every historical span-began. Both passes now fetch only the spanID column (propertiesToFetch); full rows load exclusively for the orphan candidates (began minus ended), which are pathologically few. Behavior unchanged — the existing orphan tests cover all three outcomes. Addresses second-review finding 1 (orphan sweep on the launch path).
The setter snapshotted observers under the lock but yielded outside it, so two threads toggling the flag could interleave their yields out of order — and with bufferingNewest(1), a subscriber that wasn't mid-consume would hold the losing value forever, leaving PeriscopeInspector's mirror terminally divergent. Yields now happen inside the lock (they only buffer; no consumer can run under us), so delivery order matches flag order. Stress test hammers the flag from two tasks and asserts the single buffered value equals the final write. Addresses second-review finding 2 (inspect-stream ordering race).
Span lifecycle records went through record()'s floor check individually, so floors could swallow the closing half of a recorded pair: an expired or superseded SpanEnded emitted under a raised floor vanished while openSpans moved on — leaving a dangling SpanBegan that the next launch would mislabel as .orphaned. Worse, level asymmetry (began .info, abnormal ends .warning) meant a .warning floor recorded lonely ends with no began. The floor decision is now made exactly once, at begin (or at the top of measure), and the whole pair follows it: OpenSpan carries beganRecorded, and ends — normal, expired, superseded — bypass delivery-time floors via an internal LogRecord.bypassesFloors flag when their began was recorded, or stay silent when it wasn't (overdue sentinels included). A floored began silences the entire span; a recorded began always gets its end, even if floors rise mid-span. Signposts are unaffected — a separate channel. Recorded as a Core AGENTS invariant, covered across begin/end, expiry, supersession, and both measure paths. Addresses second-review finding 3 (floors could swallow span closings).
The watchdog task captured self strongly and sleeps until the earliest bounded deadline, so a discarded Periscope system — test suites create dozens — stayed alive until its next wake (up to a full budget). The loop now promotes a weak reference per call and never holds it across the sleep; a dead system's watchdog simply retires at its next check. Verified by a test that opens a 120s-budget span, drops the system, and asserts the weak reference clears — plus the previously untested respawn path: a 20ms span opened while the watchdog sleeps toward a 30s deadline must expire promptly, which only the cancel-and-respawn branch makes possible. TODOs.md records the second-review pass. Addresses second-review finding 4 (watchdog retention) and its missing respawn test.
CI has been red since the first push with 'unable to type-check this expression in reasonable time' on the events(matching:) #Predicate — the macro expands the whole condition chain into one giant inference tree, and while it squeaked under the local budget, CI's slower runners hit the wall (a scratch repro shows the macro form timing out while the equivalent statement form type-checks in 0.14s). The predicate is now hand-built in exactly the shape the macro would expand to, but as statements: one small, independently type-checked let per condition, folded into a conjunction. This also removes the earlier two-variant workaround — every filter, span exit included, now combines in a single predicate, with the name+exit guard gone (the conjunction handles conflicting filters naturally) — and future filters add a constant, not compounding, type-check cost. Runtime behavior is identical; every filter path is already covered by store and viewer tests. Fixes the Build & Test (iOS) CI failure.
A budgeted measure finishing right at its budget boundary had a microseconds-wide race: the sentinel could pass its cancellation check just as the closure completed, recording a false SpanOverdue after the SpanEnded. Both emissions now serialize through a per-measure gate: the end marks-and-emits under the lock, and a sentinel that arrives after stays silent. (Emitting under the gate is safe — record() takes only the system's own lock, and nothing takes the gate inside it.) Addresses second-review low item: sentinel false positive at the budget boundary.
Six copies of the same parent-walk-and-reverse had accumulated across OSLogSink, the three tool models, OpenSpansView, and NDJSONExporter, differing only in how a ScopeID resolves (a captured map, the system, locked sink state) — with one already-drifted detail (display joins with ' / ', exports with '/', now documented as deliberate). The walk lives once in Core as LogScope.ancestry(of:resolve:), root first, stopping at the first unresolvable scope; call sites keep their own resolver and separator. Covered by new LogScopeTests. Addresses second-review low item: duplicated scope-path walks.
showHosted hosted views at real animation speed, so any transition a hosted test triggered ran its full duration. Mirror WhereTesting.show: set the window layer speed to 100 for the body and restore it on exit. Addresses second-review low item: showHosted lacks the layer.speed handling show() has.
The keyed task rebuilds PeriscopeViewerModel when the viewer's store changes identity, but the export sheet and failure alert were left standing — a sheet generated against the old store could keep presenting stale NDJSON. Clear both alongside the model rebuild. Addresses second-review low item: stale export sheet across store swaps.
The original pipeline fuzz never touched spans, floors, or the open-span registry. A second seeded fuzz interleaves freeform emits, begin/end on keys shared across tasks (so re-begins supersede across tasks), global floor flips, and flushes; it asserts per-emitter delivery order under floors (subsequence, strictly increasing), that no span half ever dangles (every delivered spanID is exactly one began plus one ended — floored begins silence the whole pair), an empty open-span registry after cleanup, and scopes-before-records. Writing the invariants surfaced one soft spot, recorded as a P2 in TODOs.md: begin(for:) registers in openSpans before emitting its began, so a racing supersede can deliver a span's end before its began. Pair completeness always holds; strict intra-pair ordering would need register-and-emit to be atomic. Addresses second-review missing test: fuzz doesn't exercise spans/floors.
begin(for:) registered the span in openSpans and then emitted its SpanBegan as a second step, so a racing begin on the same key (or an end(for:)) could close the span and record its SpanEnded before the began ever entered the pipeline — the viewer and tracer, sorting by date and sequence, would render the pair as ended-before-began. LogRecorder.openSpan becomes beginSpan(key:span:began:): registration and the began record land under one state-lock acquisition, so a span is never visible for closing before its began is buffered. Redaction runs on the began before taking the lock (user code must not run under it). The superseded close now deliberately follows the *new* began — cause before effect: the re-begin is what closed it — and the caller still records it, since it needs no atomicity (the prior span left the registry with its began long since buffered). The span-lifecycle fuzz now asserts strict began-then-ended per span instead of tolerating either order, and the TODOs P2 recording this race moves to completed. Addresses review follow-up 1: end-before-began delivery.
The drop policy removed the oldest pending records regardless of kind, so overflow could split a span pair: a dropped began strands its end (nothing repairs a parentless SpanEnded), and a dropped end leaves the span reading as still open for the rest of the session — the orphan sweep only runs at the next launch. Scope definitions were already exempt; SpanBegan/SpanEnded records now are too (LogRecord.isProtectedFromDropping), with the drop accounting counting what actually dropped. SpanOverdue stays droppable — it's a disposable warning, not half of a pair. A queue saturated with protected records can briefly exceed the bound; they're rare and small, and a split pair is worse. Covered by a deterministic gated-overflow test (pair survives, the freeform flood drops and is reported) and by running the span-lifecycle fuzz under a 16-record queue so drop pressure joins the interleavings. Addresses review follow-up 2: drop policy can split span pairs.
liveRecords() observers could see records in a different order than the buffers: record() and beginSpan snapshotted observers under the lock but yielded after releasing it, so two racing emitters could deliver inverted — including a span's end reaching a live consumer before its began, even though sinks, the store, and recentRecords() were all correctly ordered. The inspect-mode flag was fixed for this exact failure class in the second review; records now get the same treatment. Yields only buffer, so nothing runs under the lock. The append-yield-drain-autoflush choreography record() and beginSpan had each hand-rolled collapses into Periscope.buffer (in-lock half) plus scheduleFollowUp (outside-lock tail), so the two paths can't drift; announceDropReport yields in-lock too. New tests pin the beginSpan bypass path — begans reach live observers and the recent buffer — and the live stream's buffered-order replay of a full begin/supersede/end lifecycle. Addresses review follow-ups 2 through 4: live-stream ordering, the untested beginSpan bypass, and the duplicated delivery choreography.
Redaction was the last remaining way to split a span pair: a hook returning nil for a SpanBegan left the span registered with beganRecorded intact, so every close path emitted an end whose began never entered the pipeline; nil for a SpanEnded stranded a began that was already delivered. The hook may still transform pair records freely, but suppression now falls back to a stripped copy (LogRecord.strippedOfSensitivePayload): tags and attachments dropped and a SpanEnded's freeform exit reason blanked — the PII carriers — while identity, date, scopes, and the floor bypass survive. Span names are typed tokens by convention, not user data. Level floors remain the supported way to silence spans, and SpanOverdue (disposable, not half of a pair) stays suppressible. Both record() and beginSpan route through one redacted(_:) helper. Addresses review follow-up 1: redaction could split span pairs.
kyleve
commented
Jul 8, 2026
| /// Start every built-in ambient source: network path, thermal state, | ||
| /// low power mode, and (where UIKit exists) app lifecycle and memory | ||
| /// warnings. | ||
| public func startDefaultAmbientSources() { |
Owner
Author
There was a problem hiding this comment.
It'd also be great to add an a11y source too.
PeriscopeCore's 30 files group into Events (the vocabulary: events, levels, tags, attachments), Loggers (Log and the scope tree), Context (task-local and per-instance derivation), Spans, Pipeline (the system, records, sinks), Store, and Ambient. PeriscopeTools groups one directory per tool (Viewer, Tracer, Alerts, InspectMode, Spans) plus Components for the shared display pieces. PeriscopeUI stays flat at one file. Pure moves (git mv) — SPM globs recursively, so no manifest changes; module AGENTS.md files document the layout. Tests stay flat, named 1:1 with their source files.
Review walkthrough finding 1: keep the ergonomic no-arg Log<Event>() init (records into Periscope.shared), and note in the module AGENTS.md that it's a conscious exception to the no-Core-defaults convention — tests must always pass system: explicitly.
Review walkthrough finding 2: the 'cannot fail' encode of the error payload used (try? ...) ?? Data(), which would silently persist a zero-byte application/json attachment that reads as a successful capture if a refactor ever made encoding fail. Per the programmer-error convention (and matching the NDJSON exporter's existing treatment): assertionFailure in debug, a valid empty JSON object as the release fallback.
Review walkthrough finding 3: a stored payload that no longer parses (on-disk corruption — payloads are JSONEncoder output at persist time) was silently dropped from the export line, indistinguishable from an event that never had one. The line now carries a payloadError marker with a byte-count hint, so a bug-report export is honest about exactly the row someone is likely chasing. Covered by an exporter test feeding garbage payload bytes.
Review walkthrough finding 4: exitReason (a SpanEnded's freeform reason lives in the payload, not a column) and prettyPayload were private on the view, so their degradation paths — an undecodable payload, garbage bytes — had no direct coverage. They're now internal StoredLogEvent extensions in the same file, pure functions of the event, with a 1:1 LogEventDetailViewTests covering reason round-trip, reasonless exits, undecodable payloads, key-sorted pretty printing, and garbage degradation.
Review walkthrough finding 5: the LogSpan file comment and README
claimed span names are 'typed tokens, not raw strings', but
Event.SpanName defaults to String — measure("save") is the
out-of-box spelling. Dropping the default isn't viable (an
associatedtype with no default or inference path would force an empty
SpanName enum onto every LogEvent conformance, Message included), so
the docs now state the actual contract: names resolve against
Event.SpanName, String by default; declare a SpanName enum for
compiler-checked leading-dot tokens, the recommended style for
structured events.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Periscope, a typed, hierarchical observability framework, as three new SPM modules under
Shared/Periscope/, wired intoPackage.swift,Project.swift, and theStuff-iOS-TestsCI scheme. It coexists withLogKit/LogViewerUI— noWhereLogmigration in this PR.PeriscopeCore — model and machinery
LogEvent(Codable; stableeventName+eventVersion), freeformMessage, extensibleLogLevelstruct (OTel-style name + severity).Log<Event>value loggers over a deterministic scope tree (same path = same scope in any process/launch): typed children (root(PhotoLogs.self)), entity children (photos(for: album.id)), single-expression derive-and-emit (album(for: id) { .uploaded }), links (model + ui), tags (tagged(.paymentID, id)), attachments (withError/Codable/image conveniences).withContext/Log.current, andLogContextProvidingfor derived per-instance loggers (identity-safe viaInstanceID+ associated-object deallocation trackers, so a recycled pointer can never inherit a dead object's logging identity).measurewith auto-derived exits (success/failure/cancelled) and optional budgets (aSpanOverduewarning fires while the closure hangs, race-free against the span's own end),begin/endwith explicitSpanLifetime(bounded spans expire via a clock-driven watchdog),SpanExitmodes,SpanRelaunchPolicy(relaunch orphan-closes spans the dead process left open),OSSignpostermirroring.LogSinkfan-out (OSLog sink built in) with level floors (global + per-subtree), coalesced auto-flush at error+, bounded drop policy with syntheticDroppedEventsgap reports, redaction hook, and a bounded live-records stream that replays exact buffered order.PeriscopeStore(@ModelActorsink) — indexed schema, per-launch sessions (app/OS/device metadata), versioned JSON payloads that outlive their Swift types, external-storage attachments, rich query API (time/level/type/session/scope-subtree/tag/span-exit/search, paged), retention pruning, rollback-on-failure so a poisoned batch can't wedge later saves.AmbientEventSourcewith built-ins for app lifecycle, memory warnings, network path, thermal state, and low power mode.PeriscopeUI
.logContext(_:)modifiers (acceptingLogvalues orLogContextProvidingmodels; stacking links contexts) and the\.logContextenvironment accessor.PeriscopeTools
PeriscopeViewer(searchable/filterable latest-logs viewer — level, event, scope, session, span exit — with NDJSON export),LogTraceView(walks an error back through time, linked scopes, and the ancestor chain),OpenSpansView(what's in flight right now, longest-running first),PeriscopeAlerter(hookable debug toast, local-notification default with cached authorization), and log view mode (.logInspectable(_:)+PeriscopeInspector, two-way synced with the system flag).Review hardening
Three full code-review passes followed the initial build, each landing fixes as dedicated commits.
First pass — every high/medium finding fixed: store rollback + failure injection, instance identity via dealloc trackers, subscribe-before-load in live models, coalesced auto-flush, bounded live buffers, the span lifecycle model, linear drain chunking, relationship prefetching. The closing seeded fuzz test caught a real bug —
add(sink:)appended its scope replay after already-pending records, so late-added sinks could see records before their scope definitions; the replay is now prepended.Second pass — the orphan sweep fetches only
spanIDcolumns; inspect-mode changes yield inside the state lock so racing setters can't strandbufferingNewest(1)subscribers; span pairs floor together (the begin-time decision governs the whole lifecycle — no dangling halves across floor changes); the watchdog holds the system weakly. Low items: overdue sentinels serialize with their span's end through a per-measure gate, the six copies of scope-path walking collapsed intoLogScope.ancestry(of:resolve:),showHostedgotshow()'s animation-speed handling, and the viewer clears export state on store swaps. Also fixed here: the store's nine-condition filter predicate is hand-built in statement form (PredicateExpressions, oneletper condition) after the#Predicatemacro's single inference tree blew the type-checker budget on CI runners.Third pass — span pair integrity. Writing a second seeded fuzz (spans, floor flips, cross-task supersession, drop pressure) surfaced that "no dangling halves" had three remaining attack vectors; all are now closed, and the invariant holds on every channel:
LogRecorder.beginSpanregisters a span and buffers itsSpanBeganunder one lock acquisition, so no racing supersede orend(for:)can deliver a span's end before its began.SpanBegan/SpanEnded(like scope definitions): a dropped began would strand its end; a dropped end reads as "still open" until the next launch's orphan sweep.SpanOverduestays droppable.nil(suppression) falls back to a stripped copy (tags/attachments dropped, exit reason blanked) instead of stranding the partner; floors are the supported way to silence spans.Live streams got the same ordering rigor: observer yields moved inside the state lock (
record()andbeginSpannow share one delivery helper), soliveRecords()consumers see exact buffered order — previously two racing emitters could invert live delivery even though sinks and the store were ordered.Remaining staged work (
survivesRelaunchresume mechanics) is tracked inShared/Periscope/TODOs.md, which also logs every completed finding.Testing
257 Swift Testing tests across 38 files in three hosted bundles (
PeriscopeCoreTests,PeriscopeUITests,PeriscopeToolsTests): deterministic pipeline tests via gated sinks, clock-injected watchdog sweeps, store failure injection, hosted SwiftUI tests for the tools, and two seeded fuzz suites — pipeline interleavings (no loss/duplication, per-emitter order, scopes-before-records on every sink) and span lifecycles (strict began-then-ended pairs under concurrent supersession, floor flips, and drop pressure; empty open-span registry after cleanup)../swiftformat --lintandStuff-iOS-Testspass locally and on CI.