feat(testing): supply arbitrary test values from a seedable source#137
Merged
Conversation
Introduce `Any`, a first-party, dependency-free source of valid-but-arbitrary values for the inputs a test needs yet never asserts on (error codes, messages, instants, ids, enums, primitives). An explicit `Any` call reads as "this value is incidental" where a hand-picked literal reads as "this matters". The source is stored in an AsyncLocal, so it flows with the execution context and is safe under parallel test runners — the same contract the clock and instance-id seams already keep. Values are arbitrary by default (which exposes tests that secretly depend on one) and become deterministic inside an `Any.UseSeed(...)` scope for reproducibility. The clock and instance-id seams gain matching `UseAny()` variants layered on the same source: `Clock.UseAny()` freezes OccurredAt to one arbitrary instant, and `InstanceIds.UseAny()` assigns a distinct arbitrary id per error; both take an optional seed. ErrorContextKey is deliberately left out of this first surface: its process-wide registry has no public reset, so minting arbitrary keys would accumulate global state. The decision and its reasoning are recorded in ADR-0006 (Proposed). - FirstClassErrors.Testing: Any, Clock.UseAny, InstanceIds.UseAny - Tests covering seeding determinism, scope isolation, sentinel exclusion - Testing guide updated in English and French, plus the NuGet readme - ADR-0006 proposed and indexed Refs: #135 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ULkMmGS2EgnALAJtwpxtP
Extract the seedable, context-local random engine and its primitive generators into an internal `ArbitrarySource`, leaving `Any` as a thin, error-aware facade over it. The clock and instance-id seams now draw their arbitrary values from the same engine. The public surface of `Any` (and of `Clock.UseAny` / `InstanceIds.UseAny`) is unchanged — this is a pure reorganization. Isolating the generic engine from the error-specific helpers keeps a future extraction into a standalone, error-agnostic utility a mechanical move rather than a rewrite; ADR-0006 records that as a follow-up trigger. Refs: #135 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ULkMmGS2EgnALAJtwpxtP
The testing guide was split upstream: assertions in Testing.*.md and the clock/instance-id determinism seams in DeterministicTesting.*.md. The Any factory straddles both, so give it its own page rather than wedging it into either. - Add doc/ArbitraryTestValues.en.md and its French translation, covering the Any value helpers, Any.UseSeed, and the Clock.UseAny / InstanceIds.UseAny seams. - Replace the interim Any section in DeterministicTesting.*.md with a short cross-reference to the new page. - Insert the page into the navigation chain (Deterministic Error Tests -> Arbitrary Test Values -> Operational Integration) and nest both testing sub-pages under the Testing Guide in the README table of contents, English and French in lockstep. - Point the ADR-0006 reference at the new page. Refs: #135 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ULkMmGS2EgnALAJtwpxtP
Replace the seed scopes with a single reproducibility runner. `Any.Reproducibly` wraps a test body, pins a fresh seed for the run, and — if the body throws — reports that seed (to Console.Error by default, or an injectable sink such as xUnit's ITestOutputHelper.WriteLine) before rethrowing the original exception. A `Reproducibly(seed, ...)` overload replays a reported seed; a `Func<Task>` overload covers async test bodies. Because the runner seeds the ambient Any source, the clock and instance-id seams need no seeded overloads of their own: run `Clock.UseAny()` / `InstanceIds.UseAny()` inside `Reproducibly` and they inherit the seed. So this removes from the public surface: - `Any.UseSeed(int)` (now an internal detail the runner uses); - `Clock.UseAny(int)` and `InstanceIds.UseAny(int)`. The unknown-vs-known-seed distinction drove the change: a known seed is only ever wanted to reproduce a failure, and that is exactly what `Reproducibly` expresses, so a standalone public seed scope had no separate purpose. Tests, the Arbitrary Test Values guide (English and French), the NuGet readme, and ADR-0006 are updated to match; the ADR's "surface the seed for replay" risk is now addressed by the runner. Refs: #135 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ULkMmGS2EgnALAJtwpxtP
Reefact
force-pushed
the
claude/testing-factory-random-values-sppwro
branch
from
July 16, 2026 00:07
6321050 to
8f57b3f
Compare
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
Any, a seedable, context-local source of arbitrary test values, and gives the clock and instance-id seamsUseAnyvariants. A test can mark an incidental input as arbitrary instead of hard-coding it, so the value under assertion stands out and accidental value dependencies surface. A value-sensitive test wrapped inAny.Reproduciblystays reproducible: the runner seeds the run, reports the seed on failure, and replays it on demand.Type of change
Changes
FirstClassErrors.Testing/Any.cs(new) — public facade supplying arbitrary error-domain values (ErrorCode, diagnostic/short/detailed messages,Transience,InteractionDirection,ErrorOrigin) and generic primitives (Guid,DateTimeOffset,string,int,bool,Enum<T>). AddsReproducibly(sync andFunc<Task>async) which seeds a run, reports the seed toConsole.Error— or an injectable sink such as xUnit'sITestOutputHelper.WriteLine— on failure, rethrows the original exception, and replays a given seed.FirstClassErrors.Testing/ArbitrarySource.cs(new) — internal, dependency-free pseudo-random engine (System.Randomin anAsyncLocalfor parallel-test safety).Anyis a thin, error-aware facade over it, kept separable for a possible future extraction.FirstClassErrors.Testing/Clock.csandInstanceIds.cs(modified) — addClock.UseAny()(freeze one arbitrary instant for the scope) andInstanceIds.UseAny()(a distinct arbitrary id per error), drawn from the same source, so they are reproducible when run insideAny.Reproducibly.AnyTests,ClockUseAnyTests,InstanceIdsUseAnyTests, covering seeded determinism, seed reporting with original-exception rethrow on failure, sentinel exclusion, value validity (ANY_CODE_*codes, UTC instants), and the async overload.doc/ArbitraryTestValues.en.mdand.fr.md;doc/DeterministicTesting.{en,fr}.mdcross-reference the new page; the navigation footers (doc/OperationalIntegration.{en,fr}.md) and tables of contents (README.md,doc/README.fr.md) are wired in;FirstClassErrors.Testing/README.nuget.mdupdated.maintainers/adr/0006-supply-arbitrary-test-values-from-a-seedable-source.md(new, indexed) — records the decision to build a first-party, dependency-free,AsyncLocal-backed source with opt-in reproducibility viaReproducibly, with alternatives considered.Testing
dotnet build FirstClassErrors.sln(CI warnings-as-errors, 0 warnings)dotnet test FirstClassErrors.sln(UnitTests 423, PropertyTests 21, Analyzers 85, Cli 63, GenDoc 136 — all passing)FirstClassErrors.Analyzers.UnitTests)Documentation
doc/updateddoc/README.fr.md) updated if user-facing behavior changedArchitecture decisions
Proposed: ADR-0006Related issues
Refs #135