WIP: DO NOT REVIEW Use generated regexes for gRPC JSON parsing

[artl93]

I am running an experiment - full analysis to follow. Contact @artl93

WIP / preliminary disposition: keep this draft. The clean evidence is complete enough to describe the result, but this is intentionally not ready for review.

Summary

Replace the two process-wide RegexOptions.Compiled timestamp and duration regex fields used by gRPC JSON transcoding with byte-for-byte-equivalent [GeneratedRegex] patterns.

The useful result is cold first use, not a broad steady-state throughput claim:

• First timestamp parse, duration parse, and timestamp serialization improve complete-process latency by 3.8-4.0 ms (9.3-10.9%) across 100 interleaved fresh processes.
• A diagnostic timer directly around the first reflected method invocation sees a larger 2.02x, 4.23x, and 10.41x difference and 52,424 B, 52,232 B, and 58,056 B less current-thread allocation. This includes one-time reflection/type-initialization/regex-runtime/JIT effects and is not a steady per-call speedup; complete-process time above is the end-to-end result.
• Field-mask validation, which does not require the regex-backed static fields, is neutral and acts as the cold control.
• Five crossover BenchmarkDotNet runs show mixed/mostly neutral warm parsing after accounting for control drift. Duration deserialization is the only stable warm improvement. Per-operation warm allocations are unchanged.
• The product DLL grows 4,608 bytes on disk (142,848 -> 147,456, +3.2%).

Why this changes first use

The current Legacy type stores both regexes in static fields. Any path that requires those fields initializes and compiles both regexes at runtime. Timestamp serialization also reads the static UnixEpoch field, so its first use pays the same type-initialization cost even though it does not match a regex. Once initialized, that same write path is regex-free and serves as a warm control. IsPathValid uses no static fields and stays neutral cold and warm.

GeneratedRegex emits specialized Regex subclasses and runners at build time. The generated methods return independent lazy singleton instances, avoiding runtime pattern parsing/dynamic-code setup and avoiding initialization of an unrelated pattern. The preserved RegexOptions.Compiled value remains observable in Regex.Options, but source generation emits the specialized runner at build time rather than reintroducing runtime IL emission. AddJsonTranscoding registration itself does not call Legacy; this targets the first timestamp/duration transcoding request or first timestamp serialization, not service-registration time.

This is most relevant to cold/serverless/scale-to-zero processes and first-request latency. Long-lived warm servers should expect little throughput change.

Clean fresh-process evidence

100 samples per implementation/path, alternating parent/candidate launch order. process is wall-clock time around the complete child process. first operation is measured directly inside the child around the unchanged MethodInfo.Invoke; it is not derived by subtraction. The inner timer includes shared reflection overhead and one-time runtime/type/JIT work, so it diagnoses where cold work moves but does not represent warm request throughput. Allocation is GC.GetAllocatedBytesForCurrentThread around that same invocation.

Path	Complete process: parent -> candidate	Delta	First operation: parent -> candidate	Diagnostic ratio	Allocation: parent -> candidate
Timestamp parse	42.806 ms -> 38.811 ms	-9.33%	15.207 ms -> 7.537 ms	2.02x	66,416 B -> 13,992 B
Duration parse	37.939 ms -> 34.117 ms	-10.08%	10.159 ms -> 2.402 ms	4.23x	61,968 B -> 9,736 B
Timestamp write	35.739 ms -> 31.836 ms	-10.92%	8.439 ms -> 0.811 ms	10.41x	60,448 B -> 2,392 B
Field-mask control	31.160 ms -> 31.185 ms	+0.08%	0.183 ms -> 0.186 ms	0.99x	376 B -> 376 B

Complete-process detail (mean; median; run SD; 95% CI):

• Timestamp: parent 42.806; 42.161; 4.184; 41.976-43.637 ms. Candidate 38.811; 38.432; 1.538; 38.506-39.116 ms.
• Duration: parent 37.939; 37.323; 1.830; 37.576-38.302 ms. Candidate 34.117; 33.629; 2.194; 33.681-34.552 ms.
• Timestamp write: parent 35.739; 35.431; 1.343; 35.473-36.006 ms. Candidate 31.836; 31.632; 1.239; 31.591-32.082 ms.
• Field mask: parent 31.160; 31.043; 0.901; 30.981-31.339 ms. Candidate 31.185; 31.146; 0.956; 30.996-31.375 ms.

Direct first-operation detail (mean; median; run SD; 95% CI):

• Timestamp: parent 15.207; 14.926; 1.427; 14.924-15.490 ms. Candidate 7.537; 7.491; 0.194; 7.498-7.575 ms.
• Duration: parent 10.159; 10.087; 0.408; 10.078-10.240 ms. Candidate 2.402; 2.361; 0.259; 2.351-2.453 ms.
• Timestamp write: parent 8.439; 8.363; 0.324; 8.375-8.503 ms. Candidate 0.811; 0.797; 0.049; 0.801-0.820 ms.
• Field mask: parent 0.183; 0.178; 0.018; 0.180-0.187 ms. Candidate 0.186; 0.177; 0.026; 0.181-0.191 ms.

The one-time allocation savings do not recur per request; warm BDN allocations below are identical.

Warm BenchmarkDotNet evidence

Normal out-of-process BDN, concurrent server GC, .NET 11 Arm64. Five runs per implementation in crossover order: parent/candidate, candidate/parent, parent/candidate, candidate/parent, parent/candidate. The table uses paired per-run percent changes and a t interval across five run-level deltas.

Method	Parent mean across runs	Candidate mean across runs	Paired change mean (95% CI)	Paired run deltas	Allocated
ParseTimestamp	280.483 ns (run SD 6.070)	272.129 ns (run SD 4.020)	-2.96% (-4.90% to -1.02%)	-4.51%, -1.60%, -1.03%, -4.21%, -3.44%	928 B
ParseDuration	137.374 ns (run SD 5.709)	126.611 ns (run SD 8.248)	-7.73% (-16.03% to +0.56%)	-14.55%, +3.00%, -7.77%, -7.54%, -11.80%	680 B
DeserializeTimestamp	328.117 ns (run SD 3.431)	323.563 ns (run SD 7.458)	-1.38% (-4.40% to +1.63%)	-2.09%, -0.05%, -5.32%, +0.61%, -0.05%	1,064 B
DeserializeDuration	197.635 ns (run SD 1.942)	180.886 ns (run SD 7.410)	-8.48% (-12.65% to -4.32%)	-5.02%, -12.15%, -9.53%, -10.83%, -4.89%	784 B
WriteTimestamp control	82.645 ns (run SD 1.110)	81.728 ns (run SD 1.615)	-1.09% (-4.17% to +1.99%)	-2.08%, -4.21%, -1.00%, -0.80%, +2.63%	400 B
ValidateFieldMask control	6.144 ns (run SD 0.111)	6.014 ns (run SD 0.167)	-2.11% (-3.68% to -0.54%)	-4.22%, -2.20%, -1.83%, -1.25%, -1.06%	0 B

The controls show roughly 1-2% candidate-side run bias. Therefore:

• Do not interpret the timestamp numbers as meaningful warm wins; they are control-sized.
• Duration parse direction is not stable enough for a steady-state claim; its interval includes zero.
• Duration deserialization improves in every replicate and remains the only defensible warm signal, roughly 6-8% after considering control drift.
• There is no warm allocation improvement or regression.

Semantic coverage

The generated regex is compared directly with the previous compiled regex for:

• match success, match value/index/length;
• named and numbered group sets;
• group and capture success/value/index/length/count;
• RegexOptions.Compiled, infinite timeout, and right-to-left state;
• en-US, tr-TR, and ar-SA current cultures;
• ASCII versus Unicode digits, casing, optional captures, $ final-newline behavior;
• valid fractional timestamps, +18:00/-18:00, invalid dates, -00:00, and range overflow;
• duration leading zeros, unsupported plus sign, existing negative-duration normalization rejection, fractional limits, and range overflow;
• 100,000-character malformed/adversarial inputs;
• exact downstream output tuples, exception types, and error messages.

The pattern literals, options, and downstream parsing logic are otherwise unchanged. The regex accessors widen from private fields to internal methods solely so the friend test assembly can perform direct differential checks; there is no public API change.

AOT and trimming boundary

The emitted compiler source contains sealed generated Regex subclasses and specialized runners for both patterns; generated source SHA-256: 4d7f30e982f5c4857eeceae284a129beb4a2e7c759e590dd2388c5e4a9d79569.

The byte-identical regex probe source publishes and passes differential behavior under the installed .NET 10 NativeAOT toolchain. Publishing the net11 probe was attempted but the exact preview.6 runtime/NativeAOT packs are no longer available from configured feeds. Therefore this is not a claim that the full net11 gRPC package was NativeAOT-published. The narrower claim is that source generation removes runtime regex parsing/dynamic code generation and is structurally friendlier to trimming/AOT than runtime compilation.

Provenance and retained evidence

• Parent product source: f297235c48c2cffc8fb61ff41803d709b30ddab2.
• GeneratedRegex product commit: c50f3fc2506f461298dc6b6d2e3db07c6682cccf.
• Final branch with benchmark controls: 3cd8f078952eb2aa3197b0466dca14f3b0565ae1.
• Parent is an ancestor of the product commit.
• The final benchmark did not exist at the parent, so its exact source was overlaid untracked into a detached parent worktree. Parent and candidate benchmark SHA-256 are both eed49381a6f7d65558d63acbb4b940e67c42bfdf2b659f8ed8a7c4d0e075cedf.
• Clean product DLLs: parent 52c461b1160e027febbfdb4d05b26d8e6836e0fb69ea85623c9939438939606e (142,848 B); candidate 66f132ea23859374213ebffd3178b846638144e8cf6ee183dddb189138fe0fe1 (147,456 B).
• Cold sample CSV contains 800 rows: 100 per implementation for each of four paths. SHA-256 c56485cd6abea94e8485034b982a020935701b4a90b47bfa03a584cd6d3793d5.
• Evidence bundle manifest contains 104 files; manifest SHA-256 550f284d72b4809be6ad8e1b61729283a43c9fd9b163f24f996080229dd861b2.
• BDN logs and Markdown/CSV/JSON exporters are retained for all ten final runs. The exact commands and environment are recorded with the bundle.

The evidence bundle is local session evidence and is not committed to this source-only PR. Canonical tables and hashes are included here so the coordinator can archive the raw bundle separately.

Validation

• ./src/Grpc/build.sh -test: succeeded, 0 warnings, 0 errors.
• Microsoft.AspNetCore.Grpc.JsonTranscoding.Tests: 340 passed, 0 failed.
• Microsoft.AspNetCore.Grpc.JsonTranscoding.IntegrationTests: 24 passed, 0 failed.
• Final benchmark project: builds successfully for net11/osx-arm64.
• PR CI is running; this draft will remain WIP regardless of CI state.

Tradeoffs and alternatives

• Trade: +4,608 B on-disk DLL size for lower first-use time/allocation and no runtime regex compilation setup.
• Keeping RegexOptions.Compiled fields preserves smaller output but retains the measured cold cost.
• A manual timestamp/duration parser could improve warm performance further, but would be a larger semantic rewrite and is intentionally out of scope.
• RegexOptions.NonBacktracking was not introduced because that would change engine/capture behavior; this change preserves existing options and semantics.

Status

Still WIP. Do not review. Do not mark ready.

[artl93]

Final experiment evidence update — still WIP

The preliminary source-toggle evidence has been replaced with detached-SHA evidence and the PR body now contains the canonical cold and five-run crossover BDN tables.

Corrections made after skeptical review:

• The broad warm-throughput claim is withdrawn. Timestamp results are control-sized, duration parse changes sign in one replicate, and only duration deserialization is stable across all five runs.
• The 2.02-10.41x inner first-operation ratios are explicitly diagnostic timings that include reflection/type-init/JIT effects; the defensible end-to-end result is the 3.8-4.0 ms (9.3-10.9%) complete-process first-use reduction.
• Steady-state allocations are unchanged; 52-58 KiB savings are one-time cold current-thread allocations.
• NativeAOT wording is bounded to the exact-pattern .NET 10 smoke. Full net11 package NativeAOT was not validated because the exact preview packs are unavailable.
• Size is reported as +4,608 B on-disk DLL size, not IL.

Retained evidence: 800 cold sample rows (c56485cd6abea94e8485034b982a020935701b4a90b47bfa03a584cd6d3793d5), ten final BDN runs with Markdown/CSV/JSON, clean product binary hashes, emitted regex source, commands, environment, test logs, and a 104-file manifest (550f284d72b4809be6ad8e1b61729283a43c9fd9b163f24f996080229dd861b2).

Local validation: full gRPC build succeeded with 0 warnings/errors; 340 unit and 24 integration tests passed. PR CI is mostly green and still running.

This PR intentionally remains DRAFT / WIP. DO NOT REVIEW and do not mark it ready.