WIP: DO NOT REVIEW — ResponseCaching: avoid one-element string[] allocation on vary-by lookup

[artl93]

I am running an experiment - full analysis to follow. Contact @artl93

---

Status: DRAFT / WIP — DO NOT REVIEW. This PR is an in-progress performance experiment. Numbers below are preliminary and will be replaced by a canonical, reproducible A/B before any review is requested.

Summary (preliminary)

Response caching vary-rule lookups currently allocate one single-element string[] per lookup. This change removes that array. Preliminary estimate: ~32 B/lookup eliminated (a length-1 string[] on 64-bit), for a 0 B allocation on the focused lookup-key path.

Where the array comes from

ResponseCachingMiddleware.TryServeFromCacheAsync calls the internal IResponseCachingKeyProvider.CreateLookupVaryByKeys(context) and foreach-es the result to probe the cache. The implementation was:

public IEnumerable<string> CreateLookupVaryByKeys(ResponseCachingContext context)
    => new string[] { CreateStorageVaryByKey(context) };

i.e. it wraps a single computed storage key in a heap string[1] purely to satisfy the IEnumerable<string> contract, even though the production implementation only ever yields one key. The experiment changes the internal API to return Microsoft.Extensions.Primitives.StringValues (a struct that holds a single string with no array allocation and enumerates via a struct enumerator), preserving the exact lookup order/behavior.

Vary-by semantics (why this is safe)

The lookup key is built from CachedVaryByRules:

• Vary-by-header: for each configured header name, the request's header values are read, sorted ordinally, and appended (H group). Header names preserve case; values compared ordinally; multiple values joined by a sub-delimiter.
• Vary-by-query-key: analogous Q group; query key names normalized upper-invariant, values sorted ordinally. VaryByQueryKeys = ["*"] varies by all query keys (aggregated case-insensitively by key, values ordinal).
• Empty Headers and empty QueryKeys → key is just the VaryByKeyPrefix (the rules GUID), no string builder work.

Because the production provider computes exactly one storage key and the middleware probes keys in enumeration order, returning that single key directly (vs. a 1-element array) is order- and behavior-preserving. The interface is internal, so there is no public API change.

Applicability (cache hit/miss)

CreateLookupVaryByKeys runs only when the base-key entry resolves to a CachedVaryByRules — i.e. the resource was previously stored with a Vary header or VaryByQueryKeys. It fires on the vary-resolution step of such requests (both eventual hits and vary-misses), once per qualifying request.

Known gaps in this preliminary post

• No canonical A/B yet. The repo benchmark src/Middleware/perf/ResponseCaching.Microbenchmarks/ResponseCachingBenchmark.cs currently exercises a non-vary ServeFromCache path, so it does not cover this code. A rigorous, hashed allocation A/B across vary scenarios (no-vary control, single/multiple Vary headers, query-key variation, wildcard *, empty/invalid, hit/miss/store, casing, multi-values, small/large key sets) is in progress and will be posted here.
• Absolute saving is small; justification for hot cached endpoints will accompany the final data.

Preliminary — full analysis, canonical before/after tables, and tests to follow.

[artl93]

Preliminary analysis — deterministic allocation A/B (still WIP)

Correction to the PR body estimate: the preliminary body says ~32 B/lookup. Rigorous A/B measurement shows the real saving is 64 B/call, not 32. The old IEnumerable<string> path allocated two objects per enumerated lookup, not one:

1. the single-element string[1] itself (32 B), and
2. a boxed SZGenericArrayEnumerator produced by ((IEnumerable<string>)array).GetEnumerator() when the middleware foreach-es it (32 B).

StringValues is a readonly struct wrapping the one key string; it enumerates via a struct enumerator, so both allocations disappear → 0 B on the lookup path.

Where the array came from

ResponseCachingMiddleware.TryServeFromCacheAsync calls _keyProvider.CreateLookupVaryByKeys(context) and foreach-es the result to probe the store for a cached vary-by response. The old implementation built new string[] { CreateStorageVaryByKey(context) } purely to hand a single key to that foreach. The single storage key string is still allocated (that is the actual cache key and is unchanged); only the array + enumerator wrapper around it is removed.

Semantics preserved

• Return type changed on the internal IResponseCachingKeyProvider only (no public API surface).
• The yielded value is byte-for-byte the same CreateStorageVaryByKey(context) string, so lookup order and equality are identical (there was only ever one element).
• New unit tests assert exactly this across: no-vary (prefix-only), vary-by-header, vary-by-query-key, wildcard * query, header+query combined, and delimiter-injection validation.

Before / after (deterministic allocation, per call)

GC.GetAllocatedBytesForCurrentThread delta, Iters=20000, min-of-7 trials after 2000-iter warmup, single-threaded. Metric = bytes(CreateLookupVaryByKeys enumerated) − bytes(CreateStorageVaryByKey) to isolate the enumeration wrapper from the shared key-string allocation.

Scenario	A (before) B/call	B (after) B/call	Saved B/call
no-vary-control	64	0	64
1-header-1-value	64	0	64
1-header-multi-value	64	0	64
1-header-absent	64	0	64
3-headers	64	0	64
1-querykey	64	0	64
wildcard-query-star	64	0	64
header-plus-query	64	0	64
large-20-headers	64	0	64

The saving is a constant 64 B independent of vary-rule size, because it is the fixed array-header + boxed-enumerator cost, not a per-key cost.

Provenance

• A (before): f297235c48c2cffc8fb61ff41803d709b30ddab2 (parent) · B (after): 5765da25e9997cbc92d5cbf1035924af34ed81e4
• Host: Apple M5 Pro, macOS 26.5.2 arm64, .NET SDK 11.0.100-preview.6.26318.108, net11.0 Release
• Harness sha256 994828a2…a2a24a0 (identical bytes at both revisions).

When this is hit

CreateLookupVaryByKeys runs on every request that reaches a cached resource carrying vary-by rules (Vary header and/or VaryByQueryKeys) — i.e. the cache-hit probe path. Non-vary cached responses and non-cacheable requests take a different branch. So this is a hot-path win only for endpoints that both use ResponseCachingMiddleware and vary their cache. The absolute per-call number is small; the value is zero-allocation determinism on a genuinely per-request path for those endpoints, at no behavioral cost.

Limitations / review risk

• Deterministic allocation is primary evidence; no full BDN wall-clock run is included (the ResponseCaching.Microbenchmarks project has no InternalsVisibleTo, so a key-provider-level BDN benchmark would require a coupling change, and the existing ServeFromCache benchmark sets no Vary header so it never exercises this path). Time-delta is negligible and not the point; allocation elimination is.
• Risk is minimal: internal-only API, single-element semantics unchanged, covered by new + existing tests (267 pass).

Still WIP — do not review.

[artl93]

Precision note + status

Minor correction to my prior comment for accuracy: the second 32 B in the old path is a heap-allocated reference-type enumerator (SZGenericArrayEnumerator<string>, returned by ((IEnumerable<string>)array).GetEnumerator()), not a boxed value type. Decomposition of the measured 64 B/call: string[1] object = 32 B + SZGenericArrayEnumerator<string> instance = 32 B. StringValues removes both by using a struct value + struct enumerator.

Everything else in the analysis stands and is backed by the deterministic A/B table above.

This PR remains WIP — DO NOT REVIEW. It is intentionally left as a draft. Contact @artl93.