[API Proposal]: Create independent shared-style ArrayPool instances

[joaocpaiva]

Background and motivation

ArrayPool<T>.Shared provides excellent throughput for highly concurrent servers, but it is process-wide. Unrelated components and workload classes therefore share capacity, retention, contention, diagnostics, and reuse behavior. Some services need separately owned pools for scenarios such as ordinary byte buffers, character buffers, unusually large byte buffers, and sensitive cryptographic material.

The supported way to create an independent pool is ArrayPool<T>.Create(...). However, that factory currently returns the configurable implementation, whose concurrency and retention strategy differs substantially from the implementation behind ArrayPool<T>.Shared. In particular, a high-concurrency server choosing isolation must give up the shared pool's scalable thread-local/per-core caching and memory-pressure-aware trimming behavior. This can make the supported isolated design materially less attractive under load.

The request is for a supported way to create an independently owned pool with performance and adaptive trimming characteristics comparable to ArrayPool<T>.Shared. The exact cache topology and trimming algorithm should remain runtime implementation details. Each returned pool must own independent cache and partition state so renting from one pool does not consume another pool's retained capacity.

A concrete use case is an API gateway that separates ordinary byte buffers, JSON character buffers, large payload buffers, and sensitive buffers containing material such as raw access tokens. We have encountered application lifetime/ownership bugs involving the process-wide shared pool that resulted in buffer corruption and stale sensitive data being observed by unrelated pooling consumers. Dedicated pools do not correct such bugs or eliminate the need to clear sensitive data, but they provide defense in depth: only code with access to the sensitive pool normally receives its retained arrays, reducing the cross-workload blast radius of a mistake. They also make retention attributable and prevent a burst of large-payload work from displacing ordinary request buffers.

This is also a concrete use case requested by the broader ArrayPool discussion in #52098.

API Proposal

One possible minimal shape is:

namespace System.Buffers;

public abstract partial class ArrayPool<T>
{
    public static ArrayPool<T> CreateShared();
    public static ArrayPool<T> CreateShared(int maxArrayLength);
}

CreateShared is a tentative name: it means a pool intended to be shared by callers of that returned instance, not the process-wide Shared singleton. A less ambiguous name would be welcome during API review.

The parameterless overload creates an independent pool using runtime-selected defaults. The overload limits the largest array retained by the pool, in elements; larger requests may allocate but are not retained, consistent with the existing Create(int maxArrayLength, int maxArraysPerBucket) concept.

The public contract should promise an independent, thread-safe, adaptively trimmed pool suitable for high concurrency, rather than expose thread-local slots, per-core partitions, bucket counts, or specific GC notification mechanisms. Those details should remain free to evolve.

API Usage

using System.Buffers;

internal static class ServiceArrayPools
{
    // Limits are element counts, not byte counts.
    internal static ArrayPool<byte> DefaultBytes { get; } =
        ArrayPool<byte>.CreateShared(1024 * 1024);

    internal static ArrayPool<char> DefaultChars { get; } =
        ArrayPool<char>.CreateShared(1024 * 1024);

    internal static ArrayPool<byte> LargeBytes { get; } =
        ArrayPool<byte>.CreateShared(8 * 1024 * 1024);

    internal static ArrayPool<byte> SensitiveBytes { get; } =
        ArrayPool<byte>.CreateShared(1024 * 1024);
}

byte[] buffer = ServiceArrayPools.DefaultBytes.Rent(minimumLength);
try
{
    Process(buffer);
}
finally
{
    ServiceArrayPools.DefaultBytes.Return(buffer);
}

byte[] tokenBuffer = ServiceArrayPools.SensitiveBytes.Rent(tokenLength);
try
{
    WriteRawToken(tokenBuffer);
}
finally
{
    ServiceArrayPools.SensitiveBytes.Return(tokenBuffer, clearArray: true);
}

The instances are long-lived service-level pools, not pools created per operation. Callers select a pool based on the workload's retention, sizing, and data-isolation policy.

This API would not change data-clearing semantics. Sensitive callers must still use clearArray: true or clear the initialized region before returning an array. The dedicated pool limits ordinary cross-workload reuse; clearing protects against reuse within that pool and against other lifetime bugs.

Alternative Designs

1. Continue using ArrayPool<T>.Shared. This preserves throughput but provides no ownership or workload isolation and no per-scenario maximum retained length.

2. Use ArrayPool<T>.Create(maxArrayLength, maxArraysPerBucket). This provides independent ownership and tuning, but currently selects the configurable implementation rather than the scalable, adaptively trimmed implementation used by Shared.

3. Change the existing Create implementation globally. This could improve independent pools without adding API, but may alter established retention, sizing, and performance behavior for existing applications. It also leaves no explicit way to request the server-oriented policy.

4. Add an options-based factory, for example Create(ArrayPoolOptions options) with a strategy value and maximum retained length. This is more extensible, but a public strategy enum risks exposing implementation categories that the runtime should be able to evolve. It may be preferable if more policy controls are expected.

5. Implement a custom pool in each application. Reproducing the runtime's high-concurrency caching, GC integration, trimming, and future optimizations is complex and likely to diverge over time.

6. Instantiate the internal shared implementation via reflection. This is unsupported and does not reliably provide isolation because internal static/thread-static state may be shared by closed generic type rather than owned by an instance.

Risks

Multiple shared-style pools can retain more aggregate memory than the single process-wide pool, particularly if every instance has independent thread-local slots. The implementation must keep per-instance overhead bounded and participate in memory-pressure trimming. Documentation should recommend long-lived, small-in-number instances rather than per-request creation.

The name CreateShared may be confused with the Shared singleton. API review should choose terminology that communicates independent ownership without making current internals contractual.

A maximum retained length is useful for workload isolation, but additional tuning knobs such as partition counts or arrays per partition would expose implementation details and can encourage configurations that regress memory or throughput. Runtime-selected defaults are preferable unless concrete evidence justifies more controls.

Pool isolation is defense in depth, not a security boundary. It reduces normal cross-workload buffer reuse but does not prevent stale sensitive data from being rented again within the same pool, does not fix use-after-return or double-return bugs, and does not replace clearing sensitive data. Returning an array to the wrong pool is already caller misuse and should not require expensive provenance tracking.

Finally, maintaining two optimized code paths could increase runtime complexity. Ideally the process-wide Shared property and independent instances would use the same implementation with instance-owned state, minimizing behavioral drift.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-buffers
See info in area-owners.md if you want to be subscribed.

[EgorBo]

I think the answer to your proposal is still #52098 (comment)

ArrayPool proved itself as an unreliable source of use-after-free/double-free bugs, I don't think we want to continue devoloping it in the current shape. It needs some redesign paired with an advanced ownership system to be fully memory safe.