Box-free POCO read fast path (#509)

[alex-clickhouse]

What

Makes ClickHouseClient.QueryAsync<T> materialize rows without per-value boxing, and adds multiple read representations per column. Mirrors the box-free write path (#434).

Today the POCO read path boxes twice per value-type cell: once in Read() (into the shared object[] row buffer) and once unboxing in the compiled Action<T,object> MapTo<T> setter. QueryAsync<T> now compiles a per-column delegate that reads each value straight from the stream into the strongly-typed property, eliminating both.

How

• ITypedReader<T> (analog of ITypedWriter<T>) implemented across every scalar/value type. Each type's boxed Read() delegates to its typed ReadValue, so the two paths are byte-identical by construction.
• PocoReadExpressionFactory dispatches via ITypedReader<targetClrType>, transparently unwrapping Nullable(T) and LowCardinality(T).
• PocoTypeRegistry compiles + caches a per-wire-column delegate array, keyed by (T, wire-column signature). Per-column mixing: box-free typed read, a per-column boxed fallback for composites (byte-identical to MapTo, with fail-fast validation), or a discard for unmapped columns — so one composite column doesn't disable the fast path for its scalar siblings.
• Shared PocoColumnAssignment keeps MapTo and the fallback's validation/errors identical.
• A read value converter or unregistered type falls back entirely to the existing path.

Multiple read representations (selected by property type; QueryAsync<T> only — MapTo<T> keeps strict exact-type rules)

Column	Property types
DateTime / DateTime64 / Date	DateTime, DateTimeOffset, DateOnly
String / FixedString	string, byte[]
Decimal	decimal, ClickHouseDecimal
Enum8 / Enum16	string (label), int (stored numeric value)
Int128 / UInt128	BigInteger, or native System.Int128/UInt128 (net8+)

Results (PocoReadBenchmark, long/string/double, local WSL)

Count	Before	After	Δ	Gen0/1k
100k	12.23 MB	7.66 MB	−37%	2000→1000
500k	61.06 MB	38.17 MB	−38%	10000→6000

GetValue/GetFieldValue<T> (ADO contract) unchanged.

Testing

• New PocoReadFastPathTests (23 tests): all-scalar parity vs the boxed path, every multi-type binding, Nullable variants, LowCardinality, per-column mixing with composite fallback, unmapped-column alignment, fail-fast validation, unregistered-type error.
• Enum coverage: Enum8/Enum16 read as both label and int over the same column (negative and beyond-sbyte values pin each wire width), Nullable(Enum8) into int?, an int-property insert round-tripping through a real Enum8 column, and the fail-fast error for a non-nullable int on a Nullable(Enum) column (which needs int?).
• Sub-agent review: no blockers, byte-identity confirmed across all ~25 types.

[github-actions]

Triage

Category: perf • Risk: high

Summary
Adds a box-free POCO read fast path for ClickHouseClient.QueryAsync<T>. Today the POCO read path boxes every value-type column into object[] (in Read()) and immediately unboxes it (in the MapTo<T> compiled setter); this PR eliminates both operations by compiling per-column delegates (ITypedReader<T>) that read each value straight from the binary stream into the typed property, bypassing the shared row buffer. The PR also introduces multiple CLR-type bindings per column (DateTime→DateTimeOffset/DateOnly, String→byte[], Decimal→ClickHouseDecimal, Int128→System.Int128), and a new Map(K,V) → List<KVP<K,V>>/KVP<K,V>[] materialization path. Reported allocation reduction: ~38% in a 500k-row/3-column POCO read.

What this impacts

• ClickHouse.Driver/Types/ — 25+ type files gain ITypedReader<T> implementations; the existing Read() on each now delegates to ReadValue(), changing the boxed read path as well
• ADO/Readers/ClickHouseDataReader.cs — two new internal methods (TryGetRowMaterializer<T>, TryMaterializeNextRow<T>) drive the fast path from the stream
• ClickHouseClient.QueryAsync<T> — branching logic selects fast vs. fallback per result shape; observable behaviour changes (new type coercions, Map→KVP collection, per-column mixing)
• Poco/ — three new source files: PocoReadExpressionFactory, PocoColumnAssignment, MapMaterializer; delegate compilation and caching in PocoTypeRegistry

Concerns

• Type system — High rule fires: 25+ files in Types/ modified; read path split into typed fast and boxed fallback branches. Byte-identity is asserted by construction but any divergence in stream alignment would silently corrupt all subsequent columns — exactly the class of bugs integration tests can miss if column ordering in test queries doesn't hit edge cases.
• Cross-module refactor — High rule fires: touches ADO/ (DataReader), Types/ (25+ scalar types), and Utility/-level ClickHouseClient.cs — three or more hot-path modules.
• TryMaterializeNextRow stream state: the fast path sets hasCurrentRow = false only on EOF, not on success; the state is not consumed by this path, but it should be verified that no code path can interleave MapTo<T> and TryMaterializeNextRow on the same reader and see stale state.
• Reflection in delegate compilation: PocoReadExpressionFactory.TryBuildTypedRead calls typeof(ITypedReader<>).MakeGenericType(clrType) and .GetMethod(...) — acceptable because results are cached per (T, wire-column signature), but adds GC pressure on first use per distinct POCO shape.
• Behavioural surface expansion beyond perf: new type coercions (DateOnly, DateTimeOffset, byte[], KVP collections) are user-visible feature additions bundled into a perf PR; reviewers should check whether these belong in a separate issue/PR or whether the scope is intentional.

Required reviewer action

• PR body must include an architectural description before review.

[codecov]

Codecov Report

❌ Patch coverage is 84.30657% with 43 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...lickHouse.Driver/Poco/PocoReadExpressionFactory.cs	81.13%	3 Missing and 7 partials ⚠️
ClickHouse.Driver/Poco/PocoTypeRegistry.cs	83.92%	8 Missing and 1 partial ⚠️
...ckHouse.Driver/ADO/Readers/ClickHouseDataReader.cs	76.47%	7 Missing and 1 partial ⚠️
ClickHouse.Driver/Types/MapType.cs	76.19%	2 Missing and 3 partials ⚠️
ClickHouse.Driver/Poco/PocoColumnAssignment.cs	72.72%	1 Missing and 2 partials ⚠️
ClickHouse.Driver/Types/AbstractDateTimeType.cs	71.42%	2 Missing ⚠️
ClickHouse.Driver/Types/FixedStringType.cs	60.00%	2 Missing ⚠️
ClickHouse.Driver/Poco/MapMaterializer.cs	92.85%	0 Missing and 1 partial ⚠️
ClickHouse.Driver/Types/DateTimeType.cs	66.66%	1 Missing ⚠️
ClickHouse.Driver/Types/DecimalType.cs	93.33%	1 Missing ⚠️
... and 1 more

📢 Thoughts on this report? Let us know!

[Copilot]

Pull request overview

This PR introduces a new box-free materialization fast path for ClickHouseClient.QueryAsync<T>(), reducing per-cell allocations by reading values directly from the response stream into POCO properties. It extends the type system with typed read support (ITypedReader<T>) and adds selective per-column fallback so composite columns don’t disable fast reads for scalar siblings.

Changes:

• Add ITypedReader<T> and implement typed, non-boxing reads across scalar/value ClickHouse types (and selected multi-representation reads like DateTimeOffset, DateOnly, byte[], native Int128/UInt128 on .NET 8+).
• Implement a POCO read fast-path pipeline: expression factory dispatch (PocoReadExpressionFactory), assignment rule sharing (PocoColumnAssignment), and cached per-shape row materializers (PocoTypeRegistry).
• Update QueryAsync<T>() to use the new fast path when available, and add a dedicated test suite validating parity and binding behavior.

Reviewed changes

Copilot reviewed 39 out of 39 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
RELEASENOTES.md	Document allocation/perf impact and new QueryAsync<T> binding capabilities.
CHANGELOG.md	Changelog entry for the new POCO read fast path and multi-representation bindings.
ClickHouse.Driver/Types/ITypedReader.cs	Introduce ITypedReader<T> to enable non-boxing deserialization.
ClickHouse.Driver/Types/AbstractDateTimeType.cs	Add typed reads for DateTime/DateTimeOffset/DateOnly and centralize canonical boxed read.
ClickHouse.Driver/Types/DateType.cs	Adapt Date decoding to the new AbstractDateTimeType read model.
ClickHouse.Driver/Types/Date32Type.cs	Adapt Date32 decoding to the new AbstractDateTimeType read model.
ClickHouse.Driver/Types/DateTimeType.cs	Split instant decoding and add typed DateTimeOffset support via base hooks.
ClickHouse.Driver/Types/DateTime64Type.cs	Share wire read between DateTime and DateTimeOffset typed reads.
ClickHouse.Driver/Types/AbstractBigIntegerType.cs	Add typed BigInteger read path to support fast materialization.
ClickHouse.Driver/Types/Int128Type.cs	Add native Int128 typed read on .NET 8+.
ClickHouse.Driver/Types/UInt128Type.cs	Add native UInt128 typed read on .NET 8+.
ClickHouse.Driver/Types/DecimalType.cs	Add typed reads for decimal and ClickHouseDecimal, keeping boxed behavior canonical.
ClickHouse.Driver/Types/StringType.cs	Provide both string and byte[] typed representations while preserving boxed mode selection.
ClickHouse.Driver/Types/FixedStringType.cs	Provide both string and byte[] typed representations while preserving boxed mode selection.
ClickHouse.Driver/Types/UuidType.cs	Add typed Guid read for fast POCO assignment.
ClickHouse.Driver/Types/IPv4Type.cs	Add typed IPAddress read for fast POCO assignment.
ClickHouse.Driver/Types/IPv6Type.cs	Add typed IPAddress read for fast POCO assignment.
ClickHouse.Driver/Types/BooleanType.cs	Add typed bool read for fast POCO assignment.
ClickHouse.Driver/Types/BFloat16Type.cs	Add typed float read for fast POCO assignment.
ClickHouse.Driver/Types/Int8Type.cs	Add typed sbyte read for fast POCO assignment.
ClickHouse.Driver/Types/Int16Type.cs	Add typed short read for fast POCO assignment.
ClickHouse.Driver/Types/Int32Type.cs	Add typed int read for fast POCO assignment.
ClickHouse.Driver/Types/Int64Type.cs	Add typed long read for fast POCO assignment.
ClickHouse.Driver/Types/UInt8Type.cs	Add typed byte read for fast POCO assignment.
ClickHouse.Driver/Types/UInt16Type.cs	Add typed ushort read for fast POCO assignment.
ClickHouse.Driver/Types/UInt32Type.cs	Add typed uint read for fast POCO assignment.
ClickHouse.Driver/Types/UInt64Type.cs	Add typed ulong read for fast POCO assignment.
ClickHouse.Driver/Types/Float32Type.cs	Add typed float read for fast POCO assignment.
ClickHouse.Driver/Types/Float64Type.cs	Add typed double read for fast POCO assignment.
ClickHouse.Driver/Types/Enum8Type.cs	Add typed string read to avoid boxing for enum materialization.
ClickHouse.Driver/Types/Enum16Type.cs	Add typed string read to avoid boxing for enum materialization.
ClickHouse.Driver/Poco/PocoReadExpressionFactory.cs	Build per-column, typed read expressions driven by ITypedReader<T>, with Nullable/LowCardinality handling.
ClickHouse.Driver/Poco/PocoColumnAssignment.cs	Centralize assignment validation/error text used by both MapTo<T> and fast-path fallback.
ClickHouse.Driver/Poco/PocoTypeRegistry.cs	Cache and build per-shape per-column row materializers with mixed fast/fallback/discard behavior.
ClickHouse.Driver/ADO/Readers/ClickHouseDataReader.cs	Expose internal fast-path hooks (TryGetRowMaterializer, TryReadRow) and reuse shared assignment logic.
ClickHouse.Driver/ClickHouseClient.cs	Switch QueryAsync<T> to use fast path when available, otherwise retain the existing boxed loop.
ClickHouse.Driver.Tests/Copy/PocoReadFastPathTests.cs	Add coverage for parity, multi-type bindings, Nullable/LowCardinality, mixed composite fallback, and failure modes.