TCP I2: Add Array(T) support

[alex-clickhouse]

Stacked on tcp/epic-i1-nullable. Implements Array(T) for the TCP/Native client and — because the ergonomic write path needed it — finalizes the codec write interface.

Write interface: IColumnWriteState

Array is the first codec whose state prefix and body must agree on work derived from the values, so IColumnCodec grows a per-slice scratch phase:

• BeginWrite(column, start, length) → IColumnWriteState (default null) — computed once per slice, handed to new state-aware WriteStatePrefix/WriteColumn overloads, disposed by the caller after the body. A codec needing no scratch returns null and those overloads fall back to the self-contained ones, so every existing codec is unchanged.
• WriteStatePrefix now takes the column slice. Each block's prefix reflects only that block's rows, and a type whose prefix is data-dependent (a future Dynamic's runtime type list) must see the same rows the body will write. Fixed-prefix codecs ignore the arguments; NullableColumnCodec forwards them to its inner unchanged.
• BlockWriter drives BeginWrite → prefix → body → dispose per column. ColumnCodecExtensions.WriteFull does the same for callers outside the block layer.

Array(T)

• ArrayColumnCodec<TElement> — delegates the state prefix to the inner codec, then reads/writes a per-row num_rows × UInt64 LE offsets stream (cumulative element ends) followed by the inner type's encoding for every element concatenated end-to-end. An empty column is zero bytes.
• ArrayValueColumn<TElement> — a flat inner column plus per-row offsets, each row materialized lazily as TElement[]. This is the wire's own layout, so it doubles as a zero-copy write source when a read-back column is re-inserted.
• Ergonomic (jagged) write builds no flat buffer. Per-row lengths are summed once into slice-relative offsets, then the inner is driven by its shape:
  • a leaf inner — the fixed-width types and String, i.e. those implementing the new internal ISpanWritableCodec<T> — takes each row as a contiguous run (a bulk blit for fixed-width), straight from the source arrays;
  • a sectioned inner (Nullable's null-map, LowCardinality's dictionary, a nested Array's offsets) has to see every element as one column so each section is emitted once spanning the whole run — driving it row by row would interleave those sections and corrupt the stream. It gets ConcatColumn<T>, a lazy flattening view (row cursor, binary-search fallback), not a copy.
• The registry pipeline is non-generic (codecs come back as IColumnCodec and the inner element type arrives as a runtime Type), so a small non-generic ArrayColumnCodec factory closes the generic codec over the inner's element type — one reflective instantiation per element type, cached as a constructor delegate. The single (IColumn<TElement>) cast at the read boundary is the only cast this leaves. Composites recurse: Array(Array(T)), Array(Nullable(T)).

New public API: IArrayColumn<TElement>

The columnar read surface of a decoded array column. Values/the indexer allocate a TElement[] per row; this exposes the flat wire layout instead — InnerValues + Offsets, where row i is InnerValues.Slice(Offsets[i], Offsets[i + 1] - Offsets[i]) — so a read-only caller iterates without allocating a row array. Inner hands back the flat inner column so a nested composite (Array(Array(T)), later Array(Tuple(...))) can be walked all the way down without materializing intermediate levels. All borrowed views, valid only while the owning block is alive; Offsets is bounded to RowCount + 1 rather than exposing the rented buffer's length.

Insert memory bound

Blocks split by row count alone, so peak client memory during a large insert is held by the between-column flush backstop — which is now caller-tunable via a new maxSendBufferBytes parameter on the internal InsertAsync (default BlockWriter.DefaultFlushThresholdBytes, validated positive). A single column larger than the cap still buffers in full. Inserted before handlers, so it is source-breaking for positional callers of that internal API.

Safety / correctness

• Read: offsets validated monotonic and in-int-range as they are decoded; malformed or oversized offsets throw ClickHouseProtocolException. The offsets stream's own byte length is checked against Array.MaxLength before renting.
• Write: the flattened element count is guarded against Array.MaxLength rather than truncating long→int. A null row throws an ArgumentException naming the row and pointing at Array(Nullable(T)).
• The read path's pooled buffers (offsets, wire scratch) are returned exactly once, and ownership transfers to the column only on success — the read-boundary cast sits inside the try, so an element-type mismatch leaks neither the buffer nor the inner column. The write path pools nothing.

Tests

• Unit (ArrayColumnCodecTests) — trimmed to what a server round-trip cannot reach: corruption rejection (non-monotonic offsets, an offset past int.MaxValue), an empty column consuming no bytes, row access past RowCount, the short-offsets constructor guard, CanWrite/ElementType/NullPlaceholder, and the type-resolution errors.
• Integration — an Array(T) round-trip case for every supported inner type (all integer widths, enums, floats, bool, string, dates/datetimes/DateTime64, UUID, IPs, all decimal widths, intervals, BFloat16, Time/Time64), plus Array(Array(T)), Array(Nullable(T)) and Array(Array(Nullable(T))); jagged and dense columns split across blocks; a read-back dense column re-inserted; a server-generated array via range(); and the IArrayColumn zero-copy surface including nested recursion.
• Nullable(Array(T)) is server-rejected, so Array is the one type without a Nullable-wrapped case; Array(Nullable(T)) composes nullability the other way instead.
• One pre-existing test that used Array as its "unsupported type" example is repointed at Tuple (still unsupported).

Full TCP suite green against a live server: 741 tests (454 unit + 287 integration).

🤖 Generated with Claude Code

[github-actions]

Triage

Category: feature • Risk: high

Summary
This PR adds Array(T) read/write support to the TCP/Native client (epic I2). It introduces ArrayColumnCodec<TElement> (binary offsets + element stream codec), ArrayValueColumn<TElement> (the dense wire-shaped column), ConcatColumn<T> (a lazy jagged-to-flat view for sectioned inner codecs), and IArrayColumn<TElement> (zero-copy public read interface). It also extends IColumnCodec with four new members — BeginWrite, two new WriteStatePrefix overloads, and a new WriteColumn overload — replacing the old zero-arg WriteStatePrefix(writer) signature. BlockWriter is updated to call the new BeginWrite/state pattern, ClickHouseTcpConnection.InsertAsync gets a new maxSendBufferBytes parameter, and NullableColumnCodec.WriteStatePrefix is updated to the new signature. Tests cover 40+ inner element types, nested/nullable composition, block-splitting, offset validation, and server-generated array reads via live integration.

What this impacts

• ClickHouse.Driver.Tcp/Types/ — new ArrayColumnCodec, ArrayValueColumn, ConcatColumn, IArrayColumn, IColumnWriteState; IColumnCodec interface revised; NullableColumnCodec updated
• ClickHouse.Driver.Tcp/Format/BlockWriter.cs — now drives BeginWrite/state-aware write pattern
• ClickHouse.Driver.Tcp/Protocol/ClickHouseTcpConnection.cs — new maxSendBufferBytes param on InsertAsync
• Public API: new IArrayColumn<TElement> interface exposed; IColumnCodec interface extended

Concerns

• Reflection — ArrayColumnCodec.BuildFactory uses BindingFlags.NonPublic + MakeGenericMethod to close the generic codec; cached per element type, but this is a new use of reflection in the codec-resolution path. High-risk rule: "any use of reflection."
• IColumnCodec interface breaking change — the old WriteStatePrefix(ClickHouseBinaryWriter writer) member is removed and replaced with WriteStatePrefix(writer, column, start, length). Any external implementor that overrode the old signature will compile (it's now unrelated) but silently stop being called. The new signature has a default impl, so compilation doesn't break, but call semantics change for overriders.
• Binary protocol / TCP write path — ArrayColumnCodec owns the offsets stream encoding and the per-element serialization ordering; a bug here would produce a silently corrupt stream accepted by some server versions.
• Stacked PR — based on tcp/epic-i1-nullable (I1); the diff should be reviewed with that base branch loaded, as changes from I1 may appear in this diff.
• ConcatColumn binary search mutable state — hintRow is a mutable field on an instance handed to the inner codec's BeginWrite; concurrent read would race, but the write path is single-threaded, so this is acceptable — worth confirming reviewers agree.

Required reviewer action

• PR body must include an architectural description before review. (Body is present and detailed; confirm it covers the IColumnCodec signature migration rationale and the reflection caching strategy explicitly.)

[Copilot]

Pull request overview

Adds ClickHouse Array(T) support to the TCP/Native client type system, enabling native-format read/write of array columns (including nested arrays and arrays of nullable elements) and extending the TCP test suite to cover these composites.

Changes:

• Register Array(T) in the TCP ColumnCodecRegistry and implement ArrayColumnCodec (offsets + flattened values stream).
• Introduce ArrayValueColumn<TElement> as the dense (wire-layout) decoded column shape for zero-copy writes.
• Add dedicated unit tests for ArrayColumnCodec and expand TCP insert round-trip coverage to include Array(T) across supported inner types.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
ClickHouse.Driver.Tcp/Types/ColumnCodecRegistry.cs	Registers the Array codec factory for recursive type resolution.
ClickHouse.Driver.Tcp/Types/Codecs/ArrayColumnCodec.cs	Implements Array(T) read/write, measuring, and type bridging via cached shapes.
ClickHouse.Driver.Tcp/Types/ArrayValueColumn.cs	Adds a dense array column representation (inner flat column + offsets).
ClickHouse.Driver.Tcp.Tests/Utilities/InsertRoundTripCase.cs	Adds integration round-trip cases for Array(T) across supported element types and compositions.
ClickHouse.Driver.Tcp.Tests/Types/NullableColumnCodecTests.cs	Updates an “unsupported inner” test to use Tuple instead of Array.
ClickHouse.Driver.Tcp.Tests/Types/ColumnCodecRegistryTests.cs	Updates an “unsupported type” test to use Tuple instead of Array.
ClickHouse.Driver.Tcp.Tests/Types/ArrayColumnCodecTests.cs	Adds unit tests covering array read/write, slicing, measuring, element typing, and corruption rejection.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated no new comments.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!