docs: record the #405 late-materialization plan and its retraction

[jdatcmd]

Adds design/ISSUE_405_LATE_MATERIALIZATION.md, the plan-before-code document for #405 item 1 that the issue comments reference by path.

The doc records, in order:

• the multi-agent investigation and the measurement-gated plan,
• the Step 1 gate that reported 41-58% recoverable, and
• the retraction of that gate: it compared count(*) on the batch-fold path against sum(payload) on the row path, two different execution paths. The error was caught by implementing Step 2 and finding its work-done counter read 0.

The corrected like-with-like measurement (row path only, selectivity 1-99%) shows payload cost already scales 4.5x with selectivity because #452 Phase 1a defers payload materialization to surviving rows on the row path. The batch-fold path, the only place lacking position-level deferral, is byval-only and never carries payload aggregates, so there is nothing to defer there. Item 1 closes as done-by-#452; @ChronicallyJD confirmed the same conclusion independently from the source on the issue.

The pre-retraction plan is kept below the retraction, marked superseded, for its verified architecture facts: the no-table-AM-change plumbing verdict, the two decode granularities, and the #452 1a row-path deferral mechanism.

Docs only, no code change.

Closes nothing on its own; the #405 disposition is being settled on the issue.

🤖 Generated with Claude Code

[ChronicallyJD]

The retraction's central architecture fact is refuted; the doc needs it corrected before it becomes the durable record

The row-path facts in this doc verify cleanly against main — I re-checked every
file:line citation at 744b7ca and re-ran the shapes at 99c6a58. The plumbing
verdict, the #452 1a row-path deferral, the byval-only fold gate, and the
decode-is-sequential kill shot all hold. What does not hold is the retraction's
premise, and it is the TL;DR:

pgcolumnar_native_batch_fold — the only place lacking position-level
late-mat — is taken only for aggregates with no payload column
(count(*)). [...] So there is nothing to defer on the fold path: it never
carries payload.

That is an int8 artifact, not an architecture fact. The classifier maps
sum(int8) to COLUMNAR_AGG_SUM_INT8 (returns numeric) and
pgcolumnar_batch_agg_ok rejects it — alongside SUM_NUMERIC, AVG_INT8,
AVG_NUMERIC, and min/max. But it accepts SUM_INT (int2/int4),
SUM_FLOAT (float4/float8), AVG_INT, AVG_FLOAT, and COUNT_COL — all of
which carry payload through the fold gather. The Step-2 counter read 0 because
the fixture's payload was int8, and "every sum(pN) shape reports Batch Fold: no"
was a true observation whose scope was the fixture's type, not the fold path.

Measured on main @ 99c6a58, pg18a assert build (.so md5 b148f00d31d933909f50a06b436022f7), enable_ungrouped_vector_agg = on, serial

shape	Columnar Batch Fold
count(*), count(i4)	yes
sum(i2), sum(i4), sum(f4), sum(f8), avg(i4), avg(f8)	yes — payload-carrying
sum(i8), sum(numeric), sum(i4),sum(i8) mixed, min/max	no

Correctness control: the fold and row arms agree on sum(i4), sum(f8),
count(*) over the same fixture (GUC toggled, same session).

The like-with-like Step-1 measurement the int8 fixture could not run

4M rows, q int4 uniform 1..100 scattered (27/27 chunk groups read, 0 vectors
skipped — asserted), 8 float8 payload columns, PG18 non-assert
(.so md5 fd007021170a1a4d13bece2bae653108), fold pinned in every timed run's own
EXPLAIN, arms interleaved, median of 7. Selectivity premise measured: 40,000 and
3,960,000 of 4,000,000.

arm	median ms
fold count(*) @1%	76
fold sum(p1..p8) @1%	2,998
fold sum(p1..p8) @50%	3,091
fold sum(p1..p8) @99%	3,147
fold sum(p1..p8) @1%, constant payload	419
fold count(*) @1%, constant payload	89
row path sum(p1..p8) @1% / @99%	3,126 / 3,834

Survivors rise 99x, fold time rises 5%: payload cost on the fold path is flat
in selectivity, i.e. paid for every non-skipped row before the key check. The
gap the pre-retraction plan targeted exists, on exactly the shapes above.

Decomposed at 1%: payload cost = 2,922 ms, of which ~2,579 ms is whole-vector
decode (your kill shot 1 stands — the reorder cannot recover it) and ~330 ms is
the gather (~10.3 ns per row·col — reproducing your 11.7 ns fetch_att figure on
the valid comparison). So on this worst-case-for-decode payload the Step-2
reorder is worth ~11% of the wide query. On the constant payload, where decode is
near-free, the gather is essentially the whole payload cost: 419 ms → ~92 ms
derived ceiling, ~4.6x on that query shape. (Derived, not measured — the true
number needs the reorder built. And your own comp-vs-rand int8 numbers put int8
decode small, so the recoverable share on int8-shaped payloads likely sits
toward the high end. Not measured here; float8 is the one class I measured.)

What I'd change in the doc

1. Replace "taken only for aggregates with no payload column" and "it never
   carries payload" with the census: fold accepts count(*)/count(col) and
   sum/avg over int2/int4/float4/float8; it rejects sum/avg over int8/numeric,
   min/max, and any mixed list containing one ineligible aggregate. State that
   the Step-2 counter read 0 because the fixture payload was int8.
2. Rescope the TL;DR: "no recoverable gap on the row path; on the
   (default-off) fold path the gap is real, flat in selectivity, and bounded by
   the gather share of payload cost — measured ~11% of the wide query on
   incompressible float8 @1%, ceiling ~4.6x on highly compressible payload."
3. Keep everything else — the plumbing verdict, the byval-only gate (so this
   still does nothing for q24), kill shots 1/3/4/5, and the cost-model
   requirement, which this measurement re-confirms: recoverable ≈ gather x
   (1 − selectivity), so at high survival there is nothing to win.

Two scope notes for prioritization, not objections: enable_ungrouped_vector_agg
defaults to off, so no default configuration reaches this gap; and whether
the corrected numbers justify rebuilding Step 2 is a scheduling call I am not
making here — the doc just must not record "there is no gap" when the gap is
measured.

Requesting changes because this PR's entire content is the record, and the
record's headline is the one refuted sentence. Happy to re-review the same day
it's updated; full scripts and raw outputs are on the bench under
/home/jd/i405/ if you want to re-run any arm.