From e122de2ccdd13aef325f3512cb58f63c6a5a0af8 Mon Sep 17 00:00:00 2001 From: "Joshua D. Drake" Date: Wed, 12 Aug 2026 10:20:51 -0600 Subject: [PATCH 1/2] docs: document parallel_flush and log phase 2, detoast-once, parallel_flush The parallel-flush series (#589/#591/#592) and the phase-2 and detoast-once write work landed without their user-facing docs. - configuration.md gains the pgcolumnar.parallel_flush GUC row, in the write settings block: opt-in, off by default, up to 14 percent on a wide bulk load, regresses frequent small flushes, byte-identical to serial. - CHANGELOG [Unreleased] gains three entries: parallel_flush under Added, and under Changed the unprunable-filter decode gating (#452) and the detoast-once write-path saving (#445). docs_style.sh passes (9/9); ste_check reports configuration.md ok. The ClickBench benchmark numbers in benchmarks.md predate phase 2 and still credit #452/#426 as open; that is a separate re-run, being handled elsewhere. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01WmQJqcXdwyuoAiHHt2znBr --- CHANGELOG.md | 22 ++++++++++++++++++++++ docs/configuration.md | 1 + 2 files changed, 23 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4f7da995..fcec0641 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,14 @@ which was true until that script existed. ### Added +- `pgcolumnar.parallel_flush` dispatches a stripe flush across background workers + (#445). Default off. When on, a flush of two or more columns fans the per-column + encode and compress work out to a worker pool. Any column a worker does not + finish is completed serially in the backend, so the stored bytes match the + serial path either way. It helps one large flush of many columns by up to 14 + percent, and it regresses frequent small flushes. So it is a per-session opt-in + for a wide bulk load, not a default. + - `pgcolumnar.fsst_verdict_reuse` caches a column's FSST keep/drop verdict for a bounded number of row groups (#472). Default 16; `0` asks every time, which is the behaviour before this setting existed. @@ -171,6 +179,20 @@ which was true until that script existed. ### Changed +- A columnar scan whose filter cannot be pushed down now skips decoding the + projected columns of a 1024-row vector that holds no matching row (#452). The + scan decodes the filter columns first, rules out the vectors with no match, and + decodes the rest only for the vectors that survive. A `SELECT *` under a + leading-wildcard `LIKE` that matches few rows then approaches the cost of + `count(*)`. It no longer decodes every column of every row scanned. A count over + one column gains nothing, because it has no projected column to skip. + +- The writer detoasts each value once per row (#445). It was detoasted once for + the encoder, once for the bloom filter, and once for each of the two zone-map + comparisons. For a toasted column each of those was a separate decompression. A + load of a large compressed text column is about 11 percent faster, and the + stored bytes are unchanged. + - `pgcolumnar.analyze()` places `histogram_bounds` at PostgreSQL's own positions (#414). The bounds were evenly spaced quantiles; core places bound i at `values[floor(i * (nvals - 1) / (num_hist - 1))]` among the rows left after diff --git a/docs/configuration.md b/docs/configuration.md index fc9542ef..c2983ea5 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -29,6 +29,7 @@ pgColumnar has two kinds of settings: | `pgcolumnar.compression` | enum | `zstd` | Default codec for new chunks. One of `none`, `pglz`, `lz4`, `zstd`. `lz4` and `zstd` are available only when the extension was built with those libraries. | | `pgcolumnar.compression_level` | integer | `3` | Level for the `zstd` codec. Range 1 to 22. Higher levels compress more and write more slowly. | | `pgcolumnar.fsst_min_gain_percent` | integer | `5` | Minimum size reduction, in percent, for FSST string encoding to be kept for a column chunk. Range 0 to 99. See below. | +| `pgcolumnar.parallel_flush` | boolean | `off` | Opt-in. When on, a stripe flush of two or more columns fans the per-column encode and compress work out to background workers. The stored bytes match the serial path. It helps one large flush of many columns by up to 14 percent. It regresses frequent small flushes, so it is off by default. Enable it for a wide bulk load in the session that runs it. | To build the FSST codes for each vector is one of the larger costs of a load of text data. A value of `0` keeps FSST if it makes any reduction after the block From 8dc6e4cd70aaeb6bc09189acf4f1edcf6f4edb38 Mon Sep 17 00:00:00 2001 From: "Joshua D. Drake" Date: Wed, 12 Aug 2026 10:30:38 -0600 Subject: [PATCH 2/2] docs: parallel_flush win is numeric columns; a wide text flush regresses Per ChronicallyJD's review of #593: the parallel-flush win is specifically many CHEAP (numeric) columns; a wide text-heavy flush regresses (~16%) because it copies the buffered bytes through shared memory. Sharpen both the config row and the CHANGELOG entry so a wide-text-table user is not misled into enabling it. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01WmQJqcXdwyuoAiHHt2znBr --- CHANGELOG.md | 7 ++++--- docs/configuration.md | 2 +- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index fcec0641..8dd23d64 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,9 +18,10 @@ which was true until that script existed. (#445). Default off. When on, a flush of two or more columns fans the per-column encode and compress work out to a worker pool. Any column a worker does not finish is completed serially in the backend, so the stored bytes match the - serial path either way. It helps one large flush of many columns by up to 14 - percent, and it regresses frequent small flushes. So it is a per-session opt-in - for a wide bulk load, not a default. + serial path either way. It helps one large flush of many numeric columns by up + to 14 percent. A wide text-heavy flush regresses, and so do frequent small + flushes, because it copies the buffered bytes through shared memory. So it is a + per-session opt-in for a wide numeric bulk load, not a default. - `pgcolumnar.fsst_verdict_reuse` caches a column's FSST keep/drop verdict for a bounded number of row groups (#472). Default 16; `0` asks every time, which is diff --git a/docs/configuration.md b/docs/configuration.md index c2983ea5..172038ed 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -29,7 +29,7 @@ pgColumnar has two kinds of settings: | `pgcolumnar.compression` | enum | `zstd` | Default codec for new chunks. One of `none`, `pglz`, `lz4`, `zstd`. `lz4` and `zstd` are available only when the extension was built with those libraries. | | `pgcolumnar.compression_level` | integer | `3` | Level for the `zstd` codec. Range 1 to 22. Higher levels compress more and write more slowly. | | `pgcolumnar.fsst_min_gain_percent` | integer | `5` | Minimum size reduction, in percent, for FSST string encoding to be kept for a column chunk. Range 0 to 99. See below. | -| `pgcolumnar.parallel_flush` | boolean | `off` | Opt-in. When on, a stripe flush of two or more columns fans the per-column encode and compress work out to background workers. The stored bytes match the serial path. It helps one large flush of many columns by up to 14 percent. It regresses frequent small flushes, so it is off by default. Enable it for a wide bulk load in the session that runs it. | +| `pgcolumnar.parallel_flush` | boolean | `off` | Opt-in. When on, a stripe flush of two or more columns fans the per-column encode and compress work out to background workers. The stored bytes match the serial path. It helps one large flush of many numeric columns by up to 14 percent. A wide text-heavy flush regresses, because it copies the buffered bytes through shared memory. Frequent small flushes regress too, so it is off by default. Enable it for a wide numeric bulk load in the session that runs it. | To build the FSST codes for each vector is one of the larger costs of a load of text data. A value of `0` keeps FSST if it makes any reduction after the block