diff --git a/CHANGELOG.md b/CHANGELOG.md index 4f7da99..8dd23d6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,15 @@ which was true until that script existed. ### Added +- `pgcolumnar.parallel_flush` dispatches a stripe flush across background workers + (#445). Default off. When on, a flush of two or more columns fans the per-column + encode and compress work out to a worker pool. Any column a worker does not + finish is completed serially in the backend, so the stored bytes match the + serial path either way. It helps one large flush of many numeric columns by up + to 14 percent. A wide text-heavy flush regresses, and so do frequent small + flushes, because it copies the buffered bytes through shared memory. So it is a + per-session opt-in for a wide numeric bulk load, not a default. + - `pgcolumnar.fsst_verdict_reuse` caches a column's FSST keep/drop verdict for a bounded number of row groups (#472). Default 16; `0` asks every time, which is the behaviour before this setting existed. @@ -171,6 +180,20 @@ which was true until that script existed. ### Changed +- A columnar scan whose filter cannot be pushed down now skips decoding the + projected columns of a 1024-row vector that holds no matching row (#452). The + scan decodes the filter columns first, rules out the vectors with no match, and + decodes the rest only for the vectors that survive. A `SELECT *` under a + leading-wildcard `LIKE` that matches few rows then approaches the cost of + `count(*)`. It no longer decodes every column of every row scanned. A count over + one column gains nothing, because it has no projected column to skip. + +- The writer detoasts each value once per row (#445). It was detoasted once for + the encoder, once for the bloom filter, and once for each of the two zone-map + comparisons. For a toasted column each of those was a separate decompression. A + load of a large compressed text column is about 11 percent faster, and the + stored bytes are unchanged. + - `pgcolumnar.analyze()` places `histogram_bounds` at PostgreSQL's own positions (#414). The bounds were evenly spaced quantiles; core places bound i at `values[floor(i * (nvals - 1) / (num_hist - 1))]` among the rows left after diff --git a/docs/configuration.md b/docs/configuration.md index fc9542e..172038e 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -29,6 +29,7 @@ pgColumnar has two kinds of settings: | `pgcolumnar.compression` | enum | `zstd` | Default codec for new chunks. One of `none`, `pglz`, `lz4`, `zstd`. `lz4` and `zstd` are available only when the extension was built with those libraries. | | `pgcolumnar.compression_level` | integer | `3` | Level for the `zstd` codec. Range 1 to 22. Higher levels compress more and write more slowly. | | `pgcolumnar.fsst_min_gain_percent` | integer | `5` | Minimum size reduction, in percent, for FSST string encoding to be kept for a column chunk. Range 0 to 99. See below. | +| `pgcolumnar.parallel_flush` | boolean | `off` | Opt-in. When on, a stripe flush of two or more columns fans the per-column encode and compress work out to background workers. The stored bytes match the serial path. It helps one large flush of many numeric columns by up to 14 percent. A wide text-heavy flush regresses, because it copies the buffered bytes through shared memory. Frequent small flushes regress too, so it is off by default. Enable it for a wide numeric bulk load in the session that runs it. | To build the FSST codes for each vector is one of the larger costs of a load of text data. A value of `0` keeps FSST if it makes any reduction after the block