Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,15 @@ which was true until that script existed.

### Added

- `pgcolumnar.parallel_flush` dispatches a stripe flush across background workers
(#445). Default off. When on, a flush of two or more columns fans the per-column
encode and compress work out to a worker pool. Any column a worker does not
finish is completed serially in the backend, so the stored bytes match the
serial path either way. It helps one large flush of many numeric columns by up
to 14 percent. A wide text-heavy flush regresses, and so do frequent small
flushes, because it copies the buffered bytes through shared memory. So it is a
per-session opt-in for a wide numeric bulk load, not a default.

- `pgcolumnar.fsst_verdict_reuse` caches a column's FSST keep/drop verdict for a
bounded number of row groups (#472). Default 16; `0` asks every time, which is
the behaviour before this setting existed.
Expand Down Expand Up @@ -171,6 +180,20 @@ which was true until that script existed.

### Changed

- A columnar scan whose filter cannot be pushed down now skips decoding the
projected columns of a 1024-row vector that holds no matching row (#452). The
scan decodes the filter columns first, rules out the vectors with no match, and
decodes the rest only for the vectors that survive. A `SELECT *` under a
leading-wildcard `LIKE` that matches few rows then approaches the cost of
`count(*)`. It no longer decodes every column of every row scanned. A count over
one column gains nothing, because it has no projected column to skip.

- The writer detoasts each value once per row (#445). It was detoasted once for
the encoder, once for the bloom filter, and once for each of the two zone-map
comparisons. For a toasted column each of those was a separate decompression. A
load of a large compressed text column is about 11 percent faster, and the
stored bytes are unchanged.

- `pgcolumnar.analyze()` places `histogram_bounds` at PostgreSQL's own positions
(#414). The bounds were evenly spaced quantiles; core places bound i at
`values[floor(i * (nvals - 1) / (num_hist - 1))]` among the rows left after
Expand Down
1 change: 1 addition & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ pgColumnar has two kinds of settings:
| `pgcolumnar.compression` | enum | `zstd` | Default codec for new chunks. One of `none`, `pglz`, `lz4`, `zstd`. `lz4` and `zstd` are available only when the extension was built with those libraries. |
| `pgcolumnar.compression_level` | integer | `3` | Level for the `zstd` codec. Range 1 to 22. Higher levels compress more and write more slowly. |
| `pgcolumnar.fsst_min_gain_percent` | integer | `5` | Minimum size reduction, in percent, for FSST string encoding to be kept for a column chunk. Range 0 to 99. See below. |
| `pgcolumnar.parallel_flush` | boolean | `off` | Opt-in. When on, a stripe flush of two or more columns fans the per-column encode and compress work out to background workers. The stored bytes match the serial path. It helps one large flush of many numeric columns by up to 14 percent. A wide text-heavy flush regresses, because it copies the buffered bytes through shared memory. Frequent small flushes regress too, so it is off by default. Enable it for a wide numeric bulk load in the session that runs it. |

To build the FSST codes for each vector is one of the larger costs of a load of
text data. A value of `0` keeps FSST if it makes any reduction after the block
Expand Down
Loading