Add bounded codec runtime and compression handlers

[xiangfu0]

Context

This is the second layer extracted from #18229 and depends on #19284 through GitHub's native stacked-PR feature.

What changed

• Adds a closed codec runtime under pinot-segment-local; it does not publish a pluggable codec SPI.
• Adds LZ4, Snappy, Gzip, and Zstd handlers, including canonical ZSTD(level) options and the legacy ZSTANDARD input alias.
• Adds wrapperless multi-stage execution with forward encode and reverse decode order.
• Bounds encoded size, each intermediate allocation, cumulative work, outer input size, and exact final decoded size.
• Cleans direct scratch buffers on success and failure and validates corrupt/truncated/trailing frames.
• Keeps transform ordering generic so concrete transforms can be reviewed in later stack layers.

Safety and scope

Only CodecPipelineExecutor is public across local packages; registry, handlers, options, context, and validator remain package-private. The only public decode entry point requires explicit decoded-size, per-stage, and cumulative bounds.

This PR changes no forward-index creator, reader, writer, handler, config, or on-disk format. All fail-closed gates from #19284 remain in place, so codecSpec is still not usable after this layer alone.

Tests

• 49 final runtime, round-trip, ordering, bound, and corrupt-input tests
• 13 unchanged fail-closed gate regressions from Add codec spec DSL and configuration plumbing #19284
• Spotless, Checkstyle, license format, and license check for pinot-segment-local
• final git diff --check and stack range-diff are clean

[codecov-commenter]

Codecov Report

❌ Patch coverage is 76.67343% with 115 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.22%. Comparing base (613c304) to head (d4f9863).

Files with missing lines	Patch %	Lines
...ot/segment/local/io/codec/GzipCodecDefinition.java	65.45%	28 Missing and 10 partials ⚠️
...ot/segment/local/io/codec/ZstdCodecDefinition.java	64.70%	20 Missing and 10 partials ⚠️
.../segment/local/io/codec/CodecPipelineExecutor.java	77.68%	15 Missing and 12 partials ⚠️
...not/segment/local/io/codec/Lz4CodecDefinition.java	79.62%	6 Missing and 5 partials ⚠️
.../segment/local/io/codec/SnappyCodecDefinition.java	85.18%	4 Missing and 4 partials ⚠️
...che/pinot/segment/local/io/codec/CodecContext.java	75.00%	1 Missing ⚠️

Additional details and impacted files

@@                             Coverage Diff                             @@
##             xiangfu0/codex/codec-stack/01-config   #19285       +/-   ##
===========================================================================
+ Coverage                                   39.17%   67.22%   +28.04%     
- Complexity                                   1423     1424        +1     
===========================================================================
  Files                                        3465     3476       +11     
  Lines                                      220562   221055      +493     
  Branches                                    35201    35285       +84     
===========================================================================
+ Hits                                        86409   148596    +62187     
+ Misses                                     126278    60594    -65684     
- Partials                                     7875    11865     +3990     

Flag	Coverage Δ
integration	100.00% <ø> (ø)
integration1	100.00% <ø> (ø)
integration2	0.00% <ø> (ø)
java-25	67.22% <76.67%> (+28.04%)	⬆️
lane-a	100.00% <ø> (ø)
lane-b	0.00% <ø> (ø)
temurin	67.22% <76.67%> (+28.04%)	⬆️
unittests	67.21% <76.67%> (+28.04%)	⬆️
unittests1	57.67% <0.00%> (?)
unittests2	39.27% <76.67%> (+0.10%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[xiangfu0]

Two notes from review:

1. SnappyCodecDefinition.decode allocates a direct buffer from the untrusted header-declared decompressed size with only a < 0 check — LZ4/GZIP/ZSTD all apply a sanity cap. Latent today (the path is package-private and unused in production), but the V7 reader PR later in this stack makes it reachable from segment data. I'll include the cap plus a standalone Snappy corrupt-input test in that PR unless you'd rather fix it here.

2. Cross-PR invariant for the rest of the stack: ForwardIndexConfig stores the structural normalization (toDslString), which can differ from the executor's getCanonicalSpec() (e.g. ZSTANDARD → ZSTD(3)). The V7 header write and the reload spec comparison must canonicalize via the executor on both sides — never persist or compare the config-stored string.

[Copilot]

Pull request overview

Adds a closed, bounded codec runtime to pinot-segment-local, building on the codec DSL while remaining disconnected from index formats.

Changes:

• Adds LZ4, Snappy, Gzip, and Zstd handlers.
• Adds pipeline validation, bounded execution, and buffer cleanup.
• Adds round-trip, corruption, ordering, visibility, and bounds tests.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
ZstdCodecDefinitionTest.java	Tests empty Zstd frames.
CompressionCodecCorruptInputTest.java	Tests corrupt inputs and bounds.
CodecRegistryTest.java	Tests registry closure and aliases.
CodecPipelineValidatorTest.java	Tests pipeline ordering rules.
CodecPipelineExecutorTest.java	Tests execution, bounds, and visibility.
ZstdCodecDefinition.java	Implements bounded Zstd handling.
SnappyCodecDefinition.java	Implements Snappy handling.
Lz4CodecDefinition.java	Implements length-prefixed LZ4 handling.
GzipCodecDefinition.java	Implements framed Deflate/Gzip handling.
CodecRegistry.java	Defines the closed codec registry.
CodecPipelineValidator.java	Validates stage ordering and context.
CodecPipelineExecutor.java	Executes bounded codec pipelines.
CodecOptions.java	Defines codec option contracts.
CodecKind.java	Classifies pipeline stages.
CodecDefinition.java	Defines codec metadata contracts.
CodecContext.java	Carries stored-type context.
CodecBufferUtils.java	Provides direct-buffer utilities.
ChunkCodecHandler.java	Defines codec execution operations.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[xiangfu0]

Pushed `bfe108c3d9` (rebased onto the updated #19284 head) addressing review findings on the runtime.

Snappy decode size cap, fixed as a pattern rather than a fourth copy. `SnappyCodecDefinition.decode` allocated from the header-declared decompressed size with only a negative check, while LZ4/GZIP/ZSTD each carried a 1 GiB sanity cap. That gap is exactly the drift a per-codec hand-rolled constant produces, so instead of adding a fourth private copy I hoisted it into `CodecBufferUtils.checkDeclaredDecompressedSize(declared, codec, source)` and routed all four codecs through it. A future codec cannot omit the bound, and the policy is single-sourced. New corrupt-input tests cover both Snappy paths — including `decodeInto`, which is the one the bounded executor actually uses in production — at the boundary (cap + 1), plus a garbage-input test Snappy was missing.

`CodecRegistry.register()` rejects the reserved name `CODEC` as defense in depth alongside the parser and `CodecInvocation` reservations, referencing `CodecSpecParser.REMOVED_WRAPPER_NAME` (made public in #19284) rather than re-spelling the literal.

`ZSTD(0)` is now rejected; the accepted range is `[1, maxCompressionLevel]`. zstd treats level 0 as "use the default level", so `ZSTD(0)` and `ZSTD(3)` would be identical behavior under two canonical spellings — the same one-spelling-per-behavior rule that motivates the leading-zero fix, and worth settling before the header format freezes. Use bare `ZSTD` for the default.

One follow-up worth filing separately (deliberately not fixed here, since it is a different long-shipped subsystem): the legacy `SnappyDecompressor.decompressedLength()` in `io/compression` has the same unbounded-header shape, and shipped callers such as `VarByteChunkForwardIndexReaderV4` allocate from it.

Verification: 53 codec tests plus 177 downstream gate/validation/handler tests green (`CodecPipelineExecutorTest`, `CodecPipelineValidatorTest`, `CodecRegistryTest`, `CompressionCodecCorruptInputTest`, `ZstdCodecDefinitionTest`, `ForwardIndexTypeTest`, `TableConfigUtilsTest`, `ForwardIndexCreatorFactoryTest`, `OpenStructIndexTypeTest`, `ForwardIndexHandlerTest`); spotless/checkstyle/license clean.

[xiangfu0]

Note: pushed a master sync to this branch (merge commit, no code changes of its own) so the downstream stack includes the #19282 chunk-caching fix that the reload tests in #19308 depend on.

The full stack continuing this PR: #19305 (DELTA/DELTADELTA) → #19306 (T64/GORILLA) → #19307 (V7 format) → #19308 (reload + enable) → #19309 (integration tests + docs). Each is based on its predecessor's branch.