Skip to content

[core][flink] Refactor partition time resolution using PartitionTimeResolver - #9008

Open
juntaozhang wants to merge 15 commits into
apache:masterfrom
juntaozhang:pr-add-chain-min-part-opti
Open

[core][flink] Refactor partition time resolution using PartitionTimeResolver#9008
juntaozhang wants to merge 15 commits into
apache:masterfrom
juntaozhang:pr-add-chain-min-part-opti

Conversation

@juntaozhang

Copy link
Copy Markdown
Contributor

Purpose

this PR follow #8185, refactors partition time resolution as the title suggests.

Tests

Benchmark: PartitionTimeResolver vs PartitionTimeExtractor

  • Env:

    Item Value
    OS Linux 7.0.0-28-generic (Ubuntu 26.04), x86_64
    CPU Intel(R) Core(TM) Ultra 9 285H
    JDK OpenJDK 11.0.31
  • 50,000 warmup iterations + 200,000 iterations for each task.

  • Each result is reported as average time per operation.

  • Init means constructing the object.

  • Ratio < 1 means PartitionTimeResolver is faster; Ratio > 1 means slower.

  • Parse to date

    Scenario PartitionTimeExtractor.extract PartitionTimeResolver Ratio
    $dt / yyyyMM-dd 8.43 µs/op 465 ns/op 0.06x
    $y$m$d / yyyyMMdd 6.65 µs/op 360 ns/op 0.05x
    $dt $hr / yyyyMMdd HH 797 ns/op 524 ns/op 0.66x
    $a-$b-$c T $hour:$minute:00 / yyyy-MM-dd 'T' HH:mm:ss 1.86 µs/op 672 ns/op 0.36x
  • Each loop construct the object and parse to date

    Scenario Init + PartitionTimeExtractor.extract Init + PartitionTimeResolver Ratio
    $dt / yyyyMM-dd 6.27 µs/op 621 ns/op 0.10x
    $y$m$d / yyyyMMdd 6.63 µs/op 863 ns/op 0.13x
    $dt $hr / yyyyMMdd HH 786 ns/op 870 ns/op 1.11x
    $a-$b-$c T $hour:$minute:00 / yyyy-MM-dd 'T' HH:mm:ss 1.85 µs/op 8.59 µs/op 4.63x
  • Resolve date to partition values

    Scenario ChainTableUtils.calPartValues PartitionTimeResolver Ratio
    $dt / yyyyMM-dd 668 ns/op 244 ns/op 0.37x
    $y$m$d / yyyyMMdd 849 ns/op 300 ns/op 0.35x
    $dt $hr / yyyyMMdd HH 855 ns/op 261 ns/op 0.31x
    $a-$b-$c T $hour:$minute:00 / yyyy-MM-dd 'T' HH:mm:ss 2.32 µs/op 574 ns/op 0.25x
  • Each loop construct the object and resolve date to partition values

    Scenario ChainTableUtils.calPartValues Init + PartitionTimeResolver Ratio
    $dt / yyyyMM-dd 668 ns/op 271 ns/op 0.41x
    $y$m$d / yyyyMMdd 849 ns/op 655 ns/op 0.77x
    $dt $hr / yyyyMMdd HH 855 ns/op 661 ns/op 0.77x
    $a-$b-$c T $hour:$minute:00 / yyyy-MM-dd 'T' HH:mm:ss 2.32 µs/op 8.25 µs/op 3.56x

Summary

  • Hot path (Parse / Resolve): PartitionTimeResolver is significantly faster, often 2–20x over PartitionTimeExtractor.
  • Construction cost: PartitionTimeResolver is heavier because it validates and maps pattern ↔ formatter upfront. PartitionTimeExtractor constructs almost instantly because its do nothing.
  • Overall: For common patterns, Init + Parse / Init + Resolve is still faster with the PartitionTimeResolver. Only complex patterns become slower overall due to recursive matching in construction.

@JingsongLi

Copy link
Copy Markdown
Contributor

The new PartitionTimeResolver is no longer fully compatible with Java DateTimeFormatter. Existing formats such as yyyy-DDD and yyyyMMdd hh a will fail after the upgrade. It has been reproduced that yyyy-DDD + 2026-218 reports that it does not support D.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants