refactor(sdk): introduce NonZeroIggyDuration for retries and heartbeats - #3891
Conversation
|
/request-review @hubcio |
|
/ready |
|
The problem is real for consumers:
producers:
and connection lifecycle:
Clarification:
Solution:
|
|
Thanks for the review. I agree that using So I would prefer to keep the current approach. What do you think? |
|
@ethanlin01x fair points.
|
|
@haubur breaking changes are totally fine at this stage, as we're very close to the next release, with lots of breaking changes anyway :) |
|
Thanks @spetz! So @ethanlin01x lets have a NonZeroIggyDuration for retries and heartbeats, while keeping the busy loop for AutoCommit on an interval = 0. Would you want to rewrite? Otherwise, I can also take this. |
f923068 to
f47d684
Compare
|
@haubur Thanks, I just rewrote the whole PR to introduce |
Intervals that pace a loop panic or spin when they are zero. A dedicated type lets those fields reject the zero at construction instead of at every call site, and rejects the 'none', 'disabled' and 'unlimited' aliases that IggyDuration parses as zero.
The transport heartbeat interval, the reconnection interval, the consumer init and polling retry intervals and the producer send retries interval all pace a loop, so zero either spins a core or panics tokio::time::interval. They now hold NonZeroIggyDuration, which rejects the zero where the config is built or the connection string is parsed. An auto-commit interval stays an IggyDuration: zero there stores the offsets in a busy loop, which is a supported choice. So does reestablish_after, where zero means reconnecting immediately.
…onally The binding follows the Rust SDK: a zero reconnection interval is now rejected whatever the retry policy is, and a zero auto-commit interval is accepted and stores the offsets in a busy loop.
The consumer builder takes NonZeroIggyDuration for the retry intervals, so the binding's own zero check hands that type over.
f47d684 to
b66b045
Compare
haubur
left a comment
There was a problem hiding this comment.
Thanks! I add some minor comments, basically just location and making sure that the IggyDuration and NonZeroIggyDuration API will align.
The type is a thin wrapper over IggyDuration and reads better next to it than in a module of its own. Its opening doc paragraph is gone as well: it explained why the type was introduced, and the type is not limited to that context.
Callers that hold a NonZeroIggyDuration had to unwrap it into an IggyDuration for anything beyond the few accessors, so the same surface is now on the type itself: new, as_secs_f64, abs_diff, Add and the conversions from std::time::Duration and humantime::Duration. The conversions are TryFrom rather than From because zero has to stay rejected, and abs_diff returns an IggyDuration because the gap between two equal durations is zero.
…tion-fence # Conflicts: # foreign/python/src/consumer.rs
|
/request-review @spetz |
spetz
left a comment
There was a problem hiding this comment.
Looks solid overall, just a few minor improvements left :)
NonZeroIggyDuration only rejected zero, but serialization emits whole microseconds, so a shorter duration such as 1ns came back as zero on the other side. TryFrom<IggyDuration> rejects it with its own error variant now, and every constructor goes through that path.
The HTTP connection string parses a heartbeat interval, but HttpClientConfig had nowhere to hold it and the client hardcoded five seconds, so whatever the caller asked for was dropped. The config carries the interval now and the client reads it from there.
A zero interval left the background task storing the offsets in a busy loop. The interval variants of AutoCommit take NonZeroIggyDuration now, so the Rust API rules it out at compile time and the Python and PHP bindings reject it at the call. Committing on every poll is spelled When or After.
Zero means no throttling and both the Rust and the Python SDK accept it, so the binding no longer turns it into an error. The non-zero check stays on the retry intervals.
|
/ready |
`IggyDuration::as_micros` truncates to `u64`, so the non-zero guard read exactly 2^64 microseconds as sub-microsecond and let 2^64 + 1 through as 1. Compare the underlying `Duration` and let serialization fail once the value no longer fits the wire format.
The client builder now takes a `NonZeroIggyDuration`, so an interval coming from C++ has to be validated instead of converted.
|
/ready |
Which issue does this PR address?
Relates to #3776 (Found while reviewing)
Rationale
Zero-valued durations reach loops that spin or panic. A zero retry interval panics tokio::time::interval, and a zero heartbeat interval turns the heartbeat task into a continuous ping loop. IggyDuration::from_str maps 0, none, disabled and unlimited to the same zero, so a user asking to turn something off arrives there.
A duration that paces a loop is a different thing from a duration that measures a delay, so it gets its own type. NonZeroIggyDuration rejects the zero where the value is built, which means no call site has to remember the rule and every binding that funnels through the Rust SDK gets the guarantee.
What changed?
New type
NonZeroIggyDuration(core/common/src/utils/duration.rs):TryFrom<IggyDuration>,TryFrom<u64>,TryFrom<Duration>,FromStr,Displayand serde, all rejecting zero and the three aliases that parse as zero. Serialization emits whole microseconds, so a duration shorter than that is rejected as well. The error is a typedNonZeroDurationError.Fields that now hold it:
heartbeat_intervalon the TCP, QUIC, WebSocket and HTTP client configs and connection string options, plusBinaryTransport::get_heartbeat_intervalandSystemClient::heartbeat_interval. The field is new onHttpClientConfig, where the connection string option was parsed and thrown away.reconnection.intervalon all three transportsIggyConsumerinit_retry_intervalandpolling_retry_intervalProducerCoresend_retries_intervalAutoCommit::Interval,AutoCommit::IntervalOrWhenandAutoCommit::IntervalOrAfterZero stays legal where it means something, and this is now pinned by tests:
reestablish_after(zero reconnects immediately),poll_interval(zero means no throttling) and the background producerlinger_time.The Python and PHP bindings follow the same types.
Breaking changes
NonZeroIggyDuration::from_str("5s")?instead ofIggyDuration::from_str("5s")?.heartbeat_interval=none|disabled|unlimited|0orreconnection_interval=0now fails withInvalidConnectionStringinstead of silently spinning. This includes the HTTP connection string, where the option was parsed and discarded.AutoCommitinterval is now rejected. The three interval variants takeNonZeroIggyDuration, so Rust rules it out at compile time and the Python and PHP bindings raise at the call. Zero stored the offsets in a busy loop; committing on every poll is spelledAutoCommit::WhenorAutoCommit::After.TcpReconnectionConfig(interval=timedelta(0))is now rejected whatever the retry policy is.Local Execution
AI Usage