fix: Subnet splitting for new HTTP outcalls pricing - #11042
Draft
eichhorl wants to merge 2 commits into
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
Under the new HTTP outcalls pay-as-you-go pricing, the caller's payment is split into a per-replica allowance which is stored in the context. Each replica consumes part of this allowance to produce their share of the response. Any unspent cycles out of this allowance are transferred back to the caller (initial refund). A late replica which didn't participate in the response, may still issue its refund later, after the call already received a response. Both the initial and late refunds are credited to the canister directly.
For this purpose, there are two collections holding HTTP outcalls in the call context manager: one holding in progress requests (waiting for the response and initial refund), and one holding delivered requests (waiting for late asynchronous refunds).
Problem
During a subnet split, some canisters may move to a different subnet. Currently, all HTTP contexts stay on the original subnet (A'). Most (if not all) of the in-progress contexts will likely time out because:
For legacy pricing, this is generally fine, since any refund is part of the timeout response, which can still be routed to the calling canister, even if it moved to a different subnet.
However, under pay-as-you-go pricing, any refunds can no longer be credited if the canister moved to a different subnet.
Proposed Changes
Starting with this PR, we reject all in-flight HTTP contexts of canisters that were split off, and attach any outstanding refund to the response directly, without waiting for any refund shares (the call is refunded in full). This way, no new delivered context waiting for further refunds is created.
Any existing delivered contexts for migrated canisters are also removed. The outstanding refunds are inserted as refund messages into the refund pool, which are routed to the receiving canisters.