PYTHON-5805 CSFLE/QE Support for HTTP Proxies - #20
Closed
blink1073 wants to merge 23 commits into
Closed
Conversation
…odb#2965) Co-authored-by: Noah Stapp <noah@noahstapp.com>
Co-authored-by: Jeffrey 'Alex' Clark <alex.clark@mongodb.com> Co-authored-by: Jeffrey 'Alex' Clark <aclark@aclark.net>
Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Implements all six cases of spec section 28 "KMS Connect Callback": plain and TLS proxy tunneling via kms_connect_callback, auto encryption through a proxy, callback error propagation, timeout visibility on KMSConnectContext, and retry after a callback network error.
… 5 gap The docstring promised the driver could pass timeout=None, but the only producer (max(_csot.clamp_remaining(...), 0.001)) is always a positive float. Reworded to describe actual behavior without narrowing the Optional[float] type. Also added a comment on the case 5 timeout assertion in TestKmsConnectCallbackProse recording that explicit ClientEncryption operations set no CSOT deadline, so timeoutMS on the key-vault client does not currently tighten the value asserted.
Case 5 asserts the KMS connect callback receives a non-zero timeout. That cannot fail in PyMongo: ClientEncryption does not support timeoutMS and explicit encryption operations establish no CSOT deadline, so the callback always receives the default KMS connect timeout. Skip the case rather than leave it passing vacuously, and record the deviation on KMSConnectContext, which the CSOT specification requires for any blocking section timeoutMS does not cover. Tracked in PYTHON-6037.
| ref: ${{ inputs.ref }} | ||
|
|
||
| - uses: actions/setup-python@v6.3.0 | ||
| - uses: actions/setup-python@v7.0.0 |
| uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0 | ||
| with: | ||
| python-version: "3.10" | ||
| - uses: mongodb-labs/drivers-github-tools/python/uv-lock-update@bc33fb6aca874b4c37a4ee79bc7061a1ee68baad # v3 |
Owner
Author
|
Superseded: opened while this fork's main was 12 commits stale, so the diff pinned a base of 500e1d0 and showed 23 commits / 76 files. Fork main is now synced; reopening so the diff shows only this branch's 11 commits. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PYTHON-5805
Changes in this PR
Adds
kms_connect_callback, which lets the caller open the TCP connection to a KMS host so that CSFLE and Queryable Encryption KMS traffic can be tunneled through an HTTP proxy. Networks that force outbound port 443 through a proxy currently make CSFLE and QE unusable, and no workaround exists.New public API:
kms_connect_callbackonAutoEncryptionOpts,ClientEncryption, andAsyncClientEncryption. The callback receives the KMS host, port, and remaining timeout, and returns a connected socket. The driver then performs the KMS TLS handshake over that socket, so certificate and hostname verification continue to target the KMS host rather than the proxy. The async API requires a coroutine function; the synchronous API requires a regular function.KMSConnectContext, a frozen dataclass carrying the values passed to the callback. Its docstring holds worked HTTP CONNECT examples for a plain proxy and for a TLS proxy, in both synchronous and async form.Internal changes:
ssl.SSLSocket, so a TLS proxy callback must relay through asocket.socketpairand return the plain end. Without the explicit check, the failure surfaces as an opaque handshake error with no indication of the cause.Implements the six prose tests from section 28 of the client-side-encryption test specification. Case 5 is skipped as a known specification discrepancy: PyMongo does not implement
ClientEncryptionOpts.timeoutMS, so explicit encryption operations establish no CSOT deadline and the assertion that case makes cannot fail. Tracked in PYTHON-6037.Test Plan
Prose tests against real AWS KMS, with the proxies from drivers-evergreen-tools, in both flavors: 10 passed, 2 skipped, 0 failed. Cases 1, 2, 3, 4 and 6 pass; case 5 skips on PYTHON-6037. Both proxies recorded
connect_count 1, and the plain proxy loggedconnect_target kms.us-east-1.amazonaws.com:443, confirming that real KMS traffic went through the tunnel. Case 2 is the meaningful security check, because a real AWS KMS certificate would not validate against the proxy's own CA.Unit tests covering the callback contract: 12 passed, needing no server or credentials.
Evergreen patch across 7 encryption variants (RHEL8, macOS and Windows, each with and without crypt_shared, plus PyOpenSSL on RHEL8): https://spruce.corp.mongodb.com/version/6a84cd3db6c0e20007477689
just lint,just typingandjust docsare all clean.just docsrunssphinx-build -W, so the new docstrings render without warnings.For the TLS-wrapping refactor, which touches every connection the driver makes, I compared the sets of failing test IDs between the merge base and this branch using an identical test scope. The symmetric difference is exactly the newly added tests, so no pre-existing test changed state.
Checklist
Checklist for Author
Checklist for Reviewer