Skip to content

PYTHON-5805 CSFLE/QE Support for HTTP Proxies - #20

Closed
blink1073 wants to merge 23 commits into
mainfrom
PYTHON-5805
Closed

PYTHON-5805 CSFLE/QE Support for HTTP Proxies#20
blink1073 wants to merge 23 commits into
mainfrom
PYTHON-5805

Conversation

@blink1073

Copy link
Copy Markdown
Owner

PYTHON-5805

Changes in this PR

Adds kms_connect_callback, which lets the caller open the TCP connection to a KMS host so that CSFLE and Queryable Encryption KMS traffic can be tunneled through an HTTP proxy. Networks that force outbound port 443 through a proxy currently make CSFLE and QE unusable, and no workaround exists.

New public API:

  • kms_connect_callback on AutoEncryptionOpts, ClientEncryption, and AsyncClientEncryption. The callback receives the KMS host, port, and remaining timeout, and returns a connected socket. The driver then performs the KMS TLS handshake over that socket, so certificate and hostname verification continue to target the KMS host rather than the proxy. The async API requires a coroutine function; the synchronous API requires a regular function.
  • KMSConnectContext, a frozen dataclass carrying the values passed to the callback. Its docstring holds worked HTTP CONNECT examples for a plain proxy and for a TLS proxy, in both synchronous and async form.

Internal changes:

  • Split TLS wrapping out of the configured-socket helpers so a caller-supplied socket can be wrapped against the KMS host. Pure code motion; the connection path is otherwise unchanged.
  • A callback that returns anything other than a plain connected socket fails immediately instead of being retried. Network errors from the callback remain retryable, so libmongocrypt re-invokes the callback.
  • Rejecting an already-TLS-wrapped socket produces an actionable error. Python cannot layer TLS over an ssl.SSLSocket, so a TLS proxy callback must relay through a socket.socketpair and return the plain end. Without the explicit check, the failure surfaces as an opaque handshake error with no indication of the cause.
  • Added a guard so the flavor-specific wording in the generated synchronous docstring cannot silently revert.

Implements the six prose tests from section 28 of the client-side-encryption test specification. Case 5 is skipped as a known specification discrepancy: PyMongo does not implement ClientEncryptionOpts.timeoutMS, so explicit encryption operations establish no CSOT deadline and the assertion that case makes cannot fail. Tracked in PYTHON-6037.

Test Plan

Prose tests against real AWS KMS, with the proxies from drivers-evergreen-tools, in both flavors: 10 passed, 2 skipped, 0 failed. Cases 1, 2, 3, 4 and 6 pass; case 5 skips on PYTHON-6037. Both proxies recorded connect_count 1, and the plain proxy logged connect_target kms.us-east-1.amazonaws.com:443, confirming that real KMS traffic went through the tunnel. Case 2 is the meaningful security check, because a real AWS KMS certificate would not validate against the proxy's own CA.

Unit tests covering the callback contract: 12 passed, needing no server or credentials.

Evergreen patch across 7 encryption variants (RHEL8, macOS and Windows, each with and without crypt_shared, plus PyOpenSSL on RHEL8): https://spruce.corp.mongodb.com/version/6a84cd3db6c0e20007477689

just lint, just typing and just docs are all clean. just docs runs sphinx-build -W, so the new docstrings render without warnings.

For the TLS-wrapping refactor, which touches every connection the driver makes, I compared the sets of failing test IDs between the merge base and this branch using an identical test scope. The symmetric difference is exactly the newly added tests, so no pre-existing test changed state.

Checklist

Checklist for Author

Checklist for Reviewer

  • Does the title of the PR reference a JIRA Ticket?
  • Do you fully understand the implementation? (Would you be comfortable explaining how this code works to someone else?)
  • Is all relevant documentation (README or docstring) updated?

NoahStapp and others added 23 commits July 31, 2026 14:02
Co-authored-by: Jeffrey 'Alex' Clark <alex.clark@mongodb.com>
Co-authored-by: Jeffrey 'Alex' Clark <aclark@aclark.net>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Implements all six cases of spec section 28 "KMS Connect Callback":
plain and TLS proxy tunneling via kms_connect_callback, auto encryption
through a proxy, callback error propagation, timeout visibility on
KMSConnectContext, and retry after a callback network error.
… 5 gap

The docstring promised the driver could pass timeout=None, but the only
producer (max(_csot.clamp_remaining(...), 0.001)) is always a positive
float. Reworded to describe actual behavior without narrowing the
Optional[float] type. Also added a comment on the case 5 timeout
assertion in TestKmsConnectCallbackProse recording that explicit
ClientEncryption operations set no CSOT deadline, so timeoutMS on the
key-vault client does not currently tighten the value asserted.
Case 5 asserts the KMS connect callback receives a non-zero timeout. That
cannot fail in PyMongo: ClientEncryption does not support timeoutMS and
explicit encryption operations establish no CSOT deadline, so the callback
always receives the default KMS connect timeout. Skip the case rather than
leave it passing vacuously, and record the deviation on KMSConnectContext,
which the CSOT specification requires for any blocking section timeoutMS
does not cover. Tracked in PYTHON-6037.
ref: ${{ inputs.ref }}

- uses: actions/setup-python@v6.3.0
- uses: actions/setup-python@v7.0.0
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with:
python-version: "3.10"
- uses: mongodb-labs/drivers-github-tools/python/uv-lock-update@bc33fb6aca874b4c37a4ee79bc7061a1ee68baad # v3
@blink1073

Copy link
Copy Markdown
Owner Author

Superseded: opened while this fork's main was 12 commits stale, so the diff pinned a base of 500e1d0 and showed 23 commits / 76 files. Fork main is now synced; reopening so the diff shows only this branch's 11 commits.

@blink1073 blink1073 closed this Aug 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants