feat: version-aware MockServer abstraction for client scenarios#321
Draft
felixweinberger wants to merge 6 commits into
Draft
feat: version-aware MockServer abstraction for client scenarios#321felixweinberger wants to merge 6 commits into
felixweinberger wants to merge 6 commits into
Conversation
MockServer encapsulates the lifecycle scaffold a client-conformance scenario presents to the client-under-test: - createServerStateful: 2025-x lifecycle. SDK Server + StreamableHTTPServerTransport (sessionless mode); the SDK handles the initialize handshake. - createServerStateless: 2026-x lifecycle (SEP-2575). Raw express app that validates _meta + MCP-Protocol-Version on every request, serves server/discover, routes other methods to the supplied handlers. createServerFor(specVersion) picks the implementation. ScenarioContext bundles specVersion and a bound createServer() for the runner to hand to each scenario. This is the client-conformance mirror of src/connection (PR #318). Nothing uses it yet; wiring follows in the next commit.
Scenario.start() becomes start(ctx: ScenarioContext). The runner builds the context from --spec-version (defaulting to LATEST_SPEC_VERSION) and passes it through; scenarios receive it as _ctx and otherwise behave identically. No scenario uses ctx.createServer() yet, so behaviour is unchanged: 231/231 tests pass. Test files use a testScenarioContext() helper. The runner already threads MCP_CONFORMANCE_PROTOCOL_VERSION to the spawned client process, so the fixture-side env wiring is unchanged.
…t scenarios ToolsCallScenario now goes through ctx.createServer() instead of an inline express + SDK Server build. Same handlers, same checks; the assertion now reads from srv.recorded so it works regardless of which lifecycle scaffold the runner picked. initialize, sse-retry, and elicitation-defaults are tagged removedIn: DRAFT (initialize/GET-SSE/SSE-embedded-elicitation are gone in the 2026 lifecycle; the MRTR sibling for elicitation-defaults is a follow-up). spec-version.test.ts: the 'draft is a superset of latest' invariant no longer holds once removedIn: DRAFT exists; the test now asserts that any scenario in latest-but-not-draft is explicitly removedIn.
The auth helper now takes ctx: ScenarioContext as its first argument and branches on ctx.specVersion inside the /mcp route: the stateful path (SDK Server + StreamableHTTPServerTransport) is unchanged; under the draft version a raw stateless handler validates _meta + the MCP-Protocol-Version header, serves server/discover, and routes the same tools/list and tools/call responses. The PRM endpoint, bearer-auth middleware, and request logger sit above the branch and are version-independent. All 25 call sites across the 12 auth scenario files pass ctx through; ServerLifecycle and the express.Application return type are unchanged so stop()/getChecks() are untouched. Deviation from the MockServer wrapper approach: keeping the helper's return type as express.Application avoids restructuring 25 call sites' ServerLifecycle handling in this PR. Folding the auth seam onto ctx.createServer() fully is a follow-up once the lifecycle ownership moves into MockServer.
…PROTOCOL_VERSION Adds a statelessRequest(serverUrl, method, params) helper that POSTs with _meta + MCP-Protocol-Version (the SEP-2575 lifecycle), shimming around the SDK Client not yet supporting stateless mode. The runRequestMetadataClient handler's meta constants are extracted to share with the helper. runBasicClient (initialize, tools_call, json-schema-ref-no-deref) now branches on MCP_CONFORMANCE_PROTOCOL_VERSION: for the draft version it uses statelessRequest to call tools/list then tools/call; for dated versions it keeps the SDK Client path. The runner already passes MCP_CONFORMANCE_PROTOCOL_VERSION to the spawned client, so no runner change is needed.
…or, capability derivation, recorded parity, specVersion threading) - MockServerOptions removed (capabilities/configure had zero callers); opts param dropped from createServerStateful/Stateless/For and ScenarioContext. - validateStatelessRequest extracted from mock-server/stateless and exported; both the stateless MockServer and auth/helpers/createServer.ts call it so _meta/header/version validation cannot drift. - isStatefulVersion exported from connection/select; mock-server/select uses it instead of duplicating the version set. - runner/client.ts: env MCP_CONFORMANCE_PROTOCOL_VERSION set unconditionally to the resolved version; runInteractiveMode now takes specVersion and the CLI passes it. - createServerStateful: capabilities derived from handler method prefixes; newServer() moved inside the try so a capability mismatch surfaces as JSON-RPC -32603 instead of an HTML 500. Recording moved to the express layer so unregistered methods are captured (parity with stateless). - readFinalSseMessage return type now declares error.data. - Tests added for the capability derivation and unregistered-method recording.
commit: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #318. Mirrors the server-side
Connection/RunContextabstraction for client conformance: scenarios that act as a mock server now get a version-awarectx.createServer(handlers)instead of building the server inline.Motivation and Context
Client-conformance scenarios spin up a mock server for the client-under-test to connect to. Today every scenario builds that server inline (express + SDK
Server+StreamableHTTPServerTransport, or rawhttp.createServer), which hard-codes the 2025 lifecycle. A scenario that should run under--spec-version draftcan't unless its mock server speaks the stateless lifecycle, and there are currently three different inline patterns for doing that.What changes
src/mock-server/{index,stateful,stateless,select,testing}.ts—MockServer = {url, recorded, close}with stateful (SDK-backed) and stateless (rawhttp.createServer, validates_meta, servesserver/discover) impls.createServerFor(specVersion)picks.validateStatelessRequest()is exported for reuse.Scenario.start()→start(ctx: ScenarioContext)whereScenarioContext = {specVersion, createServer()}. Runner builds it from--spec-versionand passes the resolved version to the spawned client process viaMCP_CONFORMANCE_PROTOCOL_VERSION(now set unconditionally;runInteractiveModealso takes it).isStatefulVersion()exported fromconnection/select.tsand reused bymock-server/select.tsand the auth helper, so the version boundary is defined once.client/tools_call.tsmigrated toctx.createServer(); assertions read fromsrv.recorded.removedIn: DRAFTon 3 client scenarios —initialize,sse-retry,elicitation-defaults.client/auth/helpers/createServer.tsis version-aware viactx.specVersion: for stateless versions it calls the sharedvalidateStatelessRequest()and routestools/list/tools/calldirectly. The 25 auth call sites now forwardctx. Full restructuring of the auth helper ontoctx.createServer()is deferred (would be ~200 more mechanical lines around theServerLifecycle.start(app)pattern).everything-client.ts: readsMCP_CONFORMANCE_PROTOCOL_VERSION;runBasicClient(handlestools_call) picks SDKClientvs a rawstatelessRequest()helper accordingly. Also fixes a pre-existing registration mismatch ('tools-call'vs'tools_call') that meant the carry-forward client test had never actually run against the fixture.prompts/listdoesn't trip the SDK's capability assertion;recordedcaptured at the express layer so it includes unregistered methods (parity with stateless).How Has This Been Tested?
233/233 unit tests, typecheck/lint/build clean.
tools_callagainsteverything-client: 1/1 under both--spec-version 2025-11-25and--spec-version draft.client/auth/index.test.ts: 43/43.A
client/all-scenarios.test.tsmirror (driveeverything-clientagainst every client scenario under both spec versions) does not exist yet; that would be the fuller validation.Per-scenario category split
ctx.createServer()tools_callremovedIn: DRAFTinitialize,sse-retryremovedIn: DRAFTelicitation-defaults(server→client elicitation via SSE; MRTR-client sibling needed)introducedIn: DRAFT)request-metadata,http-base,http-standard-headers,http-custom-headers,json-schema-ref-deref,mrtr-clientauth/helpers/createServer.ts)client/auth/*Breaking Changes
Scenario.start()→start(ctx: ScenarioContext). All in-tree scenarios are updated.Types of changes
Checklist
Additional context
Deferred:
request-metadata,http-*,mrtr-client,json-schema-ref-deref) ontoMockServer— same call as feat: version-aware Connection abstraction for server scenarios #318 (DRAFT-scenario coherence pass).ctx.createServer()— version-awareness is achieved via the sharedvalidateStatelessRequest(); the express/ServerLifecycleshape is kept for now.