Skip to content

Client revalidates tool results with one-shot jsonschema.validate(), recompiling the validator on every call #3133

Description

@jlowin

ClientSession.validate_tool_result calls one-shot jsonschema.validate() on every tool result. That function builds a validator class, runs check_schema(), and re-crawls $refs on each call, then discards all of it. The output schema for a given tool doesn't change between calls, so this work is repeated for every tools/call a client makes.

jsonschema's own documentation recommends against this pattern when validating repeatedly against the same schema, for exactly this reason.

Reproduction

import time
from jsonschema import validate, validators

schema = {
    "type": "object",
    "properties": {"result": {"type": "string"}},
    "required": ["result"],
}
instance = {"result": "hello"}
N = 2000

start = time.perf_counter()
for _ in range(N):
    validate(instance, schema)
one_shot = (time.perf_counter() - start) / N

validator_cls = validators.validator_for(schema)
validator_cls.check_schema(schema)
validator = validator_cls(schema)

start = time.perf_counter()
for _ in range(N):
    validator.validate(instance)
cached = (time.perf_counter() - start) / N

print(f"one-shot: {one_shot * 1e6:.1f} us")
print(f"cached:   {cached * 1e6:.1f} us")
print(f"speedup:  {one_shot / cached:.0f}x")

Output on CPython 3.12 / macOS:

one-shot: 173.9 us
cached:     2.7 us
speedup:    63x

Expected vs. actual

Expected: validating a result against a schema the session has already seen costs roughly the validation itself.

Actual: it costs ~174us, of which ~171us is rebuilding and re-checking the validator. In an end-to-end in-memory call_tool round-trip measured at ~0.59ms, substituting a cached validator brought it to ~0.41ms — about 30% of the call.

The effect is proportionally larger for in-process and local-transport clients, where there is no network latency to hide it, and it scales with call volume rather than being a one-time startup cost.

Note

There is a second, smaller effect in the same method that may or may not be intended: when a tool isn't yet in _tool_output_schemas, it triggers a list_tools() refresh. A client that connects and immediately calls a tool therefore pays an extra round-trip on its first call. Happy to split that into its own issue if it's worth treating separately.

Environment

  • mcp from main (3a6f299), CPython 3.12, macOS
  • Found while profiling per-operation costs in a downstream project

AI assistance disclosure: this issue was investigated and drafted with AI assistance. The measurements above were reproduced independently before filing, and I understand and stand behind the diagnosis.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions