Freeze conformance requirements per spec revision, run each at its wire

.claude/skills/mcp-sdk-tier-audit/README.md

@@ -95,19 +95,21 @@ The skill lives in `.claude/skills/` in this repo, so if you open [Claude Code](
 3. Run the skill:
 
 ```
-/mcp-sdk-tier-audit <local-sdk-path> <conformance-server-url> [client-cmd]
+/mcp-sdk-tier-audit <local-sdk-path> <conformance-server-url> [client-cmd] [--requirements <revision>]
 ```
 
 Pass the client command as the third argument to include client conformance testing. If omitted, client conformance is skipped and noted as a gap in the report.
 
+**Pass `--requirements` with every revision the SDK claims**, comma-separated. Each revision's scenarios run at that revision's own wire version, and all of them must pass for Tier 1: the dated revisions through `2025-11-25` use the stateful initialize handshake while `2026-07-28` is stateless, so a scenario belonging to both has to work on both and one run does not cover the other. It also means a scenario added to the suite after a revision shipped cannot fail an SDK that had no opportunity to adopt it. Without the flag, scoring uses the suite as it stands today, which is not a tier claim. See [Conformance Requirements](../../../README.md#conformance-requirements), and run `conformance list --requirements 2025-11-25,2026-07-28` to see both sets.
+
 **TypeScript SDK example:**
 
 ```bash
 # Terminal 1: start the everything server (build first: npm run build)
 cd ~/src/mcp/typescript-sdk && npm run test:conformance:server:run
 
 # Terminal 2: run the audit (from the conformance repo)
-/mcp-sdk-tier-audit ~/src/mcp/typescript-sdk http://localhost:3000/mcp "npx tsx ~/src/mcp/typescript-sdk/test/conformance/src/everythingClient.ts"
+/mcp-sdk-tier-audit <sdk-path> http://localhost:3000/mcp "node --import tsx <sdk-path>/test/conformance/src/everythingClient.ts" --requirements 2025-11-25,2026-07-28
 ```
 
 **Python SDK example:**

.claude/skills/mcp-sdk-tier-audit/SKILL.md

@@ -5,7 +5,7 @@ description: >-
   Produces tier classification (1/2/3) with evidence table, gap list, and
   remediation guide. Works for any official MCP SDK (TypeScript, Python, Go,
   C#, Java, Kotlin, PHP, Swift, Rust, Ruby).
-argument-hint: '<local-path> <conformance-server-url> [client-cmd] [--branch <branch>]'
+argument-hint: '<local-path> <conformance-server-url> [client-cmd] [--requirements <revision>] [--branch <branch>]'
 ---
 
 # MCP SDK Tier Audit
@@ -43,6 +43,7 @@ Extract from the user's input:
 - **local-path**: absolute path to the SDK checkout (e.g. `~/src/mcp/typescript-sdk`)
 - **conformance-server-url**: URL where the SDK's everything server is already running (e.g. `http://localhost:3000/mcp`)
 - **client-cmd** (optional): command to run the SDK's conformance client (e.g. `npx tsx test/conformance/src/everythingClient.ts`). If not provided, client conformance tests are skipped and noted as a gap in the report.
+- **requirements** (optional): spec revisions to score against, comma-separated, e.g. `--requirements 2025-11-25,2026-07-28`. Each revision's scenarios run at that revision's wire version, and every listed revision must pass for Tier 1. Scores the SDK against exactly the scenarios that revision required when it shipped, rather than everything the suite carries today. Prefer it whenever the question is "does this SDK conform to revision X". Without it, scoring uses today's suite, which can fail an SDK for a scenario added after it shipped. Run `conformance list --requirements <revision>` to see the set.
 - **branch** (optional): Git branch to check on GitHub (e.g. `--branch fweinberger/v1x-governance-docs`). If not provided, derive from the local checkout's current branch: `cd <local-path> && git rev-parse --abbrev-ref HEAD`. This is passed to the tier-check CLI so that policy signal file checks use the correct branch instead of the repo's default branch.
 
 The first two arguments are required. If either is missing, ask the user to provide it.
@@ -61,11 +62,20 @@ The `tier-check` CLI handles all deterministic checks — server conformance, cl
 npm run --silent tier-check -- \
   --repo <owner/repo> \
   --branch <branch> \
+  --requirements <revision> \
   --conformance-server-url <conformance-server-url> \
   --client-cmd '<client-cmd>' \
   --output json
 ```
 
+Omit `--requirements` only if the user did not name a revision. When it is set the
+scorecard reports `requirements_revision`, both pass rates count exactly the
+scenarios that revision requires, and anything run but not scored carries a
+`notScoredReason` of `extension` or `added-after-release`. `requirements_revisions`
+lists every revision scored, and each detail carries the `revision` it came from. Quote the revision
+alongside any conformance number, and report the not-scored failures separately
+rather than folding them into the score or omitting them.
+
 If no client-cmd was detected, omit the `--client-cmd` flag (client conformance will be skipped). The `--branch` flag should always be included (derived from the local checkout if not explicitly provided).
 
 The CLI output includes server conformance pass rate, client conformance pass rate (with per-spec-version breakdown), issue triage compliance, P0 resolution times, label taxonomy, stable release status, policy signal files, and spec tracking gap. Parse the JSON output to feed into Step 4.
@@ -117,8 +127,8 @@ Combine the deterministic scorecard (from the CLI) with the evaluation results (
 
 ### Tier 1 requires ALL of:
 
-- Server conformance test pass rate == 100% (date-versioned scenarios only; `draft` and `extension` are informational and not scored)
-- Client conformance test pass rate == 100% (date-versioned scenarios only; `draft` and `extension` are informational and not scored)
+- Server conformance test pass rate == 100% across every requirement set given to `--requirements` (each run at its own wire), otherwise of date-versioned scenarios only
+- Client conformance test pass rate == 100%, on the same basis
 - Issue triage compliance >= 90% within 2 business days
 - All P0 bugs resolved within 7 days
 - Stable release >= 1.0.0 with no pre-release suffix

README.md

@@ -66,6 +66,7 @@ npx @modelcontextprotocol/conformance client --command "<client-command>" --scen
 - `--suite` - Run a suite of tests in parallel: `all`, `core`, `extensions`, `backcompat`, `auth`, `metadata`, `draft` (scenarios targeting the in-progress draft spec), or `sep-835`
 - `--spec-version <version>` - Filter scenarios by spec version (e.g., `2025-11-25`, `2026-07-28`; `draft` is accepted as an alias for the current draft identifier). The draft version selects the latest dated release plus any draft-only scenarios. When omitted, the version is inferred from the scenario's spec applicability (draft-only scenarios run at the draft version, everything else at the latest dated release); an explicitly requested version outside a scenario's applicability window skips the scenario (exit 0) unless `--force` is passed
 - `--force` - Run a scenario even if it is not applicable at the requested `--spec-version`
+- `--requirements <revision>` - Run exactly what a spec revision requires, frozen at its release (see [Conformance Requirements](#conformance-requirements))
 - `--expected-failures <path>` - Path to YAML baseline file of known failures (see [Expected Failures](#expected-failures))
 - `--timeout` - Timeout in milliseconds (default: 30000)
 - `--verbose` - Show verbose output
@@ -83,6 +84,7 @@ npx @modelcontextprotocol/conformance server --url <url> [--scenario <scenario>]
 - `--url` - URL of the server to test
 - `--scenario <scenario>` - Test scenario to run (e.g., "server-initialize"). Runs all available scenarios by default
 - `--suite <suite>` - Suite to run: "active" (default; excludes pending and draft-spec scenarios), "all", "draft" (scenarios targeting the in-progress draft spec), or "pending"
+- `--requirements <revision>` - Run exactly what a spec revision requires, frozen at its release (see [Conformance Requirements](#conformance-requirements))
 - `--expected-failures <path>` - Path to YAML baseline file of known failures (see [Expected Failures](#expected-failures))
 - `--verbose` - Show verbose output
 
@@ -116,6 +118,73 @@ Scenarios that exchange no instrumented wire traffic (see issue #418) emit
 neither check. Like any other check, `wire-schema-valid` can be baselined via
 the expected-failures file.
 
+## Conformance Requirements
+
+`--suite` and `--spec-version` describe the suite as it is today. Neither answers
+"which scenarios did I need to pass to conform to the spec released on
+2026-07-28", because the suite keeps growing: a scenario merged after a revision
+ships still carries that revision's applicability tag, so it is
+indistinguishable from one that existed at release.
+
+A requirement set answers that question. Each `requirements/<revision>.yaml`
+names the scenarios a revision requires, for the two roles the specification
+defines: an MCP server acting as an OAuth resource server, and an MCP client
+acting as an OAuth client. It deliberately covers no authorization-server
+scenarios, because the specification puts authorization server implementation
+beyond its own scope, so those scenarios serve people deploying an authorization
+server rather than implementations of MCP itself.
+
+**Scenarios run at their revision's wire version.** That is the point of a
+per-revision set, not a label on it: the dated revisions through `2025-11-25`
+use the stateful initialize handshake and `2026-07-28` is stateless with
+per-request `_meta`, and a scenario emits different checks under each. A
+scenario belonging to both revisions must therefore be run twice, once under
+each set. Passing it on one wire says nothing about the other:
+
+```bash
+# what does conforming to 2026-07-28 actually require?
+npx @modelcontextprotocol/conformance list --requirements 2026-07-28
+
+# run exactly that
+npx @modelcontextprotocol/conformance server --url http://localhost:3000/mcp --requirements 2026-07-28
+```
+
+`--requirements` replaces `--suite`, `--spec-version` and `--scenario`, since the
+set already names every scenario that runs and the revision fixes the wire they
+run at. Without it nothing changes: the default is still to run everything, which
+is where completeness lives.
+
+`tier-check` takes several at once, and every one of them must pass for Tier 1:
+
+```bash
+npx @modelcontextprotocol/conformance tier-check --repo <owner/repo> \
+  --conformance-server-url http://localhost:3000/mcp \
+  --requirements 2025-11-25,2026-07-28
+```
+
+Only scenarios a revision actually requires decide the exit code and the pass
+rate. Anything run without being scored is reported separately and cannot fail
+the run.
+
+Requirement sets are frozen, and `not_scored` holds what a revision runs and
+reports without counting. Two reasons qualify, and the report names which
+applies:
+
+| Reason                | Meaning                                                                                                                                       |
+| --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
+| `extension`           | Optional by definition. SEP-1730: "Experimental features and protocol extensions (such as Tasks and MCP Apps) are not required for any tier." |
+| `added-after-release` | The scenario did not exist when the revision shipped, so no implementation could have been passing it.                                        |
+
+Both still run, so a failing extension stays visible in the report; neither moves
+the pass rate. Promoting an entry into the required lists is a deliberate,
+reviewable change, which is how the suite grows without retroactively failing
+anyone.
+
+This is the project's contract and lives in this repository. It is the opposite
+of an [expected-failures](#expected-failures) baseline, which lives in an
+implementation's own repository and records what that implementation knows it
+fails. A baselined failure is still a failure against a requirement set.
+
 ## Expected Failures
 
 SDKs that don't yet pass all conformance tests can specify a baseline of known failures. This allows running conformance tests in CI without failing, while still catching regressions.
@@ -326,26 +395,65 @@ Clones are cached under `.sdk-under-test/` and reused (fetched) on subsequent ru
 
 ## SDK Tier Assessment
 
-The `tier-check` subcommand evaluates an MCP SDK repository against [SEP-1730](https://github.com/modelcontextprotocol/modelcontextprotocol/issues/1730) (the SDK Tiering System):
+The `tier-check` subcommand evaluates an MCP SDK repository against [SEP-1730](https://github.com/modelcontextprotocol/modelcontextprotocol/issues/1730) (the SDK Tiering System). There are two ways to run it, and they answer different questions.
+
+### 1. The CLI, for the deterministic half
+
+Conformance pass rates, issue triage, P0 resolution, labels, releases, policy files. No AI, no judgment, reproducible.
 
 ```bash
-# Without conformance tests (fastest)
 gh auth login
-npm run --silent tier-check -- --repo modelcontextprotocol/typescript-sdk --skip-conformance
 
-# With conformance tests (start the everything server first)
-npm run --silent tier-check -- \
+# fastest: everything except conformance
+npx @modelcontextprotocol/conformance tier-check \
+  --repo modelcontextprotocol/typescript-sdk --skip-conformance
+
+# with conformance, scored against the revisions the SDK claims.
+# start the SDK's everything server first, in its own terminal.
+npx @modelcontextprotocol/conformance tier-check \
   --repo modelcontextprotocol/typescript-sdk \
-  --conformance-server-url http://localhost:3000/mcp
+  --conformance-server-url http://localhost:3000/mcp \
+  --client-cmd '<command that runs the SDK conformance client>' \
+  --requirements 2025-11-25,2026-07-28
+```
+
+Omit `--client-cmd` and the client leg is skipped and reported as a gap. Omit
+`--requirements` and scoring falls back to the suite as it stands today, which is
+not what you want for a tier claim; see [Conformance Requirements](#conformance-requirements).
+
+### 2. The skill, for the whole assessment
+
+The CLI cannot judge documentation coverage, dependency policy or roadmap quality, and those decide the tier as much as conformance does. The [`mcp-sdk-tier-audit`](.claude/skills/mcp-sdk-tier-audit/README.md) skill runs the CLI, adds those evaluations, and writes a full report with a remediation plan. In Claude Code, from a checkout of this repo:
+
+```
+/mcp-sdk-tier-audit <local-sdk-path> <conformance-server-url> '<client-cmd>' --requirements 2025-11-25,2026-07-28
 ```
 
-For a full AI-assisted assessment with remediation guide, use Claude Code:
+The server must already be running and stay up for the whole audit. Expect a few minutes.
+
+### Reading the result
 
 ```
-/mcp-sdk-tier-audit <local-sdk-path> <conformance-server-url>
+Scored against 2025-11-25 and 2026-07-28, each run at its own wire version.
+
+    Server   67/67 required scenarios (100%)
+    Client   50/50 required scenarios (100%)
+
+  Not scored (8 run, 4 failing, no effect on tier):
+    ✗ auth/dpop (extension)
+    ✗ json-schema-2020-12-preservation (added-after-release)
+
+Tier 1 Blockers:
+  • triage
+  • p0_resolution
 ```
 
-See [`.claude/skills/mcp-sdk-tier-audit/README.md`](.claude/skills/mcp-sdk-tier-audit/README.md) for full documentation.
+- **Required scenarios** are the only ones that move the number. `67/67` spans every revision listed: a scenario belonging to both runs once per revision, on that revision's wire, and both have to pass.
+- **Not scored** ran and is reported so you can see it, but cannot fail the tier. `extension` means optional by definition; `added-after-release` means the scenario did not exist when that revision shipped. A failure here is information, not a blocker.
+- **Not measured** is different from either, and means the run could not happen at all, e.g. a requirement set naming a scenario this build no longer has. Treat it as a broken invocation, never as an SDK failure.
+- **Tier 1 blockers** lists every requirement short of Tier 1. Conformance absent from that list means the SDK met every requirement each listed revision imposes.
+
+The exit code follows the same rule: it reflects required scenarios only, so an implementation that meets a revision's requirements exits 0 even with failing extensions.
 
 ## Architecture
 

package.json

@@ -27,7 +27,8 @@
     "prepare": "npm run build"
   },
   "files": [
-    "dist"
+    "dist",
+    "requirements"
   ],
   "bin": {
     "conformance": "dist/index.js"