Raise Phase 6 test coverage with JWT, HTTP lifecycle, and CI matrix. - #14
Conversation
Close remaining SDK test gaps (JWT/config/guards, nested modules, HTTP JWT wiring) and add pytest-cov plus the 3.10–3.12 CI/tox matrix per dev-plan/PHASE-6-testing-coverage.md.
manish-wekan
left a comment
There was a problem hiding this comment.
Review — Phase 6 test coverage
Solid testing PR, and CI is green on 3.10/3.11/3.12. A few production fixes here are genuinely valuable — but two issues should be addressed before merge.
What's good
- Real bug fix in
core/pipeline.py: interceptor/middleware lambdas captured the loop variable by reference, so every interceptor in the chain would have invoked the last resolved one. The default-arg binding fixes a genuine late-binding bug. - Circular dependency detection in
DIContainer.resolvewith cleanfinally: discard— previously this recursed untilRecursionError. - Discovery server HTTP fixes:
Content-Length+Connection: closeprevents hanging keep-alive clients; CORS preflight handling is reasonable. - MCP error shape: tool exceptions become
CallToolResult(isError=True)instead of leaking tracebacks, with a test asserting noTraceback/HTML leaks. - CI matrix +
pip install -e ".[dev]"simplification.
Blockers
1. Dead code with contradictory security semantics — OAuthService._audience_ok
Never called by production code (raise_if_invalid uses the pre-existing _validate_audience), and the two helpers disagree on the missing-aud case: _validate_audience({}) → True (permissive, documented), _audience_ok({}) → False (reject). The only caller is a test literally named test_audience_ok_leftover_helper. Delete the helper and the test, or wire it in and reconcile the semantics — as-is, a reader can't tell which audience policy is correct.
2. _meta auth takes precedence over real HTTP headers
In _auth_metadata_from_request_ctx (core/app.py), client-supplied _meta.authorization / _meta.headers win; actual transport request headers are only a fallback. _meta is client-controlled JSON-RPC payload — behind a trusted proxy that sets/strips auth headers, a client could smuggle a different token via _meta and the guard would introspect that one. Transport headers should take precedence (with _meta as fallback for STDIO).
Should fix (cheap)
test_config_unreadable_env_file_writes_stderr:assert ... or captured.err == captured.erris a tautology — asserts nothing.test_config_env_comments_quotes_and_get_or_throw:assert ignored.get("BARE") != "plain" or ignored.get("ONLY") == "default"— right side already asserted true, so theBAREcheck is vacuous.test_raise_if_invalid_and_www_authenticatenever exercises the_validate_audiencebranch inraise_if_invalid(both cases short-circuit) — the one path that would catch a regression there is untested.- Coverage DoD says 50%+, but nothing enforces it — add
--cov-fail-under=50/fail_under = 50. - Duplicated pytest config in both
pytest.iniand[tool.pytest.ini_options]—pytest.iniwins, pyproject section is dead config that will drift. Pick one.
Minor
- DI circular-chain message joins over a
set— nondeterministic order; use a list for a diagnostic chain. EventEmitter.emit_syncsilentlyclose()es coroutine listeners — async handlers are dropped without warning.require_scopesonly works on async handlers (await func(...)) — undocumented.raise_if_invaliddocstring says it exists "for filter tests" — production docs shouldn't reference tests.test_audience_ok_leftover_helperis defined after theif __name__ == "__main__":block intest_oauth.py.- The
_call_toolcatch-all converts guardPermissionErrors intoisErrortool results (HTTP 200) rather than protocol-level auth errors — matches the new tests, but it's a behavior change worth calling out: clients can no longer distinguish "tool failed" from "unauthorized". - Stray double blank line in the
McpApplicationclass body.
Verdict: approve-worthy once blockers 1 & 2 are addressed. Happy to push a fix-up commit covering these.
- Remove dead OAuthService._audience_ok helper (contradicted _validate_audience on missing-aud; only kept alive by a test) - Give real transport headers precedence over client-controlled _meta auth slots in _auth_metadata_from_request_ctx - Add tests: raise_if_invalid active+wrong-audience branch, transport header precedence, _meta fallback when no transport headers - Fix tautological/vacuous assertions in test_auth_modules.py - Enforce the Phase 6 coverage DoD with fail_under = 50 (suite: ~86%) - Drop pytest.ini; pyproject [tool.pytest.ini_options] is the single source of truth - Make DI circular-dependency chain message deterministic (list, not set) - Warn when emit_sync skips an async listener instead of dropping silently - require_scopes now supports sync handlers too Co-authored-by: Cursor <cursoragent@cursor.com>
- Remove dead OAuthService._audience_ok helper (contradicted _validate_audience on missing-aud; only kept alive by a test) - Give real transport headers precedence over client-controlled _meta auth slots in _auth_metadata_from_request_ctx - Add tests: raise_if_invalid active+wrong-audience branch, transport header precedence, _meta fallback when no transport headers - Fix tautological/vacuous assertions in test_auth_modules.py - Enforce the Phase 6 coverage DoD with fail_under = 50 (suite: ~86%) - Drop pytest.ini; pyproject [tool.pytest.ini_options] is the single source of truth - Make DI circular-dependency chain message deterministic (list, not set) - Warn when emit_sync skips an async listener instead of dropping silently - require_scopes now supports sync handlers too
a9b7748 to
86f7de8
Compare
|
[ ] Test againt Nitrostudio |
|
NitroStudio testing is complete. The Python starter successfully connected through Streamable HTTP at http://localhost:3000/mcp. The tools/list, tools/call, and widget resources/read tests all passed. The follow-up changes from 86f7de8 are also on the branch. |
Summary
Closes the Phase 6 testing gap from dev-plan/PHASE-6-testing-coverage.md — raise coverage beyond the thin per-phase suites and add cross-cutting / integration tests before v1.0.0.
Adds dedicated coverage for DI edge cases (circular and missing deps, singletons), pipeline order (guards → middleware → pipes → interceptors → handler → filters, plus short-circuit and filter catch), MCP error shape, JWT/config/guards, nested modules, HTTP tools/call lifecycle, events, logger STDIO safety, CLI install, and the NitroTestingModule harness.
Wires measurement and the 3.10–3.12 matrix: pytest --cov=nitrostack, pytest.ini, tox.ini, and CI (.github/workflows/test.yml) now run coverage on Python 3.10, 3.11, and 3.12.
Small production fixes required so those tests are real, not mocked: circular-dep detection, OAuth token/audience errors, scope helpers, EventEmitter on/off/once, and pipeline closure binding.
Plan DoD: 34 test_*.py files (target was 25+). Branch: phase-6-testing-coverage.
Test plan: