Skip to content

Raise Phase 6 test coverage with JWT, HTTP lifecycle, and CI matrix. - #14

Open
aviabhijit55-ship-it wants to merge 2 commits into
nitrocloudofficial:developfrom
aviabhijit55-ship-it:phase-6-testing-coverage
Open

Raise Phase 6 test coverage with JWT, HTTP lifecycle, and CI matrix.#14
aviabhijit55-ship-it wants to merge 2 commits into
nitrocloudofficial:developfrom
aviabhijit55-ship-it:phase-6-testing-coverage

Conversation

@aviabhijit55-ship-it

Copy link
Copy Markdown

Summary
Closes the Phase 6 testing gap from dev-plan/PHASE-6-testing-coverage.md — raise coverage beyond the thin per-phase suites and add cross-cutting / integration tests before v1.0.0.
Adds dedicated coverage for DI edge cases (circular and missing deps, singletons), pipeline order (guards → middleware → pipes → interceptors → handler → filters, plus short-circuit and filter catch), MCP error shape, JWT/config/guards, nested modules, HTTP tools/call lifecycle, events, logger STDIO safety, CLI install, and the NitroTestingModule harness.
Wires measurement and the 3.10–3.12 matrix: pytest --cov=nitrostack, pytest.ini, tox.ini, and CI (.github/workflows/test.yml) now run coverage on Python 3.10, 3.11, and 3.12.
Small production fixes required so those tests are real, not mocked: circular-dep detection, OAuth token/audience errors, scope helpers, EventEmitter on/off/once, and pipeline closure binding.
Plan DoD: 34 test_*.py files (target was 25+). Branch: phase-6-testing-coverage.

Test plan:

  • 1. pytest tests --cov=nitrostack --cov-report=term-missing is green and line coverage is 50%+
  • 2. find tests -name 'test_*.py' | wc -l is 25+
  • 3. tox (or the CI matrix) passes on 3.10, 3.11, 3.12
  • 4. Run the full suite twice and confirm the same pass/fail (no flakes)
  • 5. Spot-check new files: test_di_edge_cases.py, test_pipeline_order.py, test_error_handling.py, test_lifecycle_http.py, test_transport_http.py
  • 6. Confirm existing Phase 1–5 tests (test_tasks, test_oauth, widgets, CLI, transports) still pass
  • 7. Confirm no raw traceback leaks on a failing HTTP/STDIO tool call (MCP error shape)

Close remaining SDK test gaps (JWT/config/guards, nested modules, HTTP JWT wiring) and add pytest-cov plus the 3.10–3.12 CI/tox matrix per dev-plan/PHASE-6-testing-coverage.md.

@manish-wekan manish-wekan left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — Phase 6 test coverage

Solid testing PR, and CI is green on 3.10/3.11/3.12. A few production fixes here are genuinely valuable — but two issues should be addressed before merge.

What's good

  • Real bug fix in core/pipeline.py: interceptor/middleware lambdas captured the loop variable by reference, so every interceptor in the chain would have invoked the last resolved one. The default-arg binding fixes a genuine late-binding bug.
  • Circular dependency detection in DIContainer.resolve with clean finally: discard — previously this recursed until RecursionError.
  • Discovery server HTTP fixes: Content-Length + Connection: close prevents hanging keep-alive clients; CORS preflight handling is reasonable.
  • MCP error shape: tool exceptions become CallToolResult(isError=True) instead of leaking tracebacks, with a test asserting no Traceback/HTML leaks.
  • CI matrix + pip install -e ".[dev]" simplification.

Blockers

1. Dead code with contradictory security semantics — OAuthService._audience_ok
Never called by production code (raise_if_invalid uses the pre-existing _validate_audience), and the two helpers disagree on the missing-aud case: _validate_audience({})True (permissive, documented), _audience_ok({})False (reject). The only caller is a test literally named test_audience_ok_leftover_helper. Delete the helper and the test, or wire it in and reconcile the semantics — as-is, a reader can't tell which audience policy is correct.

2. _meta auth takes precedence over real HTTP headers
In _auth_metadata_from_request_ctx (core/app.py), client-supplied _meta.authorization / _meta.headers win; actual transport request headers are only a fallback. _meta is client-controlled JSON-RPC payload — behind a trusted proxy that sets/strips auth headers, a client could smuggle a different token via _meta and the guard would introspect that one. Transport headers should take precedence (with _meta as fallback for STDIO).

Should fix (cheap)

  • test_config_unreadable_env_file_writes_stderr: assert ... or captured.err == captured.err is a tautology — asserts nothing.
  • test_config_env_comments_quotes_and_get_or_throw: assert ignored.get("BARE") != "plain" or ignored.get("ONLY") == "default" — right side already asserted true, so the BARE check is vacuous.
  • test_raise_if_invalid_and_www_authenticate never exercises the _validate_audience branch in raise_if_invalid (both cases short-circuit) — the one path that would catch a regression there is untested.
  • Coverage DoD says 50%+, but nothing enforces it — add --cov-fail-under=50 / fail_under = 50.
  • Duplicated pytest config in both pytest.ini and [tool.pytest.ini_options]pytest.ini wins, pyproject section is dead config that will drift. Pick one.

Minor

  • DI circular-chain message joins over a set — nondeterministic order; use a list for a diagnostic chain.
  • EventEmitter.emit_sync silently close()es coroutine listeners — async handlers are dropped without warning.
  • require_scopes only works on async handlers (await func(...)) — undocumented.
  • raise_if_invalid docstring says it exists "for filter tests" — production docs shouldn't reference tests.
  • test_audience_ok_leftover_helper is defined after the if __name__ == "__main__": block in test_oauth.py.
  • The _call_tool catch-all converts guard PermissionErrors into isError tool results (HTTP 200) rather than protocol-level auth errors — matches the new tests, but it's a behavior change worth calling out: clients can no longer distinguish "tool failed" from "unauthorized".
  • Stray double blank line in the McpApplication class body.

Verdict: approve-worthy once blockers 1 & 2 are addressed. Happy to push a fix-up commit covering these.

manish-wekan added a commit to aviabhijit55-ship-it/nitrostack-python-sdk that referenced this pull request Aug 25, 2026
- Remove dead OAuthService._audience_ok helper (contradicted
  _validate_audience on missing-aud; only kept alive by a test)
- Give real transport headers precedence over client-controlled _meta
  auth slots in _auth_metadata_from_request_ctx
- Add tests: raise_if_invalid active+wrong-audience branch, transport
  header precedence, _meta fallback when no transport headers
- Fix tautological/vacuous assertions in test_auth_modules.py
- Enforce the Phase 6 coverage DoD with fail_under = 50 (suite: ~86%)
- Drop pytest.ini; pyproject [tool.pytest.ini_options] is the single
  source of truth
- Make DI circular-dependency chain message deterministic (list, not set)
- Warn when emit_sync skips an async listener instead of dropping silently
- require_scopes now supports sync handlers too

Co-authored-by: Cursor <cursoragent@cursor.com>
- Remove dead OAuthService._audience_ok helper (contradicted
  _validate_audience on missing-aud; only kept alive by a test)
- Give real transport headers precedence over client-controlled _meta
  auth slots in _auth_metadata_from_request_ctx
- Add tests: raise_if_invalid active+wrong-audience branch, transport
  header precedence, _meta fallback when no transport headers
- Fix tautological/vacuous assertions in test_auth_modules.py
- Enforce the Phase 6 coverage DoD with fail_under = 50 (suite: ~86%)
- Drop pytest.ini; pyproject [tool.pytest.ini_options] is the single
  source of truth
- Make DI circular-dependency chain message deterministic (list, not set)
- Warn when emit_sync skips an async listener instead of dropping silently
- require_scopes now supports sync handlers too
@manish-wekan
manish-wekan force-pushed the phase-6-testing-coverage branch from a9b7748 to 86f7de8 Compare August 25, 2026 08:44
@manish-wekan

Copy link
Copy Markdown
Collaborator

[ ] Test againt Nitrostudio

@bhavani-devi0806

Copy link
Copy Markdown

NitroStudio testing is complete. The Python starter successfully connected through Streamable HTTP at http://localhost:3000/mcp. The tools/list, tools/call, and widget resources/read tests all passed.

The follow-up changes from 86f7de8 are also on the branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants