ci: isolate NULLRUN_WAL_PATH per test (CI flakefix)

[maltsev-dev]

Summary

• Sprint 0 follow-up that closes a pre-existing xdist+coverage race in tests/conftest.py. Adds a single autouse fixture _isolated_wal(monkeypatch, tmp_path) that pins NULLRUN_WAL_PATH = tmp_path / "sdk.wal" for every test.
• No runtime code change. No public API change. No SDK_MIN_VERSION bump.

Root cause

Run 29809829695 (post-0.13.12 merge) failed on test (3.12) and coverage jobs with NullRunAuthError: Invalid API key.

The traceback (from coverage job 88568154478):

tests/test_state_compare_case_insensitive.py:28 in runtime
    rt = NullRunRuntime(api_key="test-key-12345678", _test_mode=True, polling=False)
src/nullrun/runtime.py:447 in __init__
    self._transport.start()
src/nullrun/transport.py:778 in start
    self._replay_from_wal()
src/nullrun/transport.py:748 in _replay_from_wal
    self._do_flush()
src/nullrun/breaker/circuit_breaker.py:334 in _call_sync
src/nullrun/transport.py:1195 in _send_batch_with_retry_info
src/nullrun/transport.py:311 in _retry_with_backoff
    raise err
nullrun.breaker.exceptions.NullRunAuthError: Invalid API key

Tests that build a NullRunRuntime(...) inline (without the make_test_runtime factory, which already pins NULLRUN_WAL_PATH = tmp_path / "sdk.wal") inherit the default WAL path from tempfile.gettempdir()/nullrun.wal. After a previous test session left events there or a parallel xdist worker is mid-flush, _replay_from_wal reads the buffer and tries to drain it against the real backend. With a placeholder test API key, the backend returns 401, and NullRunAuthError propagates back into the test fixture setup.

CI 3.12 hits this race more often than 3.10/3.11 due to thread scheduling differences in Transport.start(). On 3.10/3.11 the race resolves as a PytestUnhandledThreadExceptionWarning (which pytest ignores), on 3.12 the exception reaches the fixture setup before pytest can downgrade it.

Sprint 0 / 0.13.12 did not introduce the flake (the same race exists on master 29caae9), but 0.13.12 made it CI-visible because the Codecov badge no longer masks test failures behind a 0% coverage report.

Fix

One autouse fixture in tests/conftest.py:

@pytest.fixture(autouse=True)
def _isolated_wal(monkeypatch, tmp_path):
    monkeypatch.setenv("NULLRUN_WAL_PATH", str(tmp_path / "sdk.wal"))
    yield

make_test_runtime already does this for callers using the factory; this autouse extends the same isolation to the ~10 inline-NullRunRuntime(...) test files.

Verification

Local — 4 sequential pytest -n auto --cov=src/nullrun --cov-branch --cov-report=xml --cov-fail-under=0 runs:

• run 1: 1237 passed, 7 skipped, 27 warnings in 32.41s (cov 80.65%)
• run 2: 1237 passed, 7 skipped, 29 warnings in 32.33s
• run 3: 1237 passed, 7 skipped, 29 warnings in 32.56s
• run 4: 1237 passed, 7 skipped, 29 warnings in 32.64s

Pre-fix, ~1/3 of the same runs hit NullRunAuthError on the same set of tests. No flake observed across 4 consecutive runs after the fix.

• ruff check src/ — All checks passed
• mypy src/ — Success: no issues found in 34 source files

Test plan

• Local pytest with the fixture — 4 sequential green runs
• ruff check src/ / mypy src/
• CI on the PR — expect all 4 jobs (test 3.10 / 3.11 / 3.12 / coverage) green and Codecov stays at 79-80%

Out of scope (already documented)

The pre-existing flake on test_status.py::TestRecentErrors::test_recent_errors_populated_by_emit (sometimes seen under pytest -n auto + pytest-cov) and a few other tests share the same WAL-replay-flavor root cause but are masked by the autouse here. They are not addressed by this PR.

Diff: +31/-0, 1 file changed (tests/conftest.py).

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!