Skip to content

[Bug]: Provider API-key capture test depends on unrelated 3-second server readiness #1052

Description

@dennisonbertram

Summary

TestMatrix_ProviderAPIKeyCapture waits for the full harness HTTP server to become healthy within 3 seconds even though its contract is only to verify the API key passed into the provider factory. Under the hosted race suite, parallel startup exceeded that deadline and failed PR #1051 after the provider/server became ready immediately afterward.

Acceptance criteria

  • The test synchronizes directly on provider-factory invocation/API-key capture, not HTTP readiness timing.
  • The test still proves OPENAI_API_KEY reaches openai.Config.APIKey.
  • The harness process is shut down deterministically and cannot leak a goroutine.
  • The test passes repeatedly under normal and -race execution.
  • The complete regression suite passes in normal, race, and coverage modes.

In scope

  • Test synchronization for TestMatrix_ProviderAPIKeyCapture.
  • Required regression, plan, impact map, logs, and indexes.

Out of scope

  • Production startup sequencing or health endpoint behavior.
  • Global timeout increases.

Current-architecture search evidence

  • cmd/harnessd/main_test.go:TestMatrix_ProviderAPIKeyCapture captures provider config in the injected provider factory.
  • The test then calls awaitHealthy(..., 3*time.Second), coupling an API-key assertion to unrelated server readiness.
  • Hosted race run 30583930460 failed at main_test.go:4130, while logs show the same server later listening on the allocated address.

Cross-surface impact

  • Harness runtime/API: none; test-only synchronization.
  • Cron/callback/continuation: none directly; this removes a false-negative gate blocking their repair chain.
  • TUI/native GUI: none.
  • Persistence/security: none; the test continues to assert only the injected sentinel key and does not log secrets.
  • CI: stabilizes the race suite without weakening product assertions.

Test-first plan

  1. Reproduce by running the focused test repeatedly under -race with concurrent package pressure or a deliberately delayed server path if needed.
  2. Add a provider-invocation channel and assert the captured key after that signal.
  3. Preserve deterministic interrupt and bounded shutdown assertion.
  4. Run focused repeated normal/race tests and ./scripts/test-regression.sh.

Rollout / rollback

Test-only change. Roll back the PR if the exact-head hosted normal/race checks or full local gate regress.

Evidence

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions