Skip to content

test(acceptance): establish fail-closed core 24-command real-PTY matrix foundation #1295

Description

@dennisonbertram

Work type

Engineering change / feature slice

Change class

Infrastructure, CI, or process change

Why this matters

Issue #1088 requires all 38 TUI registry invocation variants to have real PTY evidence. The first 24 real paths need a reviewed, fail-closed registry/runner foundation so the remaining fixture children cannot disappear from accounting or be reported as N/A.

Protected path

Registry inventory -> applicability catalog -> 100x30 real harnesscli PTY -> sealed frame/transcript -> API/SSE/store probes -> bounded cleanup. Missing command cases must remain explicit failure/unproven records.

Acceptance contract

Given the current 38-variant registry, the runner must record exactly 24 real executed variants and exactly 14 named fixture-pending variants; it must never report the matrix complete or turn pending variants into passes. Each executed path retains rendered frame plus durable/API evidence and applicable multi-message continuation.

Current architecture and search evidence

#1088 run found 38 registry variants (37 canonical plus /continue). Existing canonical PTY batch covers 18; expanded test work covers 24. Remaining named fixture work is split in #1291-#1294. Use existing cmd/harnesscli/tui registry and ptyrunner; do not duplicate dispatch.

Cross-surface impact map

Test-only registry catalog/PTY runner/artifacts; isolated daemon/config/store. API/SSE/database are probes. TUI frames/focus/keys are required. No native GUI, product schema/API/provider/deployment changes. Include lifecycle cleanup, docs/logs/indexes, focused/race/full regression.

Product and UX contract

Acceptance-only, but each executed command follows actual keyboard/render semantics. Pending cases are visibly unproven rather than hidden.

In scope

Commit the existing red-first 24-command matrix foundation, strict 38-command accounting, artifacts and required documentation.

Out of scope

The 14 fixture-dependent commands (#1291-#1294), cron/callback, native GUI, or product fixes.

Coordination and dependencies

Child of #1088/#1010; #1291-#1294 build on this merged foundation. One PR must Closes #N; it must not close #1088.

Test-first plan

Registry completeness test red without all 38 entries; real PTY cases red before fixture/generalization. Run focused/race/full regression and retained artifacts.

Verification plan

Exact SHA, 100x30 frame artifacts, API/SSE/store postconditions, missing list, focused/race/full regression, independent review and hosted CI.

Rollout and rollback

Acceptance-only, revert isolated PR; no deployment/migration.

Documentation and handoff

Update plan, impact/logs/indexes and #1088 matrix.

Definition of done

  • 24 executed and 14 named-unproven counts are mechanically enforced
  • Executed paths have real PTY/rendered/durable artifacts
  • Pending paths cannot be misreported as pass
  • Tests, review, CI and docs are current

Process acknowledgement

  • I completed search and impact analysis and will update scope if evidence changes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions