test: cover automation preflight requirements

[KirschBluteX]

• A human has tested these changes.

HUMAN:

I led this contract update from gap analysis through implementation and final validation. I traced how Agent Canvas derives deployment requirements from requires.integrations and the authoritative integration catalog, then updated the contract helper and every direct automation preflight fixture to record MCP transport, locator, authentication strategy, required secret names, and the applicable GitHub integration alternatives. I kept the security boundary explicit: preflight carries secret names only and never reads, stores, or exposes secret values. I also aligned this repository's contract with the four companion pull requests, personally ran and reviewed all 47 targeted tests, regenerated the skills, integrations, and automations catalogs, built both Python distribution formats in an isolated X: drive directory, investigated the unrelated native-Windows limitations in the broader test suite, and reviewed the deterministic Agent Canvas UI evidence. I retain responsibility for the technical decisions, implementation, validation, and final submission.

AGENT:

AI assistance was limited to targeted secondary review of the catalog-to-contract mapping, fixture completeness, secret-name-only boundary, cross-repository consistency, and organization of the verification evidence. The agent cross-checked that the reported commands and results support the PR claims and that the companion changes use the same contract. I personally evaluated that feedback, made the implementation decisions, ran the validation commands, reviewed the resulting artifacts and UI evidence, and approved the final submission.

---

Why

I traced the automation setup contract from requires.integrations, through the authoritative integration catalog, to the deployment preflight data consumed by Agent Canvas. I found that the existing contract fixtures did not fully record the deployment-checkable requirements derived from that catalog, leaving MCP transport, locator, authentication strategy, and required secret names under-specified at the contract boundary.

This change keeps those fixtures aligned with the behavior expected by Agent Canvas and the companion repositories.

Related to OpenHands/OpenHands#16565.

Summary

I updated the automation setup contract coverage to:

• derive MCP transport, locator, authentication strategy, and required secret names from the authoritative integration catalog rather than duplicating handwritten expectations
• record the resulting GitHub integration alternatives in every direct automation preflight fixture
• document explicitly that preflight data contains required secret names only and never secret values
• keep the contract fixtures aligned with the corresponding Agent Canvas and companion-repository changes

I kept the change focused on the contract helper and its fixtures. No secret values are read, stored, or exposed by this preflight contract.

Issue Number

OpenHands/OpenHands#16565

How to Test

I personally ran and reviewed the targeted contract tests, catalog generation, and package build.

Automation setup contract tests

uv run --frozen --group test pytest tests/test_automation_setup.py -q
47 passed

Catalog generation and project build

npm run build

The skills, integrations, and automations catalogs were generated successfully.

Python package build

uv build --out-dir <X-drive temp directory>

The source distribution and wheel were both built successfully. I used a temporary output directory on the X: drive so generated package artifacts remained isolated from the working tree.

I also examined the broader pytest-suite limitations on native Windows. Existing unrelated tests rely on locale-default reads for UTF-8 assets and Unix symlinks, so that broader suite is not currently portable to native Windows. The changed automation setup contract test file passed in full.

Video/Screenshots

I reviewed the deterministic mock UI failure and success states in the companion Agent Canvas PR. The corresponding screenshots are available in this Agent Canvas PR comment.

Notes

• I led this work from contract analysis and catalog mapping through implementation, fixture updates, cross-repository alignment, local validation, and final evidence review.
• I used AI assistance for targeted review and help organizing the PR evidence, while I retained responsibility for the technical decisions, implementation, validation, and final submission.
• Preflight carries secret names only; it never carries secret values.

Companion pull requests aligned with this change:

• feat: harden automation preflight validation OpenHands#16614
• feat: harden automation draft preflight validation automation#341
• feat: add repository access preflight software-agent-sdk#4504
• feat: add stored MCP preflight probe sandbox-server#6

[all-hands-bot]

👋 This PR needs a couple of things fixed before OpenHands can review it:

• the PR description's HUMAN: section needs at least 20 characters describing what you tested, not just the template placeholder

Push an update once this is addressed and this check re-runs automatically.

This is an automated check - no AI was used to generate this comment.