test_sim hides xcodebuild simulator Accessibility destination timeout details

[dpearson2699]

Summary

test_sim can return a generic failed tool result when xcodebuild times out while resolving a simulator destination, but the most actionable lines from xcodebuild are only obvious in the MCP JSONL log. In an agent workflow this looked like the test run had stalled, even though the MCP call eventually returned success: false after its internal timeout.

This is not a project-specific compile/test failure. The tests never started because xcodebuild could not finish loading the requested simulator destination.

Environment

From doctor:

XcodeBuildMCP serverVersion: 2.5.2
Xcode: Xcode 26.5 - Build version 17F42 (/Applications/Xcode.app/Contents/Developer)
process-tree: Running under Xcode: No
axe: Available: Yes; UI automation: Yes; Video capture: Yes
xcodemake: Enabled: No; Binary: No
mise: Running under mise: No; Available: No
debugger-dap: Selected backend: lldb-cli; lldb-dap available: Yes
manifest-tools: Total tools: 79; Workflows: 15
runtime-registration: Registered tools: 55
xcode-ide-bridge: Workflow enabled: Yes; Connected: No; Proxied tools: 0
sentry: Enabled: Yes

Host:

macOS/Darwin 25.5.0

Sanitized tool request

The original project/scheme/test names were private, so this is the same shape with identifiers sanitized:

{
  "name": "test_sim",
  "arguments": {
    "extraArgs": [
      "-only-testing:ExampleAppTests/ExampleFeatureTests",
      "-only-testing:ExampleAppTests/ExampleServiceTests",
      "-only-testing:ExampleAppTests/ExampleRegressionTests"
    ],
    "progress": true
  }
}

Session defaults were set to a concrete iOS Simulator UUID destination. The actual simulator UUID is also omitted here, but the generated xcodebuild command used a destination of this form:

-destination id=<IOS_SIMULATOR_UUID>

Observed behavior

The MCP call returned a failed result after about 130 seconds:

{
  "success": false,
  "durationMs": 129998
}

The actionable underlying error was present in the MCP log, but easy to miss from the summarized tool result:

xcodebuild: error: Unable to find a device matching the provided destination specifier:
        { platform:iOS Simulator, id:<IOS_SIMULATOR_UUID> }

The requested device could not be found because it timed out while loading.

The raw xcodebuild output in the MCP log also included:

[MT] IDERunDestination: Timed out while loading Accessibility

And the available destinations list did not include the requested simulator at that moment; it only showed a physical device and the generic simulator placeholder, for example:

Available destinations for the "<sanitized scheme>" scheme:
        { platform:iOS, arch:arm64, id:<PHYSICAL_DEVICE_ID>, name:<physical device name> }
        { platform:iOS Simulator, id:dvtdevice-DVTiPhonePlaceholder-iphonesimulator:placeholder, name:Any iOS Simulator Device }

Why this was hard to diagnose

From the agent side, the important failure mode was not immediately visible: xcodebuild never reached test execution; it failed during simulator destination resolution because Accessibility/device loading timed out.

The root-cause lines were available only after manually inspecting the MCP JSONL logs under the Claude MCP cache directory. That makes the failure easy to misinterpret as an MCP hang or a long-running test run.

Expected behavior

When test_sim fails before tests start because xcodebuild cannot resolve the simulator destination, the tool result should surface the key root-cause lines directly, especially:

IDERunDestination: Timed out while loading Accessibility
The requested device could not be found because it timed out while loading.

Ideally the result would also classify this as a simulator destination/readiness failure rather than a generic test failure.

Possible improvements

1. Include the last relevant xcodebuild stderr/stdout lines directly in the test_sim failure response when destination resolution fails.

2. Detect this specific pattern and return a clearer diagnostic, for example:

   xcodebuild timed out loading Accessibility while resolving simulator destination <UUID>. The simulator/device service was not ready; tests did not start.
   

3. Include the raw build/test log path prominently in the failure response.

4. Consider exposing a per-call timeout or fail-fast option for test_sim, so agent workflows can bound simulator readiness failures explicitly.

Notes

This report is intentionally sanitized: project name, scheme name, test target names, simulator UUID, device ID, user paths, and app identifiers have been replaced with placeholders.

[dpearson2699]

Follow-up from a second repro/debugging session against XcodeBuildMCP 2.5.2 and local source at commit 9d56189b.

I think there are two related but distinct failure modes here:

1. Ordinary failing/compiling tests

   • In my second run, the project under test did have real compile/runtime test failures.
   • Running the same test_sim path through the CLI with --output json surfaced those failures correctly as a structured xcodebuildmcp.output.test-result with didError: true, diagnostics, test failures, buildLogPath, and xcresultPath.
   • So a failing test is not, by itself, enough to explain the MCP hang. It should be a normal failed tool result.

2. MCP call appears to hang when the underlying xcodebuild/simulator operation does not exit promptly

   • The MCP server direct path waits for test_simLogic() to finish before returning the tool response.
   • src/mcp/tools/simulator/test_sim.ts:252-265 awaits executeTestSim(...) and only then calls setXcodebuildStructuredOutput(...).
   • src/utils/test-common.ts:154-214 and src/utils/test-common.ts:222-248 await executeXcodeBuildCommand(...) for two-phase and single-phase test runs.
   • src/utils/build-utils.ts:205-231 awaits the command executor and returns a generic failed command result when xcodebuild exits non-zero.
   • src/utils/command.ts:70-249 spawns xcodebuild and resolves only on child exit/close; I did not see a tool-level timeout/watchdog there.
   • src/utils/tool-registry.ts:250-284 only converts the completed session into an MCP response after the handler returns.

That means if xcodebuild or CoreSimulator stalls during destination resolution/test bootstrap, the MCP call remains pending indefinitely from the client perspective. The CLI path can be externally bounded with shell timeouts and its JSON output makes failures much easier to inspect, but the MCP tool call itself does not appear to have an equivalent timeout or fail-fast classification.

Additional concurrency repro:

• A single full simulator test-suite run via the XcodeBuildMCP CLI completed successfully in about 52s.
• Then I queued two simultaneous full-suite CLI invocations against the same simulator UDID, using a timeout of about single_run * 2 + buffer.
• Result: one run passed in ~48s, while the other failed in ~60s before the app finished test bootstrap:

Tests failed
<app> encountered an error (Early unexpected exit, operation never finished bootstrapping - no restart will be attempted. (Underlying Error: Test crashed with signal kill before establishing connection.))

The test_sim manifest (manifests/tools/test_sim.yaml) does not mark the tool as stateful/routed, and I did not see a per-simulator lock around test_sim. So parallel calls targeting the same UDID can race each other through xcodebuild/CoreSimulator. That seems separate from the original Accessibility destination-timeout report, but it can create the same user-visible symptom: MCP appears to be hung or unreliable, while CLI eventually shows a simulator/test-bootstrap failure.

Hypothesis:

• The original issue is not caused by app test failures alone. It is more likely that the MCP path has no timeout/diagnostic boundary around long-running xcodebuild simulator operations, and no same-UDID serialization for simulator tests.
• When xcodebuild exits, XcodeBuildMCP generally has enough data to return didError: true with diagnostics.
• When simulator destination loading or test bootstrap stalls, the MCP handler is just awaiting the child process, so the client sees a long pending tool call rather than a bounded failure.
• When two test_sim calls hit the same simulator concurrently, CoreSimulator/test bootstrap can fail before the test bundle connects.

Possible fixes/improvements:

1. Add a test_sim tool-level timeout/watchdog, ideally configurable per call/session, that terminates the xcodebuild process tree and returns a structured failure with the last stdout/stderr lines plus log artifact paths.
2. Classify known simulator readiness/bootstrap patterns separately from ordinary test failures, e.g.:
   • IDERunDestination: Timed out while loading Accessibility
   • The requested device could not be found because it timed out while loading
   • Early unexpected exit, operation never finished bootstrapping
   • Test crashed with signal kill before establishing connection
3. Consider serializing or rejecting concurrent simulator test runs per simulator UDID, or at least documenting that test_sim is not safe to run concurrently against the same simulator.
4. Surface the final structured diagnostics and artifact paths more prominently in MCP text output for pre-test failures, because the raw CLI JSON is currently much easier to debug than the MCP tool result.

[dpearson2699]

Additional follow-up: the CLI path can also appear to hang indefinitely under same-simulator contention.

This is sanitized; project, scheme, test target, app, user paths, and simulator UUID are placeholders.

Setup

• XcodeBuildMCP: 2.5.2
• Xcode: 26.5 / 17F42
• macOS/Darwin: 25.5.0
• Multiple independent Claude Code sessions/worktrees were running simulator tests concurrently.
• The test commands targeted the same concrete iOS Simulator UUID.
• Command shape:

xcodebuildmcp simulator test \
  --project-path "<PROJECT>.xcodeproj" \
  --scheme "<SCHEME>" \
  --simulator-id "<IOS_SIMULATOR_UUID>" \
  --json '{"extraArgs":["-only-testing:<TEST_TARGET>/<TEST_SUITE>"],"progress":false}' \
  --output json

Some runs redirected stdout/stderr to a temp JSON file, e.g. /tmp/<sanitized-test-output>.json.

Observed behavior

• Several CLI-launched test shells stayed active for 50+ minutes.
• The CLI emitted no structured JSON result during the hang.
• The redirected output file was empty or never finalized.
• From the client/agent perspective, this looked like a hung command with no diagnostics.
• This happened after earlier CLI runs successfully surfaced normal compile/test failures as structured JSON, so ordinary test failures are not sufficient to explain the hang.

Additional investigation

After noticing the stale/running shells, process inspection did not show a healthy active xcodebuildmcp / xcodebuild / xctest tree that was making progress. The client shell/task layer still showed the commands as running with no output. This suggests either:

1. xcodebuild/CoreSimulator wedged before the CLI could emit JSON and the surrounding task layer continued waiting, or
2. the child process exited/was lost in a way that left the client shell bookkeeping stuck without a completed result.

The strongest trigger appears to be concurrent simulator test invocations against the same simulator UUID from multiple sessions/worktrees.

Why this matters

The CLI is much easier to debug than the MCP tool when xcodebuild exits, because it can emit structured JSON with diagnostics and artifact paths. But if the underlying Apple toolchain never exits, the CLI also appears to wait indefinitely. So the CLI is not a complete workaround for the MCP hang class; it only improves the completed-error path.

Suggested improvements

1. Add an internal watchdog/timeout to the CLI and MCP simulator test paths, not just client-side timeouts.
2. On timeout, terminate the full xcodebuild process tree and return structured output with:
   • elapsed time
   • simulator UUID
   • last stdout/stderr lines
   • whether no JSON result was emitted
   • known artifact/log paths if available
3. Detect or guard concurrent test invocations targeting the same simulator UUID.
   • Possible behaviors: serialize per UDID, reject the second run with a clear error, or document that callers must allocate one simulator per concurrent worker.
4. Classify same-simulator contention / bootstrap hangs separately from ordinary test failures. Related patterns include:
   • IDERunDestination: Timed out while loading Accessibility
   • destination resolution timeout
   • app/test runner bootstrap never established
   • Early unexpected exit, operation never finished bootstrapping
   • test runner killed before establishing connection

Working hypothesis

The current failure mode is not just poor rendering of a normal failed test. It appears to be a missing timeout/concurrency boundary around long-running xcodebuild simulator operations. Concurrent same-UDID test runs can leave the CLI/MCP caller waiting indefinitely because XcodeBuildMCP only returns once the underlying xcodebuild operation exits.