Skip to content

rpc-tests.yml is chronically flaky on main #2689

Description

@leighmcculloch

The RPC Tests workflow fails on a large majority of pushes to main: 23 of the last 30 push-triggered runs failed. The failures don't point to one regression — a different integration test fails each time, and the failing job varies too (seen on both ubuntu-latest-8-cores and macos-15-large). For example, integration::tx::clawback::clawback has failed on an immediate post-clawback balance assertion on macOS, while integration::auto_build::deploy_workspace_with_package_auto_builds has failed on Ubuntu in a different run. This pattern looks like timing/eventual-consistency races and possibly resource contention in the live-network integration suite rather than a single code defect.

A few directions that might be worth exploring: replacing immediate post-action assertions (like the clawback balance check) with a poll-with-timeout, adding retries for the integration test suite (e.g. via cargo-nextest --retries), and possibly separating out the flakiest tests into a non-blocking job so they stop gating main while they're investigated.

Below are 15 recent runs on main where this workflow failed, most recent first, as evidence of the pattern:

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status
    Backlog (Not Ready)

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions