Skip to content

Parallelize PR system tests via GitHub Actions matrix.#829

Open
PranjalManhgaye wants to merge 2 commits into
precice:developfrom
PranjalManhgaye:issue-789-parallel-system-tests-matrix
Open

Parallelize PR system tests via GitHub Actions matrix.#829
PranjalManhgaye wants to merge 2 commits into
precice:developfrom
PranjalManhgaye:issue-789-parallel-system-tests-matrix

Conversation

@PranjalManhgaye
Copy link
Copy Markdown
Collaborator

Summary

This PR splits PR system tests into two GitHub Actions matrix jobs (release_test_shard_1 and release_test_shard_2). Together they cover the same 48 cases as release_test; I left release_test itself unchanged so manual runs and other workflows still work as before, I set fail-fast: false so if one shard fails, the other keeps running => that makes failures easier to read and cheaper to retry.

Why

Right now, when one test fails you often have to re-run the whole suite and dig through one huge log. With two shards, you get smaller logs per job and can re-run only the failed matrix job.

If we have two precice-tests-vm runners, the shards can run in parallel. On a single runner they may still queue, but we still get clearer CI output => which matches what we discussed for this issue.

Test plan

  • I have already ran python3 validate_release_test_shards.py =>48 cases = 24 + 24
  • Local : system-tests-dev with --rundir on the precice-data partition (pass)
  • please check from there side for this PR with trigger-system-tests so we can verify on precice-tests-vm (I don’t have access to trigger that from my side ig)

Notes

  • Manual and latest-components workflows still use release_test; we can extend the matrix there in a follow-up if you want.
  • If two shards run at once on different runners, Docker may build the same images twice. I kept v1 simple; we can tighten that later if CI shows problems.

close #789

Add release_test_shard_1/2 covering the same cases as release_test,
and run them as separate matrix jobs for clearer logs and cheaper reruns.
@PranjalManhgaye
Copy link
Copy Markdown
Collaborator Author

@MakisH follow-ups (i think later, not this PR) :

  • same matrix for manual / latest-components workflows
  • more shards if we get more runners
  • docker build sharing only if ci shows we need it
  • option 2 (systemtests.py parallel) => skipped for now

@PranjalManhgaye PranjalManhgaye requested a review from MakisH June 5, 2026 14:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parallel execution of the system tests

1 participant