Skip to content

test: deflake test-watch-mode-restart-esm-loading-error - #65623

Open
christianaurichzm wants to merge 1 commit into
nodejs:mainfrom
christianaurichzm:deflake-watch-mode-restart-esm-loading-error
Open

test: deflake test-watch-mode-restart-esm-loading-error#65623
christianaurichzm wants to merge 1 commit into
nodejs:mainfrom
christianaurichzm:deflake-watch-mode-restart-esm-loading-error

Conversation

@christianaurichzm

Copy link
Copy Markdown
Contributor

The previous deflake of this test, #63390, gave the two restart waits common.platformTimeout(10_000) and left the first restart() on the helper's 1000 ms default. That first call is the one still timing out.

It failed in 6 PRs across three daily reliability reports (2026-08-18, 2026-08-20, 2026-08-21), every time on macos15-x64, every time with:

Error: Timed out waiting for restart
    at Timeout.<anonymous> (test/sequential/test-watch-mode-restart-esm-loading-error.mjs:84:25)
    at listOnTimeout (node:internal/timers:685:17)

It has not recurred in the reports since 2026-08-21. Those reports only list failures that hit two or more PRs, and this one appeared on 3 of the 10 days covered even while it was active. It is not marked flaky in any status file, and the 1000 ms default is unchanged on main.

The first call also has a cold-start cost that the later restarts do not. --watch spawns a child process to run the script, so reaching the first Completed running costs two process startups, while subsequent restarts reuse the watch supervisor and only spawn the watched child again.

Wrapping the default in common.platformTimeout() would not have closed the gap either: on these jobs it at most doubles the value, well short of the 10 s the other two calls already ask for.

Which restart

Line 84 is inside the helper, shared by all three calls, so the stack alone does not say which one timed out. The reported durations do: 1344 ms, 1409 ms, 1494 ms. The first restart fails at roughly process startup plus 1000 ms, whereas a timeout from either later call would necessarily push the run past 10 s.

No reported failure lands anywhere near 11 s, so the 10 s waits were never the ones exhausted. A macOS-specific cold-start stall is still possible; CI should tell us if the larger budget is insufficient.

Reproduction

Idle, the first Completed running arrives ~60 ms after spawn, so the 1000 ms budget normally has a wide margin and plain CPU load does not close it: starving the parent delays its own setTimeout by the same amount, and both
sides of the race slip together. The race appears when child startup is delayed without delaying the parent timer. Parallel test execution on CI can produce that; uniform load does not.

Delaying child startup reproduces it, with NODE_OPTIONS=--require pointing at:

const t = Date.now();
while (Date.now() - t < Number(process.env.SLOW_MS || 0));

The --watch supervisor does not run user preloads, so only the spawned child is delayed, which inflates one of the two startups the budget covers without touching the test.

added child startup before after
0 ms 0 / 24 0 / 24
600 ms 0 / 24 0 / 24
900 ms 0 / 24 0 / 24
950 ms 24 / 24 0 / 24
1000 ms 24 / 24 0 / 24
1200 ms 24 / 24 0 / 24

The cliff falls between 900 and 950 ms, which with the ~60 ms baseline is exactly the 1000 ms budget. At 950 ms, over a larger sample: 400 / 400 failures before, 0 / 400 after. The widened failure carries the signature above.

Unwidened, the flake did not reproduce here at all: 0 / 480 on both variants with 24 concurrent processes on 16 cores. Everything above is Linux x64; the macOS behaviour is the reports' and CI's to confirm, not something measured locally.

Scope

That timer bounds how long the test waits for watch mode to make progress. It is not an assertion about restart latency, and the assertions on stdout and stderr are untouched. Idle runtime is unchanged at 2.14 s, since the budget is a ceiling the passing path never reaches.

The same helper is duplicated in sequential/test-watch-mode.mjs, with the same default and nine call sites that take it. That file has carried PASS, FLAKY since #44898, which is why only this one surfaces in the reports. Dropping that mark is a separate change and is not attempted here.

Testing

  • tools/test.py sequential/test-watch-mode-restart-esm-loading-error: passes
  • eslint: passes
  • core-validate-commit: passes

Refs: #63390
Refs: https://github.com/nodejs/reliability/blob/main/reports/2026-08-18.md
Refs: https://github.com/nodejs/reliability/blob/main/reports/2026-08-20.md
Refs: https://github.com/nodejs/reliability/blob/main/reports/2026-08-21.md
Refs: #44898

The previous deflake of this test, nodejs#63390, gave the two restart
waits common.platformTimeout(10_000) and left the first restart()
on the helper's 1000 ms default. That first call is the one still
timing out on CI.

It is also the call that needs the budget most. --watch spawns a
child process to run the script, so reaching the first Completed
running costs two process startups, while the later restarts pay for
one. Wrapping the default in platformTimeout() would not have been
enough on its own: none of its multipliers reaches a factor of ten,
so the budget would still be 1000 ms on these jobs, or 2000 ms in a
debug build.

Give that call the budget the other two already ask for, stated once in
the default rather than three times. The timer bounds how long the test
waits for watch mode to make progress, it is not an assertion about
restart latency, and the assertions on stdout and stderr are untouched.

The same helper is duplicated in test-watch-mode.mjs, with the same
default and nine call sites that take it. That file has carried
PASS, FLAKY since nodejs#44898, which is why only this one surfaces in the
reliability reports.

Signed-off-by: Christian Aurich <christian.aurichzm@gmail.com>
@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. test Issues and PRs related to Node.js core tests and test infrastructure. labels Aug 28, 2026
@codecov

codecov Bot commented Aug 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.06%. Comparing base (0544741) to head (c48847c).
⚠️ Report is 8 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #65623      +/-   ##
==========================================
- Coverage   90.07%   90.06%   -0.02%     
==========================================
  Files         751      751              
  Lines      254921   254921              
  Branches    48129    48122       -7     
==========================================
- Hits       229627   229600      -27     
- Misses      16479    16497      +18     
- Partials     8815     8824       +9     

see 24 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-ci PRs that need a full CI run. test Issues and PRs related to Node.js core tests and test infrastructure.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants