Skip to content

stream: fix merge() idle source draining#63567

Open
trivikr wants to merge 1 commit into
nodejs:mainfrom
trivikr:stream-iter-merge-drain-fast-sources
Open

stream: fix merge() idle source draining#63567
trivikr wants to merge 1 commit into
nodejs:mainfrom
trivikr:stream-iter-merge-drain-fast-sources

Conversation

@trivikr
Copy link
Copy Markdown
Member

@trivikr trivikr commented May 25, 2026

This updates stream/iter merge() so fast sources are not pulled to
completion while the merged consumer is idle.

Previously, after a source produced a batch, merge() immediately scheduled
another next() for that same source. If the merged iterator yielded once and
then the consumer paused, fast sources could continue draining in the
background.

The multi-source merge path now waits until the consumer requests the next
merged item before pulling again from the source that produced the previous
item.

Fixes: #63566


Assisted-by: openai:gpt-5.5

@nodejs-github-bot
Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/streams

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. stream Issues and PRs related to the stream subsystem. labels May 25, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented May 25, 2026

Codecov Report

❌ Patch coverage is 89.28571% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.32%. Comparing base (13feb34) to head (2894332).

Files with missing lines Patch % Lines
lib/internal/streams/iter/consumers.js 89.28% 12 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #63567      +/-   ##
==========================================
- Coverage   90.32%   90.32%   -0.01%     
==========================================
  Files         730      730              
  Lines      234653   234718      +65     
  Branches    43942    43964      +22     
==========================================
+ Hits       211959   212005      +46     
- Misses      14415    14433      +18     
- Partials     8279     8280       +1     
Files with missing lines Coverage Δ
lib/internal/streams/iter/consumers.js 96.48% <89.28%> (-0.14%) ⬇️

... and 40 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Defer pulling a merged source again until the consumer requests the
next merged item. This prevents fast sources from being drained while
the merged iterator is idle.

Fixes: nodejs#63566

Signed-off-by: Kamat, Trivikram <16024985+trivikr@users.noreply.github.com>
Assisted-by: openai:gpt-5.5
@trivikr trivikr force-pushed the stream-iter-merge-drain-fast-sources branch from 1d1b87f to 2894332 Compare May 27, 2026 01:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-ci PRs that need a full CI run. stream Issues and PRs related to the stream subsystem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

stream/iter: merge() proactively drains fast sources while merged consumer is idle

2 participants