Skip to content

fix: handle internal API stream chunk boundaries - #2252

Merged
felladrin merged 1 commit into
felladrin:mainfrom
snowyukitty:fix/internal-api-stream-boundaries
Jul 30, 2026
Merged

fix: handle internal API stream chunk boundaries#2252
felladrin merged 1 commit into
felladrin:mainfrom
snowyukitty:fix/internal-api-stream-boundaries

Conversation

@snowyukitty

Copy link
Copy Markdown
Contributor

Description

Fixes #2177.

The internal inference client previously decoded and parsed each transport
chunk independently. A chunk can end in the middle of either an SSE data line
or a multi-byte UTF-8 character, so a valid streamed response could fail with a
SyntaxError and discard the rest of the answer.

The parser now keeps the TextDecoder in streaming mode, buffers text between
reads, and parses only complete newline-delimited frames. It also flushes the
last unterminated frame at EOF and skips an isolated malformed data line while
preserving the existing error-frame behavior.

Regression tests feed the response one byte at a time through a multi-byte
character and verify that a malformed line does not prevent later valid
content from streaming.

Type of Change

  • Bug fix
  • New feature
  • Documentation
  • Other (refactor, build, chore)

How to test

  1. Run npm ci.
  2. Run npm test.
  3. Run npm run lint.

The focused regression suite is:

npx vitest run client/modules/textGenerationWithInternalApi.test.ts

Checklist

  • npm run lint passes
  • Tests pass (npm run test), with tests added where it made sense

@felladrin felladrin left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent! Thank you for this!

@felladrin
felladrin merged commit 78b3d8a into felladrin:main Jul 30, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Internal-API stream parsing is not resilient to chunk boundaries

2 participants