fix: handle internal API stream chunk boundaries - #2252
Merged
felladrin merged 1 commit intoJul 30, 2026
Conversation
felladrin
approved these changes
Jul 30, 2026
felladrin
left a comment
Owner
There was a problem hiding this comment.
Excellent! Thank you for this!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #2177.
The internal inference client previously decoded and parsed each transport
chunk independently. A chunk can end in the middle of either an SSE data line
or a multi-byte UTF-8 character, so a valid streamed response could fail with a
SyntaxErrorand discard the rest of the answer.The parser now keeps the
TextDecoderin streaming mode, buffers text betweenreads, and parses only complete newline-delimited frames. It also flushes the
last unterminated frame at EOF and skips an isolated malformed data line while
preserving the existing error-frame behavior.
Regression tests feed the response one byte at a time through a multi-byte
character and verify that a malformed line does not prevent later valid
content from streaming.
Type of Change
How to test
npm ci.npm test.npm run lint.The focused regression suite is:
npx vitest run client/modules/textGenerationWithInternalApi.test.tsChecklist
npm run lintpassesnpm run test), with tests added where it made sense