fix(codex): bound oversized rollout inspection - #1136
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. 🗂️ Base branches to auto review (2)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
✅ Deterministic PR hygiene checks passed. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3a3323b3dd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| latest: Record<string, unknown> | undefined, | ||
| ): NativeRoutedResidueResult | { kind: "continue"; partial: string; first: Record<string, unknown> | undefined; latest: Record<string, unknown> | undefined } { | ||
| let rest = partial; | ||
| let newline = rest.indexOf("\n"); |
There was a problem hiding this comment.
Avoid rescanning an unterminated record on every chunk
When a rollout contains one long JSONL record—or malformed content without a newline—while remaining under the 64 MiB limit, every chunk is appended to partial and this indexOf rescans the entire accumulated string, making inspection quadratic. An equivalent 32 MiB scan did not finish within 20 seconds on Bun 1.2.14; because this synchronous classifier is called from CLI and management flows, such a rollout can block the process despite the new bound. Preserve a scan cursor across chunks or impose a bounded per-record limit.
Useful? React with 👍 / 👎.
Summary
Bounds rollout inspection in
src/codex/native-residue.tsso an oversized rollout file can no longer drive an unbounded read.Reads are now capped to the observed size, UTF-8 is decoded incrementally, JSONL is parsed from bounded chunks, the pathname is re-stat'd so a file swapped between stat and read is detected, lines are scanned once instead of repeatedly, and a BOM is preserved.
Residue classification is unchanged. The bound governs how much is read, not what counts as residue — a bounded reader that quietly reclassified a truncated rollout as clean would hide exactly the state this code exists to find. The tests cover the oversized, malformed, truncated, and BOM cases.
Attribution
This is @Simon-Opopeee's work from #1115 — all five commits cherry-picked with their authorship intact, kept as a chain rather than squashed so the review history stays legible. #1115 stays open for them. Thanks also for splitting the unrelated native-history bound out on request; that made this one straightforward to land.
Stack 4 of the 260806 attribution campaign, stacked on #1135. Planning unit:
devlog/_plan/260806_stacked_bug_campaign/.Verification
bun test tests/codex-native-residue.test.ts— 65 pass, 0 failbun run typecheck— exit 0bun run privacy:scan— passedbun run testvia the repository prepush gate — passedChecklist