Skip to content

fix(runner): inject the Tier-1 reporter on one line (DEV-2557) - #213

Merged
demtario merged 2 commits into
masterfrom
fix/DEV-2557-reporter-line-offset
Aug 18, 2026
Merged

fix(runner): inject the Tier-1 reporter on one line (DEV-2557)#213
demtario merged 2 commits into
masterfrom
fix/DEV-2557-reporter-line-offset

Conversation

@demtario

Copy link
Copy Markdown
Contributor

What was broken

injectReporter prepended the whole reporter body to the JS module entry — runner/packages/runtime/src/monitor.ts:684, REPORTER_SOURCE + "\n" + source. Everything the bundler reports for that file is offset by whatever we put above it, and the visitor is shown those positions verbatim, so a syntax error typed on line 31 of a 70-line file came back as (257:22). The prefix is 283 lines today and was 226 at the two Sentry release SHAs the events are tagged with (measured with git show <sha>:runner/packages/runtime/src/monitor.ts at 1fc5b1fb and cecbc7bc, not inherited — the ticket's premise number of 186 is wrong, and 226/283 is what the code actually was).

The module branch now prepends REPORTER_MODULE_LINE, a single physical line: try{(0,eval)(${JSON.stringify(REPORTER_SOURCE)})}catch(e){}. One line of prefix, one line of shift. REPORTER_SOURCE itself is byte-unchanged, injectReporterIntoHtml and the HTML branch are untouched, and sandpack.ts's file plumbing is untouched. The indirect (0,eval) form is what makes the reporter's bare window/parent/document/location resolve, since indirect eval evaluates in global scope; the try/catch is there so that if eval is ever unavailable it costs monitoring on this path and never the demo.

The regression this PR also fixes, found on review of its own first commit

Collapsing the reporter to one line fixed the reported line number and broke the code frame. Babel prints the two lines above the fault verbatim, so a syntax error on authored line 1 or 2 of the entry renders all 12.6 KB of REPORTER_MODULE_LINE into the compile message ahead of the offending line, and MONITOR_COMPILE_MESSAGE_MAX (2,000) then cuts. Measured on the vue starter's entry with an unterminated string on line 1, driving the real @babel/standalone transform over the real injectReporter output: 289 characters of usable message with the caret intact when the reporter was still inlined, 12,872 characters with it on one line and the visitor's own source line first appearing around character 12,400 — i.e. gone. That is DEV-2550's buried-diagnostic failure (DEMOS-15) returning on the same channel, the error card and Sentry.captureException, sourced from our own bytes instead of an inline source map. boundCompileMessage in runner/packages/runtime/src/sandpack.ts:266 now replaces that line with a short marker, next to the existing source-map strip and before the cap, so the diagnostic and the caret survive.

The strip matches the exported constant with split/join rather than a pattern, deliberately: a regex over the injected shape would keep passing its own tests while silently ceasing to match a reworded injection, and the burying would come back with nothing red. The constant is the coupling.

Sentry and ClickUp

DEMOS-15 (contributing cause) — the sandpack-compile class, which is the only visitor-facing class this touches. DEMOS-17 and DEMOS-18 are runtime stack frames rather than compile diagnostics, so they are unaffected either way; the ticket lists them, and that part of its premise does not hold. Task: https://app.clickup.com/t/86cb68qpm (DEV-2557).

What was verified, with the actual commands

pnpm build then pnpm test then pnpm typecheck, run raw at runner/ (never through an rtk filter): 575 tests, 575 pass, 0 fail, 0 skipped, and all four typecheck projects clean. Seven tests are new. The module entry is shifted by exactly one line, checked line-by-line against the authored split rather than against a constant. The injected prefix carries no LF, CR, U+2028 or U+2029 — built with String.fromCharCode so a formatter cannot normalise away the characters the test exists to reject, and pointed at the two JSON.stringify does not escape. The eval form is executed, not assumed: the injected entry runs in a node:vm context whose globals are the same stubs the existing suite uses, and its payload is compared against the inlined run's, which required extracting makeReporterStubs() because runReporter's new Function(param…) trick structurally cannot reach an indirect eval. Module-entry idempotency and byte-determinism are pinned (the existing pair covered the HTML entry only). The code-frame test drives real babel over real injectReporter output and asserts its own premise first — untreated, the message is over the cap and the visitor's line is past it — so it cannot pass by the blob never having been there; verified by mutation, dropping the strip fails it and nothing else.

The eval itself could not be checked by e2e: monitorDemos is reportingEnabled && VITE_MONITOR_DEMOS === "1", and reportingEnabled is pinned to the production host and gated on navigator.webdriver (DEV-2540), so under Playwright the injection never runs at all. It was checked by hand instead, with the gate temporarily forced open in apps/authoring/src/sentry.ts: window.__hotRunnerMonitor is true inside the real Sandpack preview iframe for both a vue and a react entry. That patch was reverted and is not in this diff.

Deliberately out of scope

This does not make line numbers correct, and the PR should not be read as claiming it does. The entry is generally transpiled before the injection — transpileFilesForParcel for the parcel entries, and babel does not use retainLines — so a reported line is still a compiled line. Measured on the vue starter, a syntax error typed on authored line 11 now reports line 14: the residual +3 is that entry's own TS transform. What changed is that we stopped adding 283 lines of our own on top of it. Source maps are the actual path to correct line numbers, and with the injection down to one line the transpile map is the only remaining unknown; that is a separate ticket, and this change is a prerequisite for it rather than a competitor.

The HTML entry keeps its ~283-line shift. That path is Tier-2's only monitoring channel (runner/workers/api/src/monitor-inject.ts), and widening the eval bet to it wants its own decision, so a visitor with an inline <script> syntax error in index.html still gets a wrong line.

The eval is the sole monitoring channel for more starters than the plan assumed. runner/config/frameworks.json puts create-react-app, create-react-app-typescript and vue-cli outside HTML_ENTRY_ENVS, so resolveSandboxEntry returns the JS module for all of them and there is no <script> fallback — roughly eight starters, not the single vue entry the plan described. If a preview ever carries a script-src without 'unsafe-eval', the catch swallows it and monitoring goes off silently on all of them. Parcel and static entries keep the HTML channel and are unaffected.

The regression fixture runs @babel/standalone 8, while the message that actually reaches boundCompileMessage comes from the hosted bundler's babel — 6.26 for parcel entries, whatever the vue-cli and CRA templates run for the rest — and neither is pinned by us. No babel code-frame implementation truncates context lines, and if one ever did the strip would stop matching and the behaviour would degrade to the pre-fix message rather than to anything worse, so this is a known gap between the test and production rather than a blocker.

One comment was corrected rather than left standing: the acorn ES5 guard claimed the symptom of a slip in REPORTER_SOURCE is a blank Tier-1 preview, which stopped being true for the module path the moment the reporter went inside a try/catch — a parse failure there is now swallowed and costs monitoring silently. The guard must also stay pointed at REPORTER_SOURCE and never at the injected output, which is a single string literal that parses at ES5 forever; there is a comment saying so, because repointing it would be a permanent false green.

🤖 Generated with Claude Code

demtario and others added 2 commits August 17, 2026 16:22
The module-entry injection prepended the whole reporter body to the entry
file, so every position the bundler reported for it was shifted by the
reporter's length -- 283 lines today, 226 at the releases the Sentry events
were tagged with (both measured, not carried over). The visitor is shown
those positions verbatim, which is how a syntax error in a 70-line file came
back as "(257:22)".

The reporter is now handed to the entry as a single physical line: a
JSON-encoded indirect eval, computed once at module load.

Measured live on the vue starter, a syntax error typed on authored line 11:
reported line 316 before, 14 after. Note the residual +3 -- that entry is
transpiled before the injection, so this does NOT make line numbers correct
for any entry. It removes the distortion we add, nothing more, and the
commit and comments deliberately claim only that.

Not an arithmetic correction of the reported numbers, deliberately. The
entries run transpileFilesForParcel before the injection and babel does not
use `retainLines`, so a bundler position is already in compiled coordinates
-- subtracting a constant would replace an obviously-absurd number with a
plausible, confident, still-wrong one. Source maps are the way to close the
rest, and this is a prerequisite for that rather than a competitor.

A separate reporter module was the other candidate and was rejected: it
would put a specifier the author never wrote into the graph next to a
moving entry path, entangled with resolveSandboxEntry, sameFiles and
stampEntry, and its failure mode is a blank preview. The eval form has no
graph interaction. Its try/catch means a blocked eval costs monitoring on
that path and never the demo.

Verified live rather than inferred, since the try/catch would hide a
failure: window.__hotRunnerMonitor is true inside the real Sandpack preview
iframe for both a vue and a react entry. This needed the reporting gate
forced open locally -- monitorDemos is host-pinned and webdriver-gated, so
the obvious "build with VITE_MONITOR_DEMOS=1 and look" check silently
proves nothing.

The HTML path is untouched -- it is Tier-2's only monitoring channel and
widening the eval bet to it wants its own decision.

Byte-determinism, ES5 and idempotency all still hold and are now tested for
the module entry too: the prefix is a pure function of a constant, carries
no line terminator of any kind, and MONITOR_MESSAGE_TYPE survives the JSON
escaping so a double injection stays a no-op. The acorn ES5 guard stays
pointed at REPORTER_SOURCE, with a comment saying why -- under the new
shape the injected output is one string literal that parses at ES5 forever,
so repointing it there would check nothing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…EV-2557)

Review follow-up to 950dcfe. Collapsing the reporter to one physical line fixed
the reported line *number* and broke the code *frame*: babel prints the two lines
above the fault verbatim, so a syntax error on authored line 1 or 2 of the entry
renders all 12.6 KB of REPORTER_MODULE_LINE into the compile message ahead of the
offending line, and MONITOR_COMPILE_MESSAGE_MAX then cuts at 2,000.

Measured on the vue starter's entry with an unterminated string on line 1, via
the real @babel/standalone transform over the real injectReporter output:

  reporter inlined (before 950dcfe):  289 chars, caret and source line intact
  reporter on one line (950dcfe):  12,872 chars, source line first seen at ~12.4k
  after this commit:                  bounded, caret and source line intact

That is DEV-2550's buried-diagnostic failure (DEMOS-15) returning on the same
channel — the error card and Sentry.captureException — sourced from our own bytes
instead of an inline source map.

boundCompileMessage now replaces REPORTER_MODULE_LINE with a short marker, next
to the existing source-map strip and before the cap. Matched against the exported
constant rather than a pattern: a regex over the injected shape would keep passing
while silently ceasing to match a reworded injection, and the burying would come
back with nothing red.

The regression test drives the real babel over the real injected entry rather than
a hand-assembled string, and asserts its own premise (untreated, the message is
over the cap and the visitor's line is past it) so it cannot pass by the blob never
having been there. Verified by mutation: dropping the strip fails it.

Also corrected the acorn ES5 guard's comment. It claimed the symptom of a slip in
REPORTER_SOURCE is a blank Tier-1 preview; since 950dcfe the module path wraps the
reporter in try/catch, so a parse failure there is swallowed and costs monitoring
silently instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@demtario
demtario merged commit 11c656f into master Aug 18, 2026
1 check passed
@demtario
demtario deleted the fix/DEV-2557-reporter-line-offset branch August 18, 2026 06:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant