fix: keep trailing underscore in expression before close tag#809
Open
spokodev wants to merge 1 commit into
Open
fix: keep trailing underscore in expression before close tag#809spokodev wants to merge 1 commit into
spokodev wants to merge 1 commit into
Conversation
An expression ending in an identifier whose last character is `_` (for example `<%= foo_%>`) was parsed as `foo` plus a `_%>` whitespace-slurping close, so it read the wrong variable and stripped the following whitespace. The spaced form `<%= foo_ %>` already keeps `foo_`, so the no-space form diverged from it. `_%>` is only a close tag when its leading `_` is a delimiter prefix, not when it continues a JS identifier. Guard both places that treat `_%>` as a close (the tokenizer regex and the whitespace pre-pass) with a `(?<![$\w])` lookbehind, so `foo_%>` now matches `foo_ %>` while the documented `<%_ ... _%>` and `-%>` slurp behaviors are unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
An expression whose last token is an identifier ending in
_is truncated when the close tag immediately follows it, because_%>is matched as a substring with no left boundary.The spaced form already behaves correctly, which shows the truncation is unintended:
The slurp rule
_%>is the documented whitespace-slurping close tag ("removes all whitespace after it"). It is only a close tag when its leading_acts as a delimiter prefix. When the_continues a JavaScript identifier (foo_), it belongs to the expression, not to the close tag. The-%>newline-slurp close is not affected by this because-is not a valid identifier character, sofoo-%>is already unambiguous;_is the one slurp-prefix that is also an identifier character.Root cause
Two places treat
_%>as a close with no left boundary:createRegex(...|%>|-%>|_%>). DuringparseTemplateText,pat.execfinds the leftmost match, so forfoo_%>the_%>alternative matches at the_(index 4) and beats%>(index 5). The expression content becomesfooand the close becomes_%>._%>close, which also matchedfoo_%>and stripped the trailing whitespace.Fix
Guard the
_%>close in both places with a(?<![$\w])lookbehind so it is only treated as a close when the_is not preceded by a JS identifier character. The lookbehind is zero-width, soparseTemplateText's index/slice logic is unchanged. IncreateRegexthe guard is added after the delimiter substitution so the</>replacements do not rewrite the lookbehind syntax, which keeps custom delimiters working.Tests
Added to the existing
<%_ and _%>suite intest/ejs.js:<%= foo_%> ENDnow readsfoo_and keeps its whitespace, matching<%= foo_ %> END._%>still slurps:<%_ var x = 1; _%>and<%= 1 _%>are unchanged.Reverting only the source change makes the new truncation test fail (red) while the slurp-intact test stays green. With the fix, the full suite passes (168 tests), including the custom open/close delimiter test.