Skip to content

Fix codegen assert reloading a spilled signed small cast source#130985

Open
tannergooding wants to merge 2 commits into
dotnet:mainfrom
tannergooding:tannergooding-fix-vartypeisunsigned-codegen-assert
Open

Fix codegen assert reloading a spilled signed small cast source#130985
tannergooding wants to merge 2 commits into
dotnet:mainfrom
tannergooding:tannergooding-fix-vartypeisunsigned-codegen-assert

Conversation

@tannergooding

Copy link
Copy Markdown
Member

Fixes #124508.

A cast whose operand is used from memory (castIsLoad) computed its load type in GenIntCastDesc from src->TypeGet(). That is correct for a genuine small memory operand, but wrong for an LSRA spill temp: a spill temp always holds the register-materialized, already-extended actual-type value, not a small memory operand.

In the repro (x86, FullOpts), a signed sbyte load (s_1[0], materialized via movsx to a full int) feeds a cast to ushort. The (short) in Vector128.CreateScalar((short)(s_1[0] << ...)) is relaxed to ushort during lowering because CreateScalar<short> only consumes the low 16 bits. Under x86 register pressure LSRA spills the sign-extended int to a 4-byte temp. GenIntCastDesc then read srcLoadType = src->TypeGet() = TYP_BYTE (signed, size 1) < ushort (size 2), which:

  • trips assert(varTypeIsUnsigned(srcLoadType) || (genTypeSize(srcLoadType) >= genTypeSize(castType))), and
  • absent asserts, would emit a 1-byte movzx that incorrectly zeroes bits 8-15 -- the bits that actually hold the sign extension. So the assert is guarding a real correctness bug.

This is x86-only: on x64 the value stays in a register, and the register path already movzxes the 16-bit register correctly.

ContainCheckCast already forbids containing a signed small load under a zero-extending cast (varTypeIsUnsigned(castOp) == node->IsZeroExtending()), so the spill temp is the only path that reaches the assert -- making GenIntCastDesc the correct layer for the fix.


The fix reloads a spill temp as its actual type (srcType = genActualType(src) = TYP_INT) rather than as a small memory operand, so m_extendSrcSize becomes min(4, 2) = 2 and codegen emits movzx word from the 4-byte temp (correct low 16 bits). Genuine small memory operands still use src->TypeGet() and are unaffected.

Verification

  • x86 Checked, pre-fix: the repro asserts deterministically.
  • x86 Checked, post-fix: the repro runs clean; JitDump confirms movsx eax, byte ptr[...] -> 4-byte spill -> movzx ecx, word ptr[TEMP].
  • x64 Checked, post-fix: no diff/regression.
  • jitformat.py -r . -o windows -a x64: clean.

Test note

The added Runtime_124508 regression test is the faithful Fuzzlyn repro (seed 12192374036514626124). Note that under the merged test harness the byte-identical IL happens to compile without the spilled-signed-byte-under-ushort-cast shape (a deterministic token-layout-driven codegen divergence from the standalone console repro), so in that specific harness the test passes both with and without the fix. It still provides compile + execution coverage of the shape; the standalone console program is the reliable reproduction. Happy to drop or adjust the test if preferred.

Note

This PR description was drafted by GitHub Copilot.

A cast whose operand is used from memory (`castIsLoad`) computed its load
type from `src->TypeGet()`. For an LSRA spill temp holding a signed small
load (e.g. a sign-extended `sbyte`) that feeds a zero-extending cast to
`ushort`, that reported the small signed type, asserting in `GenIntCastDesc`
and -- absent asserts -- emitting a byte `movzx` that drops the sign bits the
4-byte temp actually holds. Reload a spill temp as its actual type instead.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 17, 2026 18:01
@github-actions github-actions Bot added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Jul 17, 2026
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 5 pipeline(s).
10 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

@tannergooding

Copy link
Copy Markdown
Member Author

CC. @dotnet/jit-contrib, @EgorBo. Small 1 line product change + regression test for #124508

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a CoreCLR JIT codegen issue where casts that reload from memory could treat an LSRA spill temp as a “small” memory operand, leading to incorrect reload sizing (and a checked-build assert) for certain signed-small-source → zero-extending-cast cases.

Changes:

  • Adjust GenIntCastDesc to compute the reload/load type from the operand’s actual type when the operand is used from an LSRA spill temp.
  • Add a new JitBlue regression test project (Runtime_124508) with a Fuzzlyn-derived repro.
Show a summary per file
File Description
src/coreclr/jit/codegenlinear.cpp Use genActualType(src) for srcLoadType when src->isUsedFromSpillTemp() to avoid treating spill temps like small memory operands.
src/tests/JIT/Regression/JitBlue/Runtime_124508/Runtime_124508.csproj Adds a new xUnit-based JitBlue regression test project with process isolation and tiered compilation disabled.
src/tests/JIT/Regression/JitBlue/Runtime_124508/Runtime_124508.cs Adds the Fuzzlyn-derived test case intended to compile the problematic cast/spill shape.

Copilot's findings

  • Files reviewed: 3/3 changed files
  • Comments generated: 1

Comment thread src/tests/JIT/Regression/JitBlue/Runtime_124508/Runtime_124508.csproj Outdated
Comment on lines +2558 to +2562
// A spill temp holds the full actual-type value, already extended per the source's own
// signedness, so it must be reloaded as that actual type rather than as a small memory
// operand -- otherwise a signed small source (e.g. TYP_BYTE) reloaded under a zero-extending
// cast would drop the sign bits it actually holds.
const var_types srcLoadType = src->isUsedFromSpillTemp() ? srcType : src->TypeGet();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps castIsLoad should instead be false for this case? cc @SingleAccretion

Comment thread src/tests/JIT/Regression/JitBlue/Runtime_124508/Runtime_124508.csproj Outdated
Comment thread src/tests/JIT/Regression/JitBlue/Runtime_124508/Runtime_124508.cs Outdated
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 17, 2026 19:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 3/3 changed files
  • Comments generated: 0 new

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

JIT: Assertion failed 'varTypeIsUnsigned(srcLoadType) || (genTypeSize(srcLoadType) >= genTypeSize(castType))' during 'Generate code'

4 participants