Skip to content

Fix step-1 NaN: run gemma attention + DeepSeek sparse-indexer logits in fp32#4136

Draft
ecnal-cienet wants to merge 2 commits into
mainfrom
fix/attention-fp32-numerics
Draft

Fix step-1 NaN: run gemma attention + DeepSeek sparse-indexer logits in fp32#4136
ecnal-cienet wants to merge 2 commits into
mainfrom
fix/attention-fp32-numerics

DeepSeek: cast sparse-indexer logits to fp32 to avoid bf16 NaN

ff174b7
Select commit
Loading
Failed to load commit list.
Google CLA / cla/google succeeded Jun 10, 2026 in 7s

✅ All contributors are covered under a CLA with Google

See https://cla.developers.google.com/ for more info about Google's Contributor License Agreement (CLA).

ℹ️ Googlers: Go here to view more details and manage scans for this pull request.

Details

The following contributors were found for this pull request:

ff174b7 Author: @ecnal-cienet <lan*****ng​@cienet.com>

(Only the first commit for a unique contributor is listed.)