Add DeepSeek-V4 H20 prefill kernel benchmark data - #15
Closed
xuqianmamba wants to merge 1 commit into
Closed
Conversation
|
|
xuqianmamba
marked this pull request as ready for review
July 22, 2026 15:06
xuqianmamba
force-pushed
the
agent/dsv4-sparse-prefill-benchmark
branch
from
July 28, 2026 08:26
ae3f1fd to
671aac9
Compare
xuqianmamba
force-pushed
the
agent/dsv4-sparse-prefill-benchmark
branch
from
July 28, 2026 08:41
671aac9 to
80e1c06
Compare
xuqianmamba
force-pushed
the
agent/dsv4-sparse-prefill-benchmark
branch
from
July 28, 2026 08:48
80e1c06 to
32bef49
Compare
Author
|
Superseded by #17. The replacement PR uses the kernel-oriented |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
TP2/DP4 and TP8/DP1.
existing grouped-GEMM data table.
in topology-specific
key_kernels.csvfiles.SGLang instead of reimplementing the kernel.
Data layout
The attention files follow InferSim's existing attention CSV schema exactly:
The topology and kernel variant are encoded in the path:
tp2-dp4/attn-64-64-c4.csvtp2-dp4/attn-64-64-c128.csvtp8-dp1/attn-16-64-c4.csvtp8-dp1/attn-16-64-c128.csvThe two MoE kernel measurements use InferSim's existing grouped-GEMM schema:
up_projis the fused gate/up kernel anddown_projis the down kernel.The measured runtime used
flashinfer_mxfp4(W4A16).The complete profile data is also retained without dropping any metrics:
attention_tp2_dp4_moe_tp8/key_kernels.csv: four TP2/DP4 attention rowscovering C4/C128 at 4K and 8K.
attention_tp8_dp1_moe_tp8/key_kernels.csv: two TP8 attention rows andthe MoE gate/up and down rows at 4K.
Both files retain all 12 source columns: phase, component, workload,
topology, calls, average latency, FLOPs per call, achieved TFLOP/s, MFU,
traffic estimate, effective bandwidth and bandwidth utilization.
All measurements in this PR were collected on NVIDIA H20 GPUs.
Attention FLOPs and MFU
Sparse-attention FLOPs are counted as:
The factor 4 includes QK and PV (2 FLOPs per multiply-accumulate for each).
For attention TP2,
local_heads = 128 / 2 = 64; for TP8,local_heads = 128 / 8 = 16. The other64is the actual DSV4 sparseattention compute width per head, not the wider 512-dimensional projection
workspace.
For example, TP2/DP4 C4 at 4K uses:
MFU is stored as a fraction, matching the existing InferSim CSV files. The
denominator is the measured H20 BF16 GEMM peak of 138.184 TFLOP/s.
Validation
python3 -m py_compile kernel_benchmark/flashmla_dsv4_sparse_prefill.py--helpsmoke testbench_data/dsa/prefill/h800/attn-128-576-2048.csvrows from the collected H20 kernel data are present
git diff --check