Skip to content

Add DeepSeek-V4 H20 prefill kernel benchmark data - #15

Closed
xuqianmamba wants to merge 1 commit into
alibaba:mainfrom
xuqianmamba:agent/dsv4-sparse-prefill-benchmark
Closed

Add DeepSeek-V4 H20 prefill kernel benchmark data#15
xuqianmamba wants to merge 1 commit into
alibaba:mainfrom
xuqianmamba:agent/dsv4-sparse-prefill-benchmark

Conversation

@xuqianmamba

@xuqianmamba xuqianmamba commented Jul 22, 2026

Copy link
Copy Markdown

Summary

  • Add H20 DeepSeek-V4 sparse-prefill attention measurements for attention
    TP2/DP4 and TP8/DP1.
  • Add the H20 TP8 DeepSeek-V4 MoE gate/up and down measurements to the
    existing grouped-GEMM data table.
  • Preserve every field and every row from the collected H20 kernel profile
    in topology-specific key_kernels.csv files.
  • Add a small benchmark that calls the same sparse FlashMLA API used by
    SGLang instead of reimplementing the kernel.

Data layout

The attention files follow InferSim's existing attention CSV schema exactly:

dtype,s_q,s_kv,latency_us,mfu

The topology and kernel variant are encoded in the path:

  • tp2-dp4/attn-64-64-c4.csv
  • tp2-dp4/attn-64-64-c128.csv
  • tp8-dp1/attn-16-64-c4.csv
  • tp8-dp1/attn-16-64-c128.csv

The two MoE kernel measurements use InferSim's existing grouped-GEMM schema:
up_proj is the fused gate/up kernel and down_proj is the down kernel.
The measured runtime used flashinfer_mxfp4 (W4A16).

The complete profile data is also retained without dropping any metrics:

  • attention_tp2_dp4_moe_tp8/key_kernels.csv: four TP2/DP4 attention rows
    covering C4/C128 at 4K and 8K.
  • attention_tp8_dp1_moe_tp8/key_kernels.csv: two TP8 attention rows and
    the MoE gate/up and down rows at 4K.

Both files retain all 12 source columns: phase, component, workload,
topology, calls, average latency, FLOPs per call, achieved TFLOP/s, MFU,
traffic estimate, effective bandwidth and bandwidth utilization.

All measurements in this PR were collected on NVIDIA H20 GPUs.

Attention FLOPs and MFU

Sparse-attention FLOPs are counted as:

FLOPs = 4 * s_q * local_heads * 64 * effective_keys

The factor 4 includes QK and PV (2 FLOPs per multiply-accumulate for each).
For attention TP2, local_heads = 128 / 2 = 64; for TP8,
local_heads = 128 / 8 = 16. The other 64 is the actual DSV4 sparse
attention compute width per head, not the wider 512-dimensional projection
workspace.

For example, TP2/DP4 C4 at 4K uses:

4 * 4096 * 64 * 64 * 5120 = 343.597 GFLOPs
343.597 GFLOPs / 2914.016 us = 117.912 TFLOP/s
MFU = 117.912 / 138.184 = 0.8533

MFU is stored as a fraction, matching the existing InferSim CSV files. The
denominator is the measured H20 BF16 GEMM peak of 138.184 TFLOP/s.

Validation

  • python3 -m py_compile kernel_benchmark/flashmla_dsv4_sparse_prefill.py
  • benchmark --help smoke test
  • exact CSV-header validation against
    bench_data/dsa/prefill/h800/attn-128-576-2048.csv
  • exact row-by-row comparison confirming that all 12 columns and all eight
    rows from the collected H20 kernel data are present
  • git diff --check

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@xuqianmamba
xuqianmamba marked this pull request as ready for review July 22, 2026 15:06
@xuqianmamba
xuqianmamba force-pushed the agent/dsv4-sparse-prefill-benchmark branch from ae3f1fd to 671aac9 Compare July 28, 2026 08:26
@xuqianmamba xuqianmamba changed the title Add DeepSeek-V4 sparse prefill benchmark Add DeepSeek-V4 sparse prefill benchmark data Jul 28, 2026
@xuqianmamba
xuqianmamba force-pushed the agent/dsv4-sparse-prefill-benchmark branch from 671aac9 to 80e1c06 Compare July 28, 2026 08:41
@xuqianmamba xuqianmamba changed the title Add DeepSeek-V4 sparse prefill benchmark data Add DeepSeek-V4 prefill kernel benchmark data Jul 28, 2026
@xuqianmamba
xuqianmamba force-pushed the agent/dsv4-sparse-prefill-benchmark branch from 80e1c06 to 32bef49 Compare July 28, 2026 08:48
@xuqianmamba xuqianmamba changed the title Add DeepSeek-V4 prefill kernel benchmark data Add DeepSeek-V4 H20 prefill kernel benchmark data Jul 28, 2026
@xuqianmamba

Copy link
Copy Markdown
Author

Superseded by #17. The replacement PR uses the kernel-oriented bench_data/dsa layout and encodes TP/DP through local head counts instead of topology subdirectories.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants