Add DeepSeek-V4 H20 benchmark data to simulation - #16
Closed
xuqianmamba wants to merge 1 commit into
Closed
Conversation
|
|
Author
|
Closing in favor of updating #15 to follow InferSim existing bench_data CSV conventions. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
MXFP4 MoE in
bench_data/dsv4from the model config
main.pygrouped-layer aggregation and the packaged H20 data
Why
PR #15 adds a standalone DeepSeek-V4 sparse-prefill kernel benchmark, but the
simulator still has no DeepSeek-V4 model path and cannot consume measured V4
component latencies. Treating V4 as generic MLA/MHA would also mix C4 and C128
layers and silently reuse incompatible MFU data.
This change makes V4 simulation table-driven. It sums measured per-layer
C4/C128 attention, mHC and MoE latencies according to the checkpoint's layer
layout, then adds communication separately using InferSim's existing
communication model.
Measurement scope
The committed rows were collected on one node with 8 NVIDIA H20 GPUs using
SGLang 0.5.15:
dsv4, FP8 weights,fp8_e4m3KV, FP32 compression stateflashinfer_mxfp4dc8a15770f7f8647Lookup is intentionally exact. Unsupported fingerprints, TP/world sizes,
backends, dtypes or serving shapes raise
MissingDSV4BenchmarkDatainstead ofextrapolating.
Validation
python3 -B -m unittest discover -s tests -v(4 tests)python3 -m py_compile config/model_config.py kernel_sim/dsv4.py models/deepseek_v4_model.py main.py tests/test_deepseek_v4_simulator.pygit diff --check