Skip to content

fix(moe): forward permute fusion to HybridEP - #3618

Draft
HuiyingLi wants to merge 1 commit into
mainfrom
huiyingl/fix/hybridep-permute-fusion
Draft

fix(moe): forward permute fusion to HybridEP#3618
HuiyingLi wants to merge 1 commit into
mainfrom
huiyingl/fix/hybridep-permute-fusion

Conversation

@HuiyingLi

Copy link
Copy Markdown
Contributor

What does this PR do ?

Forward the existing moe_permute_fusion setting through AutoModel's HybridEP
dispatch/combine autograd wrappers to DeepEP's fuse_permute_dispatch and
fuse_unpermute_combine flags.

Without this propagation, enabling moe_permute_fusion changes the local
permute/unpermute path but silently leaves the HybridEP DeepEP path unfused.

Changelog

  • Pass permute_fusion from _HybridEPManager into HybridEP dispatch and
    combine.
  • Preserve the setting on the autograd context and use the matching DeepEP
    fusion flag in backward.
  • Add unit coverage for all four DeepEP dispatch/combine forward/backward calls
    and for manager-level knob propagation.

Before your PR is "Ready for review"

Pre checks:

  • The change is focused on the existing HybridEP permute-fusion knob.
  • Unit tests cover forward and backward propagation.
  • No documentation change is required; this fixes an existing setting.
  • The commit includes a DCO sign-off.

Validation

  • ruff format --check on all four changed files: passed.
  • ruff check on the changed source files: passed.
  • pytest -q tests/unit_tests/moe/test_fused_a2a.py tests/unit_tests/moe/test_token_dispatcher.py: 13 passed.
  • H100 qualification with DeepEP 421443037f64e9d422814bec9341c0559749c215:
    • 1 node / 8 GPUs: aggregate valid TGS/GPU improved from 6550.50 to
      6936.18 (+5.89%).
    • 16 nodes / 128 GPUs: aggregate valid TGS/GPU improved from 7453.16 to
      7904.66 (+6.06%); all 40/40 steps completed with finite loss and gradient
      norm.

Additional Information

Current main (bb0fe53a) still omits this propagation and pins the same
DeepEP revision used for the H100 qualification. The relevant HybridEP code
path has no behavioral change between the qualification baseline and current
main.

Signed-off-by: HuiyingLi <willwin.lee@gmail.com>
@copy-pr-bot

copy-pr-bot Bot commented Aug 21, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant