fix(glm): align and diagnose cross-framework router parity

[yuhezhang-ai]

Summary

• Match vanilla-HF GLM 4.7 Flash's MLA latent RMSNorm epsilon, float32 gate projection, and float32 selected router weights.
• Keep the full 2,048-token parity forward and metrics, while allowing diagnosed cross-framework comparisons to gate on a prefix from that same forward.
• Add opt-in router diagnostics with concise early-layer large-margin/direction evidence and full final-token KL decomposition; flip counts remain informational.
• Automatically add a non-gating HF standing shape comparison whenever the cross-framework gate is shorter than the full forward; optional onboarding/kernel-upgrade sweeps also report sustained-route-flip amplification.
• Use the shared relaxed profile only for this EP8 model's HF source and HF-export comparisons; AutoModel reload and native resume remain standard.

This PR is stacked on #3620.

Diagnosis

The 2,048-token identical-input probe found no single discontinuous layer: every local block stayed close to HF (cosine at least about 0.99993), while exact router-set agreement was about 98.4-99.7% per routed layer. The production-fix scoped run then measured:

Token group	Tokens	Mean KL	Share of total KL mass
0-5 flipped layers	976	0.0190	2.9%
6-10	572	0.0271	2.4%
11-20	369	0.2085	11.8%
21+	131	4.1249	83.0%

The 70 tokens with no self-route flips had mean KL 0.0101, versus 0.3290 for tokens with one or more flips. This is an empirical floor rather than a pure routing-free control because causal attention can propagate earlier-token route differences.

Route replay controls

All replay controls include the three production precision fixes.

AutoModel route control against HF	Full-sequence mean KL	p95 KL	Cosine
Natural routing	0.3181	0.5963	0.97229
Replay HF expert indices	0.2002	0.2713	0.98649
Replay HF indices and mixture weights	0.1256	0.1027	0.99096

AutoModel replaying its own captured indices and weights was exactly identical (max logit difference and KL both zero, cosine 1). This rules out corruption in the replay/forced-dispatch mechanism for fixed AutoModel targets. Route bifurcation is the dominant tail amplifier, route-weight differences also matter, and a continuous cross-framework arithmetic/hidden-state residual still accumulates across 46 routed layers. The probes found no single broken layer or known semantic mismatch, although they cannot prove that no smaller mismatch remains.

Shape sensitivity and Qwen control

Changing only vanilla HF's total forward length reproduced GLM's failure shape: at 512 tokens, KL(HF@4096[:512] || HF@512) was 0.1498, versus matched-shape HF-to-AutoModel KL 0.1528. Qwen3-30B-A3B showed why route-flip count itself must not gate:

Model at L=512	Tokens with any self-route flip	Tokens flipped in at least 11 layers	Mean self-KL
GLM 4.7 Flash	481 / 512	77	0.1498
Qwen3-30B-A3B	496 / 512	73	0.0024

The models flip near-tied routes at similar rates, but GLM amplifies sustained route differences by roughly three orders of magnitude while Qwen re-converges. The new shape report therefore records both self-flip context and the final-token mean KL of sustained-flip tokens.

Scoped CI

The first exact scoped run (parent 64048635, job 407983121) ran the ordinary 50-step recipe and all robustness phases:

Comparison	Gate mean KL	Gate p95 KL	Gate cosine	Result
HF source to AutoModel	0.013405	0.030912	0.999509	Passes relaxed
AutoModel self-repeat	0	0	1	Exact
AutoModel consolidated reload	0	0	1	Exact; standard
HF export self-repeat	0	0	1	Exact
HF export to AutoModel reload	0.012111	0.030585	0.999911	Passes relaxed

Native DCP resume was exact on all eight ranks: steps 5-7 had zero loss difference, and all parameter, buffer, optimizer, gradient, and post-step digests matched. The job failed only because the two cross-framework comparisons were still configured as standard; this PR now selects relaxed for those two phases only.

• Calibrated exact-job rerun: parent 64054805, job 407998642 — passed

Local validation

• Ruff check on all changed Python files
• 193 passed in focused checkpoint-robustness, config-resolver, and CI-generator tests
• End-to-end GLM recipe resolution verified that its 128-token standing shape probe is derived automatically from the gate length

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[yuhezhang-ai]

/ok to test 50b6edd

[yuhezhang-ai]

/ok to test aca8222