Skip to content

perf(dflash): add opt-in exact prefill bands - #590

Draft
cheese-cakee wants to merge 2 commits into
Luce-Org:mainfrom
cheese-cakee:codex/perf-ds4-exact-prefill-aug10-v2
Draft

perf(dflash): add opt-in exact prefill bands#590
cheese-cakee wants to merge 2 commits into
Luce-Org:mainfrom
cheese-cakee:codex/perf-ds4-exact-prefill-aug10-v2

Conversation

@cheese-cakee

@cheese-cakee cheese-cakee commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

What this draft does

This is a hardware-blocked candidate for the August 10 DeepSeek V4 exact-prefill objective. It does not claim the 1.5x target.

  • Adds an opt-in DFLASH_DS4_EXACT_PREFILL_BANDS=1 path for exact prefill widths 2 through 4.
  • Keeps exact attention token-by-token and uses the existing compressor-safe boundary splitter.
  • Skips intermediate terminal HC collapse, output norm, vocabulary projection, and host logits readback only for the new nonfinal exact width-2-through-4 bands.
  • Keeps the final chunk, snapshot boundaries, and required capture paths producing logits.
  • Keeps all existing q=1, dense, and sparse scheduler behavior unchanged.
  • Keeps the default off. --chunk 1 remains the explicit escape hatch.
  • Prevents exact prefill capture hooks from entering the intentionally approximate fused-verification path.

Why it is a draft

Neither lucebox3 nor lucebox5 was available. This branch has no HIP/gfx1151 production-model differential, no issue-#516 cached-graph burn-in, no sanitizer result, no performance profile, and no lucebox5 four-cell benchmark. It is not ready to merge or use for a performance claim.

The target benchmark contract is also incomplete. The default acceptance interpretation remains:

median heterogeneous-candidate throughput / median Strix-candidate throughput >= 1.5

with matched heterogeneous baseline/candidate and Strix baseline/candidate cells.

Local evidence

Base: 4fa8937c69aa68658a14ec3cab16e1da22886dd0

Head: ed8991707d25cfde956e2d599f4570514bc0146c

Passed:

  • Tests were added before implementation; the first compile failed on the deliberately missing policy declarations.
  • The modified backend and unit-test C++ objects compile successfully in WSL Ubuntu with GCC 13.3 and CUDA 12.6 headers, architecture 89.
  • deepseek4_graph.cpp compiled successfully after the graph-hook change.
  • git diff --check, staged diff checks, and git show --check pass.
  • Policy coverage includes default-off behavior, widths 1/2/3/4, cap and fallback behavior, compressor-safe scheduling around ratio-4 boundaries, final/snapshot/execution logits policy, and fused-verify opt-out.

Not passed or not run:

  • The full test_deepseek4_unit target did not finish. Earlier attempts failed while compiling vendored mmvq.cu with WSL service errors; the final single-job attempt timed out after five minutes. This is compile-only evidence, not a unit-test pass.
  • No model run or exact differential exists yet.
  • No HIP/gfx1151 qualification, repeated-request/reset test, snapshot/capture content comparison, sanitizer run, timing, or lucebox5 acceptance exists.

Required next validation

On lucebox3:

  1. Build this head with the compatible, SHA-pinned model/toolchain.
  2. Run the bounded issue-Bug: ggml_set_rows faults on HIP/ROCm after many reuses of a cached single-token graph (Strix Halo / gfx1151) #516 cached set-rows burn-in.
  3. Compare exact width 1 against widths 2, 3, and 4 across prompt tails, compressor and SWA boundaries, first hash-routed and learned-router layers, repeated requests, cache reset, snapshots, and DSpark capture windows.
  4. Compare routing IDs/weights, HC, raw/compressed KV, compressor/indexer state, logits at a declared tolerance, greedy/continuation tokens, capture rows, and final cache position.
  5. Run the applicable sanitizer and record a profile.

Then run the matched four-cell acceptance on lucebox5 with identical model SHAs, inputs, settings, clocks, caches, warmups, and sample counts. Report medians, dispersion, both before/after changes, and the heterogeneous-after / Strix-after ratio.

Integration notes

This touches nearby backend/graph/docs code also changed by #548 and #554. Those changes need a semantic rebase and another review if they land first. Drafts #569/#575/#577 and lossy PFlash #589 are not acceptance evidence for this exact-prefill change.

Review in cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant