perf(cpu): raise GDN recurrence lane cap 4 -> 8 - #694
Merged
chenghuaWang merged 1 commit intoAug 9, 2026
Conversation
Aharrypotter
requested review from
chenghuaWang,
oreomaker and
yirongjie
as code owners
August 8, 2026 14:50
Contributor
📝 WalkthroughWalkthroughThe PR expands CPU GDN execution from four to eight lanes, increases the HPC task limit, fixes asynchronous lambda captures, and adds serial-versus-parallel Qwen3.5 validation with repeated thread-pool reuse. ChangesGDN CPU parallelism
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant Qwen35GDNTest
participant HpcThreadPool
participant GDN
Qwen35GDNTest->>GDN: run serial and eight-thread cases
GDN->>HpcThreadPool: schedule disjoint GDN lanes
HpcThreadPool->>GDN: execute captured tasks
GDN-->>Qwen35GDNTest: return outputs and states
Qwen35GDNTest->>Qwen35GDNTest: compare results and repeat 24 passes
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@mllm/engine/HpcThreadPool.hpp`:
- Around line 16-19: Replace the MLLM_HPC_THREAD_POOL_TASK_LIMITS macro in
HpcThreadPool.hpp with a typed C++ constant or enum value initialized to 8, and
update its usage sites as needed so the task limit remains unchanged in all
expression contexts.
In `@tests/cpu/Qwen35GDNTest.cpp`:
- Around line 320-325: Replace the ASSERT_FLOAT_EQ comparisons in the output and
state validation loops, and the corresponding comparison at the later referenced
location, with exact value-equality assertions. Preserve the existing
index-specific failure messages and comparison targets so the test enforces
bitwise-identical serial and parallel results.
- Around line 20-25: Update ScopedCpuOpThreads so thread_count is applied with
Context::setCpuOpThreads() before the one-time initializeContext() call,
ensuring the CPU pool is created with the requested count. In the destructor,
restore the saved original_thread_count_ after the test scope completes.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 8d51cae0-1efd-43f3-8c51-3adc08909a58
📒 Files selected for processing (4)
mllm/backends/cpu/kernels/common/gdn/gated_delta_net.cppmllm/engine/HpcThreadPool.cppmllm/engine/HpcThreadPool.hpptests/cpu/Qwen35GDNTest.cpp
Comment on lines
+16
to
+19
| // One task slot per expected concurrent op. 8 covers the GDN recurrence | ||
| // 8-lane cap plus sibling parallel ops; too small a limit makes splitTask fall | ||
| // back to serial on the main thread under multi-layer concurrency. | ||
| #define MLLM_HPC_THREAD_POOL_TASK_LIMITS 8 |
Contributor
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n -C 2 '\bMLLM_HPC_THREAD_POOL_TASK_LIMITS\b' .Repository: UbiquitousLearning/mllm
Length of output: 2556
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf 'Files mentioning modernize-macro-to-enum:\n'
rg -n -C 3 'modernize-macro-to-enum' . || true
printf '\nHpcThreadPool.hpp context:\n'
sed -n '1,60p' mllm/engine/HpcThreadPool.hpp
printf '\nPotential config files listing compiler/clang checks:\n'
git ls-files | rg -i '(^|/)(clang_tidy|\.clang-tidy|cmake|build|ci|github|flake|pyproject|setup|Makefile|SConstruct|buck|bazel)' | head -n 100
rg -n 'clang-tidy|Modernize|macro-to-enum|warnings-as-errors|CXXFLAGS|CMAKE_CXX_FLAGS' . -g '!build*' -g '!third_party*' -g '!*log*' | head -n 200Repository: UbiquitousLearning/mllm
Length of output: 6188
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '.clang-tidy:\n'
cat -n .clang-tidy
printf '\nIgnore config context:\n'
sed -n '1,120p' .clang-tidy.ignoreRepository: UbiquitousLearning/mllm
Length of output: 2501
Replace the task-slot limit macro with a typed constant.
MLLM_HPC_THREAD_POOL_TASK_LIMITS is only used in C++ expression contexts, so use an enum class or constant instead of the integral #define to satisfy the configured modernize-macro-to-enum check.
🧰 Tools
🪛 Clang (14.0.6)
[error] 19-19: replace macro with enum
(modernize-macro-to-enum,-warnings-as-errors)
[error] 19-19: macro 'MLLM_HPC_THREAD_POOL_TASK_LIMITS' defines an integral constant; prefer an enum instead
(modernize-macro-to-enum,-warnings-as-errors)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@mllm/engine/HpcThreadPool.hpp` around lines 16 - 19, Replace the
MLLM_HPC_THREAD_POOL_TASK_LIMITS macro in HpcThreadPool.hpp with a typed C++
constant or enum value initialized to 8, and update its usage sites as needed so
the task limit remains unchanged in all expression contexts.
Source: Linters/SAST tools
Comment on lines
20
to
25
| explicit ScopedCpuOpThreads(int32_t thread_count) : original_thread_count_(mllm::Context::instance().getCpuOpThreads()) { | ||
| // initializeContext() registers the CPU backend; SymbolTable::reg aborts on | ||
| // a duplicate key, so call it exactly once (the tests have no fixture init). | ||
| static const bool kContextInitialized = [] { mllm::initializeContext(); return true; }(); | ||
| (void)kContextInitialized; | ||
| mllm::Context::instance().setCpuOpThreads(thread_count); |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n -C 8 '\bsetCpuOpThreads\s*\(' mllm tests
rg -n -C 8 '\binitThreadPool\s*\(' mllmRepository: UbiquitousLearning/mllm
Length of output: 7100
🏁 Script executed:
#!/bin/bash
set -euo pipefail
fd -a 'CPUBackend.hpp|CPUBackend.cpp|mllm.hpp|Qwen35GDNTest.cpp' . | sed 's#^\./##' | sort
printf '\n--- mllm.cpp nearby ---\n'
sed -n '1,60p' mllm/mllm.cpp
printf '\n--- CPUBackend.h/cpp pool usage ---\n'
sed -n '1,180p' mllm/backends/cpu/CPUBackend.hpp
sed -n '1,180p' mllm/backends/cpu/CPUBackend.cpp
printf '\n--- locate HpcThreadPool definitions/usages ---\n'
rg -n -C 5 'class HpcThreadPool|struct HpcThreadPool|HpcThreadPool|--set-cpu-op-threads|cpu_op_thread|CpuOpThreads|eight|8 workers|workers' .Repository: UbiquitousLearning/mllm
Length of output: 50380
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
import re
files = {
"CPUBackend.hpp": Path("mllm/backends/cpu/CPUBackend.hpp").read_text(),
"CPUBackend.cpp": Path("mllm/backends/cpu/CPUBackend.cpp").read_text(),
"mllm.hpp": Path("mllm/mllm.hpp").read_text(),
"Context.cpp": Path("mllm/engine/Context.cpp").read_text(),
"Qwen35GDNTest.cpp": Path("tests/cpu/Qwen35GDNTest.cpp").read_text(),
}
for name, text in files.items():
print(f"\n--- {name} relevant lines ---")
for n,line in enumerate(text.splitlines(), 1):
if re.search(r'\b(setCpuOpThreads|getCpuOpThreads|initThreadPool|initializeContext|MLLM_CONDITIONAL_PARALLEL_FOR|scoped_thread_pool|thread_pool_|__threadPoolDestroy)', line):
print(f"{name}:{n}: {line}")
for n,line in enumerate(files["Qwen35GDNTest.cpp"].splitlines(), 1):
if "ScopedCpuOpThreads" in line or "kEightThreads" in line or "kCpuOpThreads" in line or "ASSERT_FLOAT_EQ" in line:
print(f"{name}:{n}: {line}")
PYRepository: UbiquitousLearning/mllm
Length of output: 2950
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- CPU backend thread-count API definitions/usages ---'
rg -n -C 4 'thread_count|thread_cnt|num_threads|cpu_op_thread|CpuOpThreads|ConditionalParallel|conditional|parallel|thread_pool_->|getThreadPool\(\)' mllm/backends mllm/common mllm/engine mllm/mllm.hpp
printf '%s\n' '--- HpcThreadPool definitions/usages ---'
rg -n -C 4 'class HpcThreadPool|struct HpcThreadPool|HpcThreadPool' mllm
printf '%s\n' '--- relevant Qwen35GDNTest setup ---'
sed -n '1,60p' tests/cpu/Qwen35GDNTest.cpp
sed -n '250,330p' tests/cpu/Qwen35GDNTest.cppRepository: UbiquitousLearning/mllm
Length of output: 50380
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- test thread count macros/constants ---'
sed -n '1,40p' tests/cpu/Qwen35GDNTest.cpp
printf '%s\n' '--- MLLM_CONDITIONAL_PARALLEL_FOR definition/usages (top-level files) ---'
rg -n '\bMLLM_(CONDITIONAL_)?PARALLEL_FOR|MLLM_AUTO_PARALLEL_FOR|MLLM_KERNEL_USE_THREADS_VENDOR_MLLM|struct\s+HpcThreadPool|class\s+HpcThreadPool' \
-g '*.{h,hpp,cpp,h.in,cmake,txt,md}' \
mllm tests | head -n 200
printf '%s\n' '--- CPUBackend exact implementation ---'
sed -n '88,102p' mllm/backends/cpu/CPUBackend.cpp
printf '%s\n' '--- Context setCpuOpThreads exact implementation ---'
sed -n '158,164p' mllm/engine/Context.cpp
printf '%s\n' '--- initializeContext exact implementation ---'
sed -n '145,158p' mllm/mllm.hpp
printf '%s\n' '--- Qwen35GDNTest relevant constructor and scoped-threads calls ---'
sed -n '18,28p' tests/cpu/Qwen35GDNTest.cpp
sed -n '256,269p' tests/cpu/Qwen35GDNTest.cpp
sed -n '312,346p' tests/cpu/Qwen35GDNTest.cppRepository: UbiquitousLearning/mllm
Length of output: 22515
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
import re
text = Path("mllm/core/Parallel.hpp").read_text()
needle = "MLLM_KERNEL_USE_THREADS_VENDOR_MLLM"
idx = text.find(needle)
print("MLLM_KERNEL_USE_THREADS_VENDOR_MLLM at", idx)
if idx != -1:
print(text[max(0, idx-400):idx+700])
print("\nmacro branch with MLLM_KERNEL_USE_THREADS_VENDOR_MLLM:")
for i,line in enumerate(text.splitlines(), 1):
if re.search(r'#(?:ifndef|define) ' + re.escape(needle)|MLLM_CONDITIONAL_PARALLEL_FOR|`#define` MLLM_KERNEL_USE_THREADS_VENDOR_MLLM|`#endif`\s*(//)?\s*?{}'.format(re.escape(needle).replace("_", r"\_")), line):
print(f"{i}: {line}")
for i,line in enumerate(Path("tests/cpu/Qwen35GDNTest.cpp").read_text().splitlines(), 1):
if re.match(r'\s*constexpr int kThreadCount\s*=', line) or re.search(r'mllm\.cmake|MLLM_KERNEL_USE_THREADS_VENDOR_MLLM|kThreadCount', line):
print(f"tests/cpu/Qwen35GDNTest.cpp:{i}: {line}")
PY
printf '%s\n' '--- GDN implementation around parallel macro ---'
sed -n '340,380p' mllm/backends/cpu/kernels/common/gdn/gated_delta_net.cpp
printf '%s\n' '--- exact hpc threadpool definition excerpt ---'
sed -n '33,120p' mllm/engine/HpcThreadPool.hppRepository: UbiquitousLearning/mllm
Length of output: 403
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- Parallel.hpp around MLLM_KERNEL_USE_THREADS_VENDOR_MLLM ---'
rg -n -C 20 'MLLM_KERNEL_USE_THREADS_VENDOR_MLLM|MLLM_CONDITIONAL_PARALLEL_FOR' mllm/core/Parallel.hpp
printf '%s\n' '--- GDN parallel loop macro usage ---'
sed -n '340,380p' mllm/backends/cpu/kernels/common/gdn/gated_delta_net.cpp
printf '%s\n' '--- HpcThreadPool definition excerpt ---'
sed -n '20,120p' mllm/engine/HpcThreadPool.hpp
printf '%s\n' '--- Qwen35GDNTest constants and flag references ---'
rg -n -C 6 'kThreadCount|kThread|MLLM_KERNEL_USE_THREADS_VENDOR_MLLM|CPU backend|eight' tests/cpu/Qwen35GDNTest.cppRepository: UbiquitousLearning/mllm
Length of output: 15390
Set the CPU pool count before creating the context.
initializeContext() creates the CPU backend pool from Context::getCpuOpThreads(), and Context::setCpuOpThreads() only updates the scalar. The current code calls initializeContext() first, so this 8-lane path can keep the prior default worker count. Set thread_count before initializing the context; reset the count in the destructor as an additional guard.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/cpu/Qwen35GDNTest.cpp` around lines 20 - 25, Update ScopedCpuOpThreads
so thread_count is applied with Context::setCpuOpThreads() before the one-time
initializeContext() call, ensuring the CPU pool is created with the requested
count. In the destructor, restore the saved original_thread_count_ after the
test scope completes.
…ing-ref fix kMaxParallelGDNLanes 4 -> 8: the GDN recurrence fans out batch x num_value_heads tasks (16 for 0.8B, 32 for 4B); with the old cap of 4, only 4 of 8 CPU cores were used. Bitwise-safe (tasks disjoint). Default-on (parallel path is unconditional in gatedDeltaRuleF32). HpcThreadPool::splitTask captured the moved-in rvalue task and a stack vector BY REFERENCE into a worker lambda; workers run asynchronously after splitTask returns, so the references dangle (use-after-free). Latent at 4 lanes, reachable at 8. Fix: capture task/true_idx by value. Also raise MLLM_HPC_THREAD_POOL_TASK_LIMITS 2 -> 8. The busy-poll worker is preserved (an event-driven rewrite added decode latency that micro-optimizations could not remove). Tests: Qwen35GDNTest 7/7 + GDN-Conv 6/6 (bitwise at threads 1/4/8, 4B-geometry 8-lane crash-guard, 24-layer reuse). On-device (OnePlus 4B): PP69 prefill -2.8%, PP517 +0.3%, decode +/-2.5% noise. Pixel 0.8B -15%.
Aharrypotter
force-pushed
the
perf/gdn-lane8-product
branch
from
August 8, 2026 14:58
9a80c4e to
67f23f9
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
mllm/engine/HpcThreadPool.cpp (1)
103-105: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick winMove-capture the task inputs.
The current captures copy the full
HpcThreadPoolTaskand thetrue_idxvector. Use init-captures withstd::moveto preserve the lifetime fix without an extra O(tiles_num) copy on each split.Proposed change
- [tiles_num, task, true_idx, this](int thread_idx) { + [tiles_num, task = std::move(task), + true_idx = std::move(true_idx), this](int thread_idx) { ... - .func = [tiles_num, task, true_idx, this](int thread_idx) { + .func = [tiles_num, task = std::move(task), + true_idx = std::move(true_idx)](int thread_idx) {As per coding guidelines, avoid unnecessary object creation in loops or hot paths.
Also applies to: 115-115
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@mllm/engine/HpcThreadPool.cpp` around lines 103 - 105, Update the lambda assigned in the task-splitting logic to init-capture task and true_idx by moving them into the closure, while retaining the existing this and tiles_num captures and lifetime behavior. Apply the same move-capture change to the corresponding capture at the other indicated location.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@mllm/engine/HpcThreadPool.cpp`:
- Around line 98-102: Update the comment above the worker lambda in splitTask to
remove the incorrect claim that splitTask’s return causes captured references to
dangle. Explain that value capture provides explicit ownership for asynchronous
worker execution while splitTask waits for callbacks to finish, and retain the
existing value-capture behavior.
---
Nitpick comments:
In `@mllm/engine/HpcThreadPool.cpp`:
- Around line 103-105: Update the lambda assigned in the task-splitting logic to
init-capture task and true_idx by moving them into the closure, while retaining
the existing this and tiles_num captures and lifetime behavior. Apply the same
move-capture change to the corresponding capture at the other indicated
location.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 886e1340-9873-46e0-b03f-c73aca8df82a
📒 Files selected for processing (3)
mllm/engine/HpcThreadPool.cppmllm/engine/HpcThreadPool.hpptests/cpu/Qwen35GDNTest.cpp
🚧 Files skipped from review as they are similar to previous changes (2)
- mllm/engine/HpcThreadPool.hpp
- tests/cpu/Qwen35GDNTest.cpp
Comment on lines
+98
to
+102
| // Capture task and true_idx BY VALUE: splitTask's parameters (a moved-in | ||
| // rvalue task and a stack vector) die when this function returns, but the | ||
| // worker threads execute this lambda asynchronously — capturing by | ||
| // reference leaves a dangling reference (use-after-free). This was latent | ||
| // with 4 lanes and became reachable at 8 lanes. |
Contributor
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Locate file:"
git ls-files | rg '(^|/)HpcThreadPool\.cpp$|(^|/)HpcThreadPool\.(hpp|h)$|(^|/)HpcThreadPool\.(cc|h)$|(^|/)HpcThreadPool\.(hh)$' || true
echo
echo "Outline:"
ast-grep outline mllm/engine/HpcThreadPool.cpp || true
echo
echo "Relevant source lines:"
cat -n mllm/engine/HpcThreadPool.cpp | sed -n '1,180p'Repository: UbiquitousLearning/mllm
Length of output: 7671
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Inspect matching declaration and relevant code:"
cat -n mllm/engine/HpcThreadPool.hpp | sed -n '1,180p'
echo
echo "Find push/splitTask call sites:"
rg -n "splitTask|push\\(" mllm --glob '*.{cpp,hpp,c' ] 2>/dev/null || true
echo
echo "Search task lifecycle outside HpcThreadPool.cpp:"
rg -n "tasks_\\[|HpcThreadPoolTask|func\\(" mllm --glob '*.{cpp,cxx,c,hpp,h,cc,hh}' || trueRepository: UbiquitousLearning/mllm
Length of output: 10765
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Check relevant call-sites:"
cat -n mllm/core/Parallel.hpp | sed -n '1,150p'
echo
echo "Search all splitTask occurrences including tests/docs:"
rg -n "splitTask" . || true
echo
echo "Inspect repository files for splitTask declarations outside tracked text:"
python3 - <<'PY'
from pathlib import Path
p = Path('mllm/engine/HpcThreadPool')
print('exists.cpp=', p.with_suffix('.cpp').exists())
print('exists.hpp=', p.with_suffix('.hpp').exists())
for suffix in ['.cpp', '.hpp', '.h', '.cc', '.hh']:
f = p.with_suffix(suffix)
if f.exists():
text = f.read_text(errors='replace')
print(f'{suffix} len={len(text)} splitTask_count={text.count("splitTask")}')
PYRepository: UbiquitousLearning/mllm
Length of output: 11179
Clarify the lifetime explanation.
splitTask waits until worker callbacks complete before it returns, and each worker flag is cleared after tasks_[i].first.func(thread_idx) returns. This path does not make the captured references dangling. The value capture is still useful for ownership, but update the comment to avoid saying the lifetime problem is caused by the splitTask return path.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@mllm/engine/HpcThreadPool.cpp` around lines 98 - 102, Update the comment
above the worker lambda in splitTask to remove the incorrect claim that
splitTask’s return causes captured references to dangle. Explain that value
capture provides explicit ownership for asynchronous worker execution while
splitTask waits for callbacks to finish, and retain the existing value-capture
behavior.
chenghuaWang
approved these changes
Aug 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Raises the GDN recurrence lane cap from 4 to 8 (
kMaxParallelGDNLanes) so all 8 CPU cores of modern phones participate in the Qwen3.5 GDN prefill hot spot, plus fixes a dangling-reference use-after-free inHpcThreadPool::splitTaskthat the 8-lane verification exposed.The GDN recurrence fans out
batch × num_value_headstasks (16 for 0.8B, 32 for 4B); with the old cap of 4, only 4 of 8 cores were used. The change is bitwise-safe — tasks are disjoint, so lane count never changes the arithmetic — and default-on (the parallel path is unconditional ingatedDeltaRuleF32).Review map
mllm/backends/cpu/kernels/common/gdn/gated_delta_net.cppkMaxParallelGDNLanes4 → 8;parallel_lanes = min(thread_count, task_count, available_cpu_count, 8)bounded; disjoint task partition (unique(batch, value_head)→ unique state/output regions)mllm/engine/HpcThreadPool.{hpp,cpp}splitTasklambdas (no dangling ref); busy-poll worker preserved; task-slot limit 2 → 8tests/cpu/Qwen35GDNTest.cppSuggested review order: lane cap → thread pool → kernel oracle.
Supported contract
sequence_length > 1(parallel recurrence); serial fallback forseq == 1gatedDeltaRuleF32Validation
Qwen35GDNTest7/7 — bitwise output+state at threads 1/4/8, 4B-geometry 8-lane crash-guard, 24-layer reuse (784 ms)arm64-v8a: runner +Mllm-Test-Qwen35-GDN+Mllm-Test-Qwen35-GDN-Convcompilegit diff --checkcleanPerformance evidence
Conclusion: on-device prefill is faster by −2.8% (OnePlus 4B PP69) to −15% (Pixel 0.8B); PP517 is neutral (+0.3%); decode is unchanged within run-to-run noise (±2.5%).
Setup — ABBA order (4-lane, 8-lane, 8-lane, 4-lane) with 5 s cooldown between runs, median reported. Baseline =
origin/main@9a0a21de(4-lane); candidate = this PR (8-lane). Same device, same model artifact, same binary set, measured back-to-back. OnePlus telemetry confirmed stable 2.4 GHz.Qwen3.5-4B — real-model, formal harness (OnePlus 13T):
Qwen3.5-0.8B — real-model prefill (ms):
Why the gain differs by device — the 8-lane cap matters most where the old 4-lane cap left cores idle:
Decode is unaffected — the recurrence is serial at
seq == 1, so the lane cap does not apply; the thread-pool fix preserves the incumbent busy-poll worker, so decode timing is unchanged within noise.Files changed
4 files, +98/−9:
mllm/backends/cpu/kernels/common/gdn/gated_delta_net.cpp(lane cap 4 → 8, +5/−4)mllm/engine/HpcThreadPool.{hpp,cpp}(value-capture dangling-ref fix, task-slot limit 2 → 8)tests/cpu/Qwen35GDNTest.cpp(8-thread bitwise cases, 4B-geometry crash-guard, 24-layer reuse)Known limits
Scope notes
Summary by CodeRabbit
Performance
Reliability
Tests