feat(a2a3 scheduler): MIX split placement with doorbell-enforced simultaneity#1272
feat(a2a3 scheduler): MIX split placement with doorbell-enforced simultaneity#1272ChaoWao wants to merge 3 commits into
Conversation
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughAdds a ChangesSplit-Placement Doorbell Gate
Estimated code review effort: 4 (Complex) | ~60 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a split-placement doorbell gate to support mixed running and pending subtask placements within a single task block, preventing stranded cores while upholding simultaneity contracts. The review feedback highlights critical concurrency and deadlock issues: it recommends upgrading memory ordering to seq_cst for spec_state and staged_core_mask to avoid Dekker's-style reordering bugs, and suggests centralizing the promoted_subtasks increment logic inside prepare_block_for_dispatch to prevent deadlocks during partial staging.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| inline bool maybe_ring_split_doorbell(PTO2TaskSlotState &slot_state, SpecReleaseSink *sink = nullptr) { | ||
| if (slot_state.payload->spec_state.load(std::memory_order_acquire) != PTO2_SPEC_STAGING) return false; |
There was a problem hiding this comment.
Change the memory order from std::memory_order_acquire to std::memory_order_seq_cst when loading spec_state in maybe_ring_split_doorbell. This is required to maintain Sequential Consistency with the staged_core_mask stores, preventing Dekker's-style reordering bugs where some cores are never rung.
inline bool maybe_ring_split_doorbell(PTO2TaskSlotState &slot_state, SpecReleaseSink *sink = nullptr) {
if (slot_state.payload->spec_state.load(std::memory_order_seq_cst) != PTO2_SPEC_STAGING) return false;| for (int w = 0; w < PTO2_SPEC_CORE_MASK_WORDS; w++) | ||
| if (my_cores[w] != 0) c->payload->staged_core_mask[w].fetch_or(my_cores[w], std::memory_order_seq_cst); | ||
|
|
||
| // If release already flipped DISPATCHED, it may have read the mask before our | ||
| // bits landed — ring our own cores so none is left gated forever. | ||
| if (staged > 0 && c->payload->spec_state.load(std::memory_order_seq_cst) == PTO2_SPEC_DISPATCHED) { | ||
| if (my_cores[w] != 0) c->payload->staged_core_mask[w].fetch_or(my_cores[w], std::memory_order_release); | ||
| // Running-direct subtasks are now in their running slots, gated-spinning — count | ||
| // them toward the doorbell gate; pending subtasks count at their promote (Case 3.3). | ||
| if (running_direct > 0) | ||
| c->payload->promoted_subtasks.fetch_add(running_direct, std::memory_order_acq_rel); | ||
| // Try the gate (rings if fully staged + fanin done + all promoted). If a peer | ||
| // already flipped DISPATCHED before our mask OR landed, self-ring our cores. | ||
| if (!sched_->maybe_ring_split_doorbell(*c) && | ||
| c->payload->spec_state.load(std::memory_order_acquire) == PTO2_SPEC_DISPATCHED) { |
There was a problem hiding this comment.
Restore std::memory_order_seq_cst for both the staged_core_mask fetch_or and the spec_state load. Using release and acquire here violates Sequential Consistency and allows the compiler or hardware to reorder the load of spec_state before the store to staged_core_mask. This can result in a race where the releasing thread misses the newly staged cores and the staging thread fails to self-ring them, leading to a permanent deadlock.
for (int w = 0; w < PTO2_SPEC_CORE_MASK_WORDS; w++)
if (my_cores[w] != 0) c->payload->staged_core_mask[w].fetch_or(my_cores[w], std::memory_order_seq_cst);
// Running-direct subtasks are now in their running slots, gated-spinning — count
// them toward the doorbell gate; pending subtasks count at their promote (Case 3.3).
if (running_direct > 0)
c->payload->promoted_subtasks.fetch_add(running_direct, std::memory_order_acq_rel);
// Try the gate (rings if fully staged + fanin done + all promoted). If a peer
// already flipped DISPATCHED before our mask OR landed, self-ring our cores.
if (!sched_->maybe_ring_split_doorbell(*c) &&
c->payload->spec_state.load(std::memory_order_seq_cst) == PTO2_SPEC_DISPATCHED) {| if (shape == PTO2ResourceShape::MIX) { | ||
| // pending_mask: per-subtask PENDING decision (set = pending slot, clear = running). | ||
| uint8_t cmask = slot_state.active_mask.core_mask(); | ||
| int n = 0; | ||
| if (cmask & PTO2_SUBTASK_MASK_AIC) { | ||
| out_handles[n++] = prepare_subtask_to_core( | ||
| thread_idx, tracker.get_aic_core_offset(core_offset), slot_state, PTO2SubtaskSlot::AIC, to_pending, | ||
| block_idx | ||
| thread_idx, tracker.get_aic_core_offset(core_offset), slot_state, PTO2SubtaskSlot::AIC, | ||
| (pending_mask & PTO2_SUBTASK_MASK_AIC) != 0, block_idx | ||
| ); | ||
| } | ||
| if (cmask & PTO2_SUBTASK_MASK_AIV0) { | ||
| out_handles[n++] = prepare_subtask_to_core( | ||
| thread_idx, tracker.get_aiv0_core_offset(core_offset), slot_state, PTO2SubtaskSlot::AIV0, to_pending, | ||
| block_idx | ||
| thread_idx, tracker.get_aiv0_core_offset(core_offset), slot_state, PTO2SubtaskSlot::AIV0, | ||
| (pending_mask & PTO2_SUBTASK_MASK_AIV0) != 0, block_idx | ||
| ); | ||
| } | ||
| if (cmask & PTO2_SUBTASK_MASK_AIV1) { | ||
| out_handles[n++] = prepare_subtask_to_core( | ||
| thread_idx, tracker.get_aiv1_core_offset(core_offset), slot_state, PTO2SubtaskSlot::AIV1, to_pending, | ||
| block_idx | ||
| thread_idx, tracker.get_aiv1_core_offset(core_offset), slot_state, PTO2SubtaskSlot::AIV1, | ||
| (pending_mask & PTO2_SUBTASK_MASK_AIV1) != 0, block_idx | ||
| ); | ||
| } | ||
| #if PTO2_PROFILING | ||
| sched_l2_swimlane_[thread_idx].phase_dispatch_count += __builtin_popcount(cmask); | ||
| #endif | ||
| return n; | ||
| } else if (shape == PTO2ResourceShape::AIC) { | ||
| out_handles[0] = | ||
| prepare_subtask_to_core(thread_idx, core_offset, slot_state, PTO2SubtaskSlot::AIC, to_pending, block_idx); | ||
| out_handles[0] = prepare_subtask_to_core( | ||
| thread_idx, core_offset, slot_state, PTO2SubtaskSlot::AIC, (pending_mask & PTO2_SUBTASK_MASK_AIC) != 0, | ||
| block_idx | ||
| ); | ||
| #if PTO2_PROFILING | ||
| sched_l2_swimlane_[thread_idx].phase_dispatch_count += 1; | ||
| #endif | ||
| return 1; | ||
| } else { | ||
| out_handles[0] = | ||
| prepare_subtask_to_core(thread_idx, core_offset, slot_state, PTO2SubtaskSlot::AIV0, to_pending, block_idx); | ||
| out_handles[0] = prepare_subtask_to_core( | ||
| thread_idx, core_offset, slot_state, PTO2SubtaskSlot::AIV0, (pending_mask & PTO2_SUBTASK_MASK_AIV0) != 0, | ||
| block_idx | ||
| ); | ||
| #if PTO2_PROFILING | ||
| sched_l2_swimlane_[thread_idx].phase_dispatch_count += 1; | ||
| #endif |
There was a problem hiding this comment.
Centralize the promoted_subtasks increment inside prepare_block_for_dispatch for running-direct subtasks. If a speculative task is partially staged, the remaining blocks are dispatched via the normal ready queue. However, the normal dispatch loop (dispatch_shape) does not increment promoted_subtasks for blocks placed in running slots. This causes promoted_subtasks to never reach total_required_subtasks, permanently deadlocking the task. Centralizing this logic ensures every subtask increments promoted_subtasks exactly once when entering a running slot.
bool is_staging = slot_state.payload != nullptr &&
slot_state.payload->spec_state.load(std::memory_order_acquire) == PTO2_SPEC_STAGING;
if (shape == PTO2ResourceShape::MIX) {
// pending_mask: per-subtask PENDING decision (set = pending slot, clear = running).
uint8_t cmask = slot_state.active_mask.core_mask();
int n = 0;
if (cmask & PTO2_SUBTASK_MASK_AIC) {
bool to_pending = (pending_mask & PTO2_SUBTASK_MASK_AIC) != 0;
out_handles[n++] = prepare_subtask_to_core(
thread_idx, tracker.get_aic_core_offset(core_offset), slot_state, PTO2SubtaskSlot::AIC,
to_pending, block_idx
);
if (is_staging && !to_pending) {
slot_state.payload->promoted_subtasks.fetch_add(1, std::memory_order_acq_rel);
}
}
if (cmask & PTO2_SUBTASK_MASK_AIV0) {
bool to_pending = (pending_mask & PTO2_SUBTASK_MASK_AIV0) != 0;
out_handles[n++] = prepare_subtask_to_core(
thread_idx, tracker.get_aiv0_core_offset(core_offset), slot_state, PTO2SubtaskSlot::AIV0,
to_pending, block_idx
);
if (is_staging && !to_pending) {
slot_state.payload->promoted_subtasks.fetch_add(1, std::memory_order_acq_rel);
}
}
if (cmask & PTO2_SUBTASK_MASK_AIV1) {
bool to_pending = (pending_mask & PTO2_SUBTASK_MASK_AIV1) != 0;
out_handles[n++] = prepare_subtask_to_core(
thread_idx, tracker.get_aiv1_core_offset(core_offset), slot_state, PTO2SubtaskSlot::AIV1,
to_pending, block_idx
);
if (is_staging && !to_pending) {
slot_state.payload->promoted_subtasks.fetch_add(1, std::memory_order_acq_rel);
}
}
#if PTO2_PROFILING
sched_l2_swimlane_[thread_idx].phase_dispatch_count += __builtin_popcount(cmask);
#endif
return n;
} else if (shape == PTO2ResourceShape::AIC) {
bool to_pending = (pending_mask & PTO2_SUBTASK_MASK_AIC) != 0;
out_handles[0] = prepare_subtask_to_core(
thread_idx, core_offset, slot_state, PTO2SubtaskSlot::AIC, to_pending,
block_idx
);
if (is_staging && !to_pending) {
slot_state.payload->promoted_subtasks.fetch_add(1, std::memory_order_acq_rel);
}
#if PTO2_PROFILING
sched_l2_swimlane_[thread_idx].phase_dispatch_count += 1;
#endif
return 1;
} else {
bool to_pending = (pending_mask & PTO2_SUBTASK_MASK_AIV0) != 0;
out_handles[0] = prepare_subtask_to_core(
thread_idx, core_offset, slot_state, PTO2SubtaskSlot::AIV0, to_pending,
block_idx
);
if (is_staging && !to_pending) {
slot_state.payload->promoted_subtasks.fetch_add(1, std::memory_order_acq_rel);
}
#if PTO2_PROFILING
sched_l2_swimlane_[thread_idx].phase_dispatch_count += 1;
#endif| // Running-direct subtasks are now in their running slots, gated-spinning — count | ||
| // them toward the doorbell gate; pending subtasks count at their promote (Case 3.3). | ||
| if (running_direct > 0) | ||
| c->payload->promoted_subtasks.fetch_add(running_direct, std::memory_order_acq_rel); |
There was a problem hiding this comment.
Since the promoted_subtasks increment for running-direct subtasks is now handled centrally inside prepare_block_for_dispatch, we can remove the manual running_direct increment logic here to simplify the code and avoid double-counting.
// Running-direct subtasks are now counted centrally inside prepare_block_for_dispatch.
ChaoWao
force-pushed
the
worktree-iterative-humming-papert
branch
2 times, most recently
from
e64dc0b to
5d1b304
Compare
…ltaneity
Allow MIX (cube+AIV) blocks to stage their AIC/AIV subtasks on whatever core
slot is free (running if idle, pending if busy+pending-free), independently.
The in-kernel simultaneity contract (FFTS syncall + cross-core pipe) is upheld
by gating the whole task behind the speculative doorbell until every subtask
has promoted to a running slot, so all subtasks activate at the same instant.
Changes:
- classify_mix_cluster: new SPLIT mode + MixPlacementPlan{mode, pending_mask}
with deadlock guard (pending subtask must NOT land behind a gated-running core)
- prepare_block_for_dispatch: per-subtask pending_mask (was single bool)
- stage_consumer_blocks: sources split clusters, sets requires_split_gate,
counts promoted_subtasks, calls maybe_ring_split_doorbell
- maybe_ring_split_doorbell: rings only when fanin complete + fully staged
(next_block_idx >= logical_block_num) + all subtasks promoted (promoted ==
total_required_subtasks). The full-staging check prevents partial-ring
deadlocks on full-occupancy syncall tasks (e.g. fa_fused).
- try_speculative_release: split tasks (requires_split_gate) defer ring via
maybe_ring; non-split tasks keep immediate ring at fanin (sticky token)
- Case 3.3 promote path: increments promoted_subtasks + triggers maybe_ring
- promoted_subtasks + requires_split_gate fields on PTO2TaskPayload
Co-Authored-By: Claude <noreply@anthropic.com>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/pto_scheduler.h (1)
1086-1115: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winWiden or bound the doorbell counter.
promoted_subtasksandtotal_required_subtasksareint16_t, buttotal_required_subtasksis computed asblock_num * popcount(active_mask)with only ablock_num >= 1check. Large launches can wrap the threshold and leave the gate closed forever; widen these counters or add an upper-bound assert before submit.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/pto_scheduler.h` around lines 1086 - 1115, The doorbell gate in maybe_ring_split_doorbell is using int16_t counters for promoted_subtasks and total_required_subtasks, but total_required_subtasks can overflow when computed from block_num and popcount(active_mask). Fix this by either widening both counters to a larger integer type or adding a hard upper-bound check before submission so the threshold cannot wrap. Use the symbols maybe_ring_split_doorbell, promoted_subtasks, total_required_subtasks, and the block_num/active_mask-based computation to locate the change.
🧹 Nitpick comments (1)
src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_types.h (1)
324-328: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winNo direct unit-test coverage for the
blocked_pendingdeadlock guard.
classify_mix_cluster'sblocked_pendingparameter is the core new deadlock-avoidance mechanism described in the PR objective, but none of the added/updated tests intest_scheduler_state.cppexercise a non-defaultblocked_pendingvalue (e.g., a core that would otherwise classify as pending-eligible but is excluded because its running task is gated). Consider adding a test that sets a bit inblocked_pendingfor a busy-but-otherwise-pending-free core and asserts the cluster falls back to REJECT/SPLIT-without-that-subtask instead of PENDING/SPLIT.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_types.h` around lines 324 - 328, Add direct test coverage for the new blocked_pending deadlock guard in classify_mix_cluster. Update test_scheduler_state.cpp to call classify_mix_cluster with a non-default blocked_pending bit set on a core that would otherwise be pending-eligible, and assert the result no longer treats that subtask as pending-eligible (for example, it falls back to REJECT or a split that excludes that subtask). Use the existing classify_mix_cluster and blocked_pending symbols so the test covers the gated-running-task case described in the scheduler_types.h contract.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/pto_scheduler.h`:
- Around line 1117-1139: The early-dispatch overflow path in
try_speculative_release leaves PTO2_SPEC_STAGING tasks stranded because the
failed push only logs and does not restore a normal dispatch path. Update the
overflow handling in this flow so that when the early-dispatch queue cannot
accept the task, spec_state is reset from PTO2_SPEC_STAGING to a non-staging
state and the task is requeued through the normal ready path or otherwise safely
retried. Keep the fix localized around try_speculative_release and the
staging/doorbell handoff so staged tasks never lose their only route to
dispatch.
In
`@src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_dispatch.cpp`:
- Around line 593-602: The split-cluster dispatch loop in scheduler_dispatch.cpp
is dropping work when re-classification no longer returns SPLIT after
split.pop_first() consumes an offset. Update the block handling around
classify_mix_cluster and prepare_block_for_dispatch so a failed re-check does
not silently skip the block; instead, reinsert the popped cluster offset or
otherwise requeue the shortfall before continuing. Use the surrounding split,
block, count, staged, and prepare_block_for_dispatch logic to keep the current
claim consistent.
- Around line 661-679: The MIX pending path is missing the same deadlock guard
used for split clusters, so a STAGING-gated running core can still be treated as
pending-eligible. Update `stage_consumer_blocks()` and/or
`prepare_block_for_dispatch()` so the `pend` path also applies `blocked_pending`
when using `get_pending_core_offset_states(PTO2ResourceShape::MIX)`, and make
sure the MIX pending selection excludes cores whose
`CoreExecState::running_slot_state->payload->spec_state` is `PTO2_SPEC_STAGING`.
Use the existing `blocked_pending` logic from `scheduler_dispatch.cpp` as the
reference point and keep the whole-cluster PENDING dispatch from selecting a
fully-running MIX cluster when a gated core is present.
---
Outside diff comments:
In `@src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/pto_scheduler.h`:
- Around line 1086-1115: The doorbell gate in maybe_ring_split_doorbell is using
int16_t counters for promoted_subtasks and total_required_subtasks, but
total_required_subtasks can overflow when computed from block_num and
popcount(active_mask). Fix this by either widening both counters to a larger
integer type or adding a hard upper-bound check before submission so the
threshold cannot wrap. Use the symbols maybe_ring_split_doorbell,
promoted_subtasks, total_required_subtasks, and the block_num/active_mask-based
computation to locate the change.
---
Nitpick comments:
In
`@src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_types.h`:
- Around line 324-328: Add direct test coverage for the new blocked_pending
deadlock guard in classify_mix_cluster. Update test_scheduler_state.cpp to call
classify_mix_cluster with a non-default blocked_pending bit set on a core that
would otherwise be pending-eligible, and assert the result no longer treats that
subtask as pending-eligible (for example, it falls back to REJECT or a split
that excludes that subtask). Use the existing classify_mix_cluster and
blocked_pending symbols so the test covers the gated-running-task case described
in the scheduler_types.h contract.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 3ce28feb-9467-469d-b4db-28d72b8844e9
📒 Files selected for processing (7)
src/a2a3/runtime/tensormap_and_ringbuffer/runtime/pto_runtime2_types.hsrc/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/pto_scheduler.hsrc/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_completion.cppsrc/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_context.hsrc/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_dispatch.cppsrc/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_types.htests/ut/cpp/a2a3/test_scheduler_state.cpp
| inline bool try_speculative_release(PTO2TaskSlotState &slot_state, SpecReleaseSink *sink = nullptr) { | ||
| // Never staged => CAS NONE->DISPATCHED wins => dispatch normally. | ||
| uint8_t expect = PTO2_SPEC_NONE; | ||
| if (slot_state.payload->spec_state.compare_exchange_strong( | ||
| expect, PTO2_SPEC_DISPATCHED, std::memory_order_seq_cst, std::memory_order_seq_cst | ||
| )) { | ||
| return false; | ||
| } | ||
| // Staged (STAGING). A STAGING task is exclusively owned by the early- | ||
| // dispatch drain — it must NEVER fall back to normal dispatch (which would | ||
| // build_payload with not_ready=1 but skip staged_core_mask / promoted_subtasks | ||
| // tracking, leaving those blocks invisible to the gate → deadlocked spin). | ||
| // | ||
| // maybe_ring fires now if all conditions are met (fully staged + fanin + | ||
| // promoted==total). If not yet fully staged, it leaves STAGING; the drain | ||
| // continues staging remaining blocks and re-checks via maybe_ring after each | ||
| // batch + on each pending subtask promote (Case 3.3). | ||
| // | ||
| // Return true unconditionally so release_fanin_and_check_ready does NOT push | ||
| // the task to the ready queue — the early-dispatch drain owns its lifecycle. | ||
| maybe_ring_split_doorbell(slot_state, sink); | ||
| return true; | ||
| } |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Find early_dispatch_queue.push overflow handling and any spec_state reset to NONE
rg -nP 'early_dispatch_queue\.push' src/a2a3/runtime/tensormap_and_ringbuffer -C4
rg -nP 'spec_state\.(store|compare_exchange).*(NONE|PTO2_SPEC_NONE)' src/a2a3/runtime/tensormap_and_ringbuffer -C2Repository: hw-native-sys/simpler
Length of output: 4529
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the relevant scheduler code paths and the queue implementation/usage.
sed -n '980,1085p' src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/pto_scheduler.h
sed -n '1090,1145p' src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/pto_scheduler.h
sed -n '650,715p' src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_dispatch.cpp
# Find the early_dispatch_queue type/definition and any overflow/drop behavior.
rg -n "early_dispatch_queue|queue full|drop \(no re-push\)|spec_state|PTO2_SPEC_STAGING|PTO2_SPEC_NONE" \
src/a2a3/runtime/tensormap_and_ringbuffer/runtime -C 3
# Inspect the runtime type initialization where spec_state is reset.
sed -n '330,365p' src/a2a3/runtime/tensormap_and_ringbuffer/runtime/pto_runtime2_types.hRepository: hw-native-sys/simpler
Length of output: 50377
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Look for any runtime transition from STAGING back to NONE or any queue-drop handler.
rg -n "PTO2_SPEC_STAGING|PTO2_SPEC_NONE|spec_state\.store|compare_exchange_strong|compare_exchange_weak|queue full|overflow|drop" \
src/a2a3/runtime/tensormap_and_ringbuffer/runtime -C 2
# Inspect the ready-queue implementation used by early_dispatch_queue.
fd -a "ready_queue" src/a2a3/runtime/tensormap_and_ringbuffer/runtime
rg -n "class PTO2ReadyQueue|struct PTO2ReadyQueue|push\(|pop\(|overflow|full|drop" \
src/a2a3/runtime/tensormap_and_ringbuffer/runtime -C 2Repository: hw-native-sys/simpler
Length of output: 50378
Reset staged tasks on early-dispatch queue overflow The overflow path only logs the failed push; it doesn’t clear spec_state or requeue the task. A PTO2_SPEC_STAGING consumer can then be left with no route back to normal dispatch and strand its gated cores.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/pto_scheduler.h`
around lines 1117 - 1139, The early-dispatch overflow path in
try_speculative_release leaves PTO2_SPEC_STAGING tasks stranded because the
failed push only logs and does not restore a normal dispatch path. Update the
overflow handling in this flow so that when the early-dispatch queue cannot
accept the task, spec_state is reset from PTO2_SPEC_STAGING to a non-staging
state and the task is requeued through the normal ready path or otherwise safely
retried. Keep the fix localized around try_speculative_release and the
staging/doorbell handoff so staged tasks never lose their only route to
dispatch.
| // Split clusters (per-subtask placement: idle cores → running, busy → pending). | ||
| // Re-classify under the deadlock guard; skip if no longer SPLIT (state raced). | ||
| while (count > 0 && split.has_value()) { | ||
| int32_t cluster_offset = split.pop_first(); | ||
| CoreTracker::MixPlacementPlan plan = tracker.classify_mix_cluster(cluster_offset, cmask, blocked_pending); | ||
| if (plan.mode != CoreTracker::MixPlacement::SPLIT) continue; | ||
| n += prepare_block_for_dispatch(thread_idx, cluster_offset, *c, shape, plan.pending_mask, block, &handles[n]); | ||
| running_direct += __builtin_popcount(cmask & static_cast<uint8_t>(~plan.pending_mask)); | ||
| block++; count--; staged++; | ||
| } |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Inspect how staged/claimed mismatches are handled elsewhere, and whether stage_consumer_blocks
# callers ever reconcile a return value < claim.
rg -n -B3 -A10 'stage_consumer_blocks' src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_dispatch.cpp
rg -n -B3 -A15 'int32_t SchedulerContext::try_speculative_early_dispatch' src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_dispatch.cppRepository: hw-native-sys/simpler
Length of output: 3107
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Read the relevant function body with line numbers.
sed -n '565,715p' src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_dispatch.cpp | cat -n
# Find the classification helper and the MixPlacement definitions.
rg -n -A20 -B10 'classify_mix_cluster|struct MixPlacementPlan|enum class MixPlacement|MixPlacement::SPLIT' src/a2a3/runtime/tensormap_and_ringbuffer/runtime/schedulerRepository: hw-native-sys/simpler
Length of output: 34946
Requeue the block when SPLIT re-classification fails
split.pop_first() consumes the cluster before the re-check. If classify_mix_cluster(...).mode no longer returns SPLIT, the continue drops that block from the current claim without advancing block/count, and there’s no retry path here. Reinsert the offset or requeue the shortfall instead of silently skipping it.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_dispatch.cpp`
around lines 593 - 602, The split-cluster dispatch loop in
scheduler_dispatch.cpp is dropping work when re-classification no longer returns
SPLIT after split.pop_first() consumes an offset. Update the block handling
around classify_mix_cluster and prepare_block_for_dispatch so a failed re-check
does not silently skip the block; instead, reinsert the popped cluster offset or
otherwise requeue the shortfall before continuing. Use the surrounding split,
block, count, staged, and prepare_block_for_dispatch logic to keep the current
claim consistent.
| // Split clusters (mixed running+pending within one MIX block) — the fix for | ||
| // stranded AIV cores. Source with the deadlock guard: a core whose running | ||
| // task is gated (STAGING) is not pending-eligible. | ||
| CoreTracker::BitStates split(0ULL); | ||
| CoreTracker::BitStates blocked_pending(0ULL); | ||
| if (shape == PTO2ResourceShape::MIX) { | ||
| const int32_t *cids = tracker.core_ids(); | ||
| int32_t nc = tracker.core_num(); | ||
| for (int k = 0; k < nc; k++) { | ||
| CoreExecState &ce = core_exec_states_[cids[k]]; | ||
| if (ce.running_slot_state != nullptr && ce.running_slot_state->payload != nullptr && | ||
| ce.running_slot_state->payload->spec_state.load(std::memory_order_acquire) == PTO2_SPEC_STAGING) { | ||
| blocked_pending |= CoreTracker::BitStates(1ULL << k); | ||
| } | ||
| } | ||
| split = tracker.get_mix_split_cluster_offset_states(c->active_mask.core_mask(), blocked_pending); | ||
| } | ||
| int32_t freecores = (idle.has_value() ? idle.count() : 0) + (pend.has_value() ? pend.count() : 0) + | ||
| (split.has_value() ? split.count() : 0); |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
ast-grep outline src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_types.h --match 'get_pending_core_offset_states' --items all
rg -n -A20 'BitStates get_pending_core_offset_states' src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_types.hRepository: hw-native-sys/simpler
Length of output: 1575
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Find where the pending helper is used in the scheduler path
rg -n -A8 -B8 'get_pending_core_offset_states\(' src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler
# Map the classification helper that the inline comment references
ast-grep outline src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_types.h --match 'classify_mix_cluster' --items all
rg -n -A80 -B20 'classify_mix_cluster' src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_types.hRepository: hw-native-sys/simpler
Length of output: 17238
🏁 Script executed:
#!/bin/bash
set -euo pipefail
sed -n '658,720p' src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_dispatch.cppRepository: hw-native-sys/simpler
Length of output: 3525
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n -A40 -B20 'stage_consumer_blocks\(' src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_dispatch.cpp
rg -n -A40 -B20 'blocked_pending' src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_dispatch.cppRepository: hw-native-sys/simpler
Length of output: 18536
Apply the deadlock guard to the MIX pending path. get_pending_core_offset_states(PTO2ResourceShape::MIX) can still return a fully-running cluster, and stage_consumer_blocks() forwards pend to prepare_block_for_dispatch(..., pending_mask=0xFF) without blocked_pending. That leaves a STAGING-gated running core eligible for whole-cluster PENDING dispatch and can deadlock the MIX path.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_dispatch.cpp`
around lines 661 - 679, The MIX pending path is missing the same deadlock guard
used for split clusters, so a STAGING-gated running core can still be treated as
pending-eligible. Update `stage_consumer_blocks()` and/or
`prepare_block_for_dispatch()` so the `pend` path also applies `blocked_pending`
when using `get_pending_core_offset_states(PTO2ResourceShape::MIX)`, and make
sure the MIX pending selection excludes cores whose
`CoreExecState::running_slot_state->payload->spec_state` is `PTO2_SPEC_STAGING`.
Use the existing `blocked_pending` logic from `scheduler_dispatch.cpp` as the
reference point and keep the whole-cluster PENDING dispatch from selecting a
fully-running MIX cluster when a gated core is present.
|
Latest perf-only comparison (PR691 pto-lib, Qwen3-14B decode,
相对收益：device 1.040%，sched 1.087%。 |
2 participants
Summary
Allow MIX (cube+AIV) task blocks to stage their AIC/AIV subtasks on whatever core slot is free — running slot if the core is idle, pending slot if the core is busy with a free pending slot — independently. This fixes the "stranded AIV cores" problem: when AIV finishes early but its sibling AIC is still running, the cluster is partially-idle and currently no MIX block can use it.
The in-kernel simultaneity contract (FFTS
syncall+ cross-core pipetpush/tpop) is upheld by gating the whole task behind the speculative doorbell until every subtask has promoted to a running slot, so all subtasks activate at the same instant at doorbell-ring.Changes
scheduler_types.hclassify_mix_cluster→MixPlacementPlan{SPLIT, pending_mask}with deadlock guard (pending subtask must NOT land behind a gated-running core)pto_scheduler.hmaybe_ring_split_doorbell: rings only when fanin complete + fully staged (next_block_idx >= logical_block_num) + all subtasks promoted.try_speculative_release: split tasks defer ring; non-split keep immediate ringscheduler_dispatch.cppprepare_block_for_dispatchper-subtaskpending_mask;stage_consumer_blockssources split clusters + countspromoted_subtasks+ callsmaybe_ring;try_speculative_early_dispatchsplit sourcing with deadlock guardscheduler_completion.cpppromoted_subtasks+ triggersmaybe_ringpto_runtime2_types.hpromoted_subtasks+requires_split_gatefields onPTO2TaskPayloadscheduler_context.htest_scheduler_state.cppKey design decisions
maybe_ring_split_doorbellchecksnext_block_idx >= logical_block_numBEFORE ringing — prevents a partial ring that would spin the launched blocks of a full-occupancy syncall task (e.g.fa_fused) holding cores the un-staged blocks need.classify_mix_clusterexcludes cores whose running task is gated (STAGING) from pending-eligibility — staging a pending subtask behind a gated running task deadlocks.requires_split_gateuse the deferred gate; non-split tasks ring immediately at fanin (sticky doorbell token handles pending subtasks).Test
ctest -R test_scheduler_state— classifier + gate UTs pass.python decode_fwd.py -p a2a3 -d <dev> --validate-fwd --fwd-layers 2 --enable-l2-swimlane— golden argmax passes.🤖 Generated with Claude Code