feat: implement transaction processor crate

[bmuddha]

What changed

Added the magicblock-processor crate, which schedules and executes transactions
over the keeper.

Why

Inbound transactions need conflict-aware scheduling that runs disjoint account
sets concurrently while reading and writing accounts through keeper/accountsdb.

Closes #11.

Impact

• A sequencer fans conflict-free transactions across a pool of SVM executors and
  serializes the rest.
• Conflicts are tracked per Pubkey with a write bit plus a per-executor
  occupancy bitset (MAX_EXECUTORS).
• Adds a simulation path that runs against current state on owned account copies
  and returns the execution record without committing.

Reviewer notes

A quiescence barrier drains in-flight work for consistent snapshots at superblock
seals and during replay — the main concurrency-correctness surface.

Follow-up

engine wires this sequencer to durable state upstack.

[bmuddha]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• feat: add state replicator crate on top of the engine #25 : 2 dependent PRs (#37 , #55 )
• feat: adds top level engine crate for global orchestration #24
• feat: added internal built-in magic root program #23
• feat: implement transaction processor crate #22 👈 (View in Graphite)
• feat: implements keeper crate - state orchestrator #17
• feat: implement superblock based ledger crate #15
• feat: add optimized accountsdb crate #14
• feat: add nucleus engine crate #18
• feat: add svm crate #21
• feat: add program runtime crate #20
• feat: add transaction context crate #19
• feat: forked transaction-view for increased limits #33
• feat: add customized solana-account crate #13
• chore: import upstream Solana runtime crates #31
• dev

This stack of pull requests is managed by Graphite. Learn more about stacking.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 7d46cb0f-3fce-4962-9fbf-b1217d5f8774

📥 Commits

Reviewing files that changed from the base of the PR and between ee58853 and ccc2f20.

📒 Files selected for processing (1)

• processor/src/sequencer/mod.rs

🚧 Files skipped from review as they are similar to previous changes (1)

• processor/src/sequencer/mod.rs

---

📝 Walkthrough

Walkthrough

The new processor crate adds parallel SVM execution, account-lock scheduling, block finalization, isolated simulation, metrics, error handling, and end-to-end tests. It is registered in the workspace and documents its execution model.

Changes

Transaction processor

Layer / File(s)	Summary
Runtime contracts and SVM execution Cargo.toml, processor/Cargo.toml, processor/src/lib.rs, processor/src/error.rs, processor/src/metrics.rs, processor/src/callback.rs, processor/src/svm.rs	Adds the workspace crate, processor messages and errors, metrics, account callbacks, and block-aware SVM execution.
Executor worker lifecycle processor/src/executor.rs	Adds worker-thread executors that process transaction batches and block transitions, report completion, and persist results with replay-mode handling.
Scheduling, locks, and block coordination processor/src/sequencer/*, processor/README.md	Adds executor-pool management, account-lock coordination, blocked-transaction retry, barriers, draining, shutdown, hash chaining, block finalization, and related tests and documentation.
Isolated transaction simulation processor/src/simulator.rs, processor/README.md	Adds a dedicated simulator worker for non-persistent transaction execution and documents simulation behavior.
End-to-end processor validation processor/src/tests.rs	Adds tests for execution and simulation, block hashes, CPI loading, sysvar transitions, replay mode, failures, and contention workloads.

Estimated code review effort: 5 (Critical) | ~90 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant Sequencer
  participant TransactionExecutor
  participant SvmContext
  participant Keeper

  Client->>Sequencer: submit resolved transactions
  Sequencer->>TransactionExecutor: dispatch conflict-free batch
  TransactionExecutor->>SvmContext: execute transaction batch
  SvmContext->>Keeper: load account state
  TransactionExecutor->>Keeper: persist execution records
  TransactionExecutor->>Sequencer: report completed batch
  Sequencer->>TransactionExecutor: release locks and dispatch blocked work

Loading

Possibly related PRs

• magicblock-labs/magicblock-engine#17: Provides Keeper APIs for account loading, state commits, transaction execution, block handling, and simulation.
• magicblock-labs/magicblock-engine#18: Provides runtime, ledger, shutdown, service, metrics, and transaction types used by the processor.
• magicblock-labs/magicblock-engine#19: Provides customized solana-transaction-context APIs used by SVM execution and callbacks.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely identifies the main change: implementing the transaction processor crate.
Description check	✅ Passed	The description explains the processor crate, its scheduling behavior, simulation path, concurrency model, and relation to issue #11.
Linked Issues check	✅ Passed	The changes implement issue #11 requirements for scheduling, parallel executor dispatch, conflict sequencing, state access, persistence, and ordering.
Out of Scope Changes check	✅ Passed	The added modules, metrics, documentation, tests, and workspace integration directly support the transaction processor objectives.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch processor

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[bmuddha]

@CodeRabbit review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

Actionable comments posted: 3

🧹 Nitpick comments (1)

processor/src/tests.rs (1)

51-66: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Document the channel-capacity coupling of the prefill tests.

unspawned lets tests fill the execution channel before the sequencer runs. The channel capacity is 1024 (processor/src/sequencer/mod.rs Line 90). The prefill tests send 128 and 512 transactions. If the capacity is lowered below a test's transaction count, Harness::execute blocks forever and the test hangs instead of failing. Add a note here so the constraint stays visible.

📝 Proposed doc note

     /// This lets tests fill the execution channel before the sequencer can
     /// consume from it, forcing contention resolution to happen from a backlog.
+    ///
+    /// Callers must keep their prefill count below the sequencer's execution
+    /// channel capacity. A larger prefill blocks `execute` and hangs the test.
     async fn unspawned(replay: bool) -> (Self, Sequencer) {

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@processor/src/tests.rs` around lines 51 - 66, Add a documentation note to
Harness::unspawned stating that prefill tests must remain at or below the
execution channel capacity of 1024, because exceeding it causes Harness::execute
to block while the sequencer is unspawned. Mention that the existing 128- and
512-transaction tests depend on this constraint.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@processor/src/sequencer/locks.rs`:
- Around line 77-78: Update the rustdoc for the `release` method to state that
it releases every account lock read from `executor.locks`, replacing the stale
`held` reference while preserving the documented behavior.
- Around line 60-73: Remove each zero-count account from executor.locks during
the rollback loop in LockTable’s release path, while preserving lock.contend and
unlock behavior for remaining counts; update processor/src/sequencer/tests.rs
lines 138-138 to assert blocked.locks.get(&a) is None. Also make
AccountLock::unlock clear WRITE_BIT only when that executor actually owns it.

In `@processor/src/simulator.rs`:
- Around line 51-65: Update the worker loop in run so self.rx.recv() explicitly
exits when the channel returns None, preserving message handling for Some(msg)
and the documented behavior that channel closure ends the worker.

---

Nitpick comments:
In `@processor/src/tests.rs`:
- Around line 51-66: Add a documentation note to Harness::unspawned stating that
prefill tests must remain at or below the execution channel capacity of 1024,
because exceeding it causes Harness::execute to block while the sequencer is
unspawned. Mention that the existing 128- and 512-transaction tests depend on
this constraint.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 7fd7928a-706f-4d11-bafe-0a0c5ca26191

📥 Commits

Reviewing files that changed from the base of the PR and between e5f37e1 and ac70027.

📒 Files selected for processing (15)

• Cargo.toml
• processor/Cargo.toml
• processor/README.md
• processor/src/callback.rs
• processor/src/error.rs
• processor/src/executor.rs
• processor/src/lib.rs
• processor/src/metrics.rs
• processor/src/sequencer/locks.rs
• processor/src/sequencer/mod.rs
• processor/src/sequencer/pool.rs
• processor/src/sequencer/tests.rs
• processor/src/simulator.rs
• processor/src/svm.rs
• processor/src/tests.rs

[bmuddha]

@CodeRabbit review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@processor/src/sequencer/mod.rs`:
- Around line 251-276: Update finalize so its documentation accurately describes
the block boundary sequencing, and preferably append the block via
state.blocks().append before notifying executor handles through
ExecutorMessage::Block. Preserve error propagation and ensure hash advancement
and slot updates remain consistent with the chosen ordering.
- Around line 134-140: Track whether the sequencer loop in the error-handling
branch exits because of a failure, setting a `failed` indicator before breaking
after logging the error. At the shutdown reason selection near `ShutdownReason`,
use that indicator to report the failure-specific reason instead of
`ShutdownReason::Signalled`, while preserving the signalled reason for normal
shutdown requests.
- Around line 65-107: Ensure Sequencer::new never creates an executor pool with
zero workers: clamp the requested executors value to at least 1 before
calculating count and initializing the channel. Preserve the existing
MAX_EXECUTORS limit so schedule always has a valid executor.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 89521626-0b3c-4d5d-937f-1e22e22327f4

📥 Commits

Reviewing files that changed from the base of the PR and between aa3abc8 and ee58853.

📒 Files selected for processing (3)

• Cargo.toml
• processor/src/executor.rs
• processor/src/sequencer/mod.rs

🚧 Files skipped from review as they are similar to previous changes (2)

• Cargo.toml
• processor/src/executor.rs