Skip to content

Fix stack overflow segfault on debug builds for Python threads#7986

Open
JamesClarke7283 wants to merge 4 commits into
RustPython:mainfrom
JamesClarke7283:fix-7941-thread-stack-overflow
Open

Fix stack overflow segfault on debug builds for Python threads#7986
JamesClarke7283 wants to merge 4 commits into
RustPython:mainfrom
JamesClarke7283:fix-7941-thread-stack-overflow

Conversation

@JamesClarke7283
Copy link
Copy Markdown
Contributor

@JamesClarke7283 JamesClarke7283 commented May 27, 2026

Summary

  • Spawned Python threads were getting Rust's std::thread::Builder default stack of 2 MB, which is too small for the call chains the Python stdlib runs on helper threads in debug builds (where Rust stack frames are substantially larger). CPython on glibc Linux relies on pthread's ~8 MB default instead — a 4× difference that explains why this hits us but not CPython.
  • Apply 8 MB in apply_thread_stack_size whenever the user hasn't explicitly set a value via threading.stack_size(N). This matches CPython's effective default while keeping the Python API contract exact: threading.stack_size() still returns 0 ("platform default"), and explicit overrides still take precedence.
  • Single-file change in crates/vm/src/stdlib/_thread.rs.

Closes #7941

Test plan

  • cargo build succeeds.
  • Original reproducer passes on a debug build: cargo run -- -m test.test_ssl ThreadedTests.test_socketserverRan 1 test in 21.59s, OK (was segfaulting on main).
  • Python API contract preserved: threading.stack_size() returns 0 by default; setting to 1 MiB then reading back returns 1048576; previous value reported correctly.
  • Basic threaded workload (5 threads doing work + join) still completes normally.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes
    • Made thread stack sizing more predictable in debug builds: when no stack size is configured, debug builds now use a consistent explicit fallback to avoid platform-dependent variance. Release builds continue to use the platform default when no stack size is set. This reduces surprising differences during development and debugging.

Review Change Stack

Rust's std::thread::Builder defaults to a 2 MB stack when no size is
set, which is too small for the call chains the Python stdlib runs on
helper threads in debug builds (e.g. test.test_ssl's threaded server).
CPython on glibc Linux relies on pthread's ~8 MB default instead.

Apply 8 MB as the default in apply_thread_stack_size when the user has
not explicitly called threading.stack_size(N). This matches CPython's
effective default across builds while preserving the existing API:
threading.stack_size() still returns 0 by default ("platform default"),
and explicit user values still take precedence.

Fixes RustPython#7941

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 27, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: d0fe7326-2007-4599-a6b8-8ea0b3df4da0

📥 Commits

Reviewing files that changed from the base of the PR and between e4011c1 and c8b134e.

📒 Files selected for processing (1)
  • crates/vm/src/stdlib/_thread.rs

📝 Walkthrough

Walkthrough

When starting helper threads, apply_thread_stack_size uses vm.state.stacksize if non-zero; if zero, debug builds set a debug-only DEFAULT_THREAD_STACK_SIZE (8 MiB), while non-debug builds do not modify the thread::Builder (Rust std default remains).

Changes

Thread Stack Sizing

Layer / File(s) Summary
Stack-size constant and selection logic
crates/vm/src/stdlib/_thread.rs
Adds DEFAULT_THREAD_STACK_SIZE (8 MiB, debug-only). apply_thread_stack_size applies vm.state.stacksize when non-zero; when zero it sets the builder to the default in debug builds and leaves the builder unchanged in non-debug builds.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Suggested reviewers

  • ShaharNaveh
  • youknowone

Poem

🐰 I nudged the stack to eight MiB bright,
In debug I set its cozy height,
When users say zero, I choose with care,
Release leaves defaults, debug makes it fair,
Threads hop steady through the night.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically describes the main change: fixing a stack overflow segfault issue that occurs on debug builds for Python threads.
Linked Issues check ✅ Passed The pull request fully addresses issue #7941 by implementing an 8 MiB default thread stack size for debug builds, eliminating segfaults in threaded tests without changing the Python API contract.
Out of Scope Changes check ✅ Passed All changes are directly scoped to fixing the debug-build stack overflow issue; the single-file modification to _thread.rs applies the necessary stack size configuration with no unrelated alterations.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

JamesClarke7283 and others added 3 commits May 27, 2026 09:47
Satisfy docstring coverage check on the PR.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous commit applied an 8 MB default unconditionally, which
slowed down multiprocessing tests on release: many forked children
each spawning many threads with oversized virtual stack mappings
caused the flaky MP CI step to time out (60 min, vs ~9 min on main).

Issue RustPython#7941 only manifested in debug builds — release builds were
already fine on Rust's 2 MB std default. Restrict the 8 MB override
to `#[cfg(debug_assertions)]` so release behavior is unchanged from
before the fix, while the original SSL segfault on debug stays fixed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@youknowone
Copy link
Copy Markdown
Member

please also add failing test before patch

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Some python tests are crashing with segmentation fault when running on debug build

2 participants