Skip to content

Allow stdin fuzz targets to suppress logs#4646

Merged
TheBlueMatt merged 1 commit into
lightningdevkit:mainfrom
joostjager:stdin-fuzz-no-logging
Jun 1, 2026
Merged

Allow stdin fuzz targets to suppress logs#4646
TheBlueMatt merged 1 commit into
lightningdevkit:mainfrom
joostjager:stdin-fuzz-no-logging

Conversation

@joostjager
Copy link
Copy Markdown
Contributor

@joostjager joostjager commented May 29, 2026

Add an environment-variable switch that lets stdin fuzz targets use the dev-null test logger. This keeps direct invocations verbose by default, while external runners can opt into quieter passing-case replays.

I use stdin_fuzz also to run big sets of test cases using a runner that parallelizes and also continues on failure to examine the rest. Cargo test and test_cases dir approach do not work for that. The logging is a big slow down.

Add an environment-variable switch that lets stdin fuzz targets use the
dev-null test logger. This keeps direct invocations verbose by default,
while external runners can opt into quieter passing-case replays.
@ldk-reviews-bot
Copy link
Copy Markdown

ldk-reviews-bot commented May 29, 2026

🎉 This PR is now ready for review!
Please choose at least one reviewer by assigning them on the right bar.
If no reviewers are assigned within 10 minutes, I'll automatically assign one.
Once the first reviewer has submitted a review, a second will be assigned if required.

@joostjager joostjager requested a review from TheBlueMatt May 29, 2026 13:46
@joostjager joostjager marked this pull request as ready for review May 29, 2026 13:46
@ldk-claude-review-bot
Copy link
Copy Markdown
Collaborator

Good - the types are compatible. DevNull and Stdout both implement Output + Clone + 'static, and the function accepts Out: test_logger::Output.

This PR is a clean, mechanical change. Every file follows the identical pattern, the template is updated, and no fuzz targets were missed. The DevNull type already existed and is properly implemented.

No issues found.

@ldk-reviews-bot
Copy link
Copy Markdown

🔔 1st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@TheBlueMatt
Copy link
Copy Markdown
Collaborator

I use stdin_fuzz also to run big sets of test cases using a runner that parallelizes and also continues on failure to examine the rest.

I guess I'm confused by this - generally once we hit a failure the rest will just be the same issue again and again reached with different inputs. Are you doing something to filter these cases? Otherwise I'm fine with this.

@joostjager
Copy link
Copy Markdown
Contributor Author

I use stdin_fuzz also to run big sets of test cases using a runner that parallelizes and also continues on failure to examine the rest.

I guess I'm confused by this - generally once we hit a failure the rest will just be the same issue again and again reached with different inputs. Are you doing something to filter these cases? Otherwise I'm fine with this.

After working on the fuzzer for a while, it turned out that I need to re-run quite large sets of fuzz strings (1000s) pretty deep inside the impl/test loop to verify that nothing accidentally broke after fixing the failures. Yes indeed, mainly to filter out what the new set of failures is, but also to keep track of all the panic sites, group them, detect timeouts, etc. Oftentimes there are multiple issues. Codex made himself a nice tool for this.

Copy link
Copy Markdown
Collaborator

@TheBlueMatt TheBlueMatt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mmm, okay, fair enough.

@ldk-reviews-bot
Copy link
Copy Markdown

👋 The first review has been submitted!

Do you think this PR is ready for a second reviewer? If so, click here to assign a second reviewer.

@TheBlueMatt TheBlueMatt merged commit bb597dd into lightningdevkit:main Jun 1, 2026
21 of 22 checks passed
@joostjager
Copy link
Copy Markdown
Contributor Author

joostjager commented Jun 1, 2026

Mmm, okay, fair enough.

I agree that it is a bit messy at the moment with the various ways the fuzzer can run, and the fake/real split didn't make it better. I can imagine that we might want to do a consolidation step some time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants