test: add class-width compile-time benchmark for issue 559 - #3118
Open
carloscasellas wants to merge 1 commit into
Open
test: add class-width compile-time benchmark for issue 559#3118carloscasellas wants to merge 1 commit into
carloscasellas wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merging this PR is not necessary, I wanted to provide the scripts in case anyone looking at issue 559 wanted to look at them.
Description
Adds a reproducible benchmark (
script/benchmark-class-width/) measuring whether narrowing the widest generated Rust struct — as issue #559 proposes — actually improves compile time. It doesn't add or change any generated-code behavior; it's a standalone investigation with scripts, a manual (README.md), and a results write-up (RESULTS.md).Related Issue
Investigates #559. Recommends closing it as obsolete based on the measurements below.
Motivation and Context
#559 claims
blns-object.json,keywords.json, andkeyword-unions.schemahave objects large enough to slow down compilation, "particularly Rust," citing profiling from #516 (cargo-expand+-Ztime-passeson rustc ~1.24, showing 30k lines concentrated in onevisit_mapfunction from serde's derive).That's an eight-year-old measurement. Before doing the work of splitting these fixtures, I wanted to confirm the underlying cost still exists on a current toolchain.
Previous Behaviour / Output
No benchmark existed for this — the issue's justification was 2018-era profiling with no way to re-verify it against today's rustc.
New Behaviour / Output
script/benchmark-class-width/reshape-blns.pyandreshape-keyword-unions.pyregroup the fixtures' keys into progressively narrower classes while holding the total field count constant, so any compile-time change is attributable to width rather than data volume.bench-rust.shtimescargo buildagainst each candidate with serde prebuilt in a sharedCARGO_TARGET_DIR, isolating the compile of the generated module itself.Result on rustc/cargo 1.97.1 (median of 10 builds per candidate):
blnscompilekeyword-unionscompilekeyword-unionsis flat within noise.blnsgets monotonically slower as classes narrow (+12%), tracking the extra code from more, smaller types. Neither supports the issue's premise on current rustc.Separately, ~91% of a Rust sample's cost in the real fixture harness turns out to be rebuilding serde from scratch per sample (no
setupCommand, no sharedCARGO_TARGET_DIRintest/languages.ts/test/fixtures.ts) — noted inRESULTS.mdas a lead worth its own follow-up spike, not a verified fix.Full write-up, including the
keyword-unions.schema→ Python 255-argument limit finding and thekeywords.jsongenerator-drift bug found along the way, is inscript/benchmark-class-width/RESULTS.md.How Has This Been Tested?
This PR is the test — a benchmark, not a code change. To reproduce: