glideapps · carloscasellas · Aug 13, 2026
diff --git a/script/benchmark-class-width/README.md b/script/benchmark-class-width/README.md
@@ -0,0 +1,135 @@
+# Class-width benchmark (issue #559)
+
+Manual for reproducing the Rust measurement used to answer
+[#559](https://github.com/glideapps/quicktype/issues/559) — see `RESULTS.md`
+for the write-up. This benchmark only concerns Rust; it's the toolchain the
+issue names and the one CI actually compiles (`.github/workflows/test-pr.yaml`,
+`fixture: rust,schema-rust`).
+
+## What it's testing
+
+#559 claims `blns-object.json` and `keyword-unions.schema` have objects big
+enough to slow down Rust compilation. The claimed mechanism: serde's
+`#[derive(Deserialize)]` expands an N-field struct into one `visit_map`
+function with N locals, an N-arm match, and N missing-field checks — cost
+concentrated in one function, not spread across the file.
+
+To test that, candidates are built holding the **same keys spread across more,
+narrower classes** — identical total field count, so a linear cost model
+predicts identical compile times. Any drop would be the superlinear part the
+issue is about. `n=1` is a control: same grouping as the checked-in file, so it
+must reproduce today's numbers or the harness is wrong.
+
+## Requirements
+
+```bash
+npm run build                              # need dist/index.js
+curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh   # if cargo isn't installed
+export PATH="$HOME/.cargo/bin:$PATH"       # rustup installs here, not on PATH by default
+```
+
+Run everything from the repo root. Nothing here writes into the repo — all
+candidates and build artifacts live under `$BENCH`, a scratch directory:
+
+```bash
+export BENCH=/tmp/bench          # or wherever you like
+mkdir -p $BENCH/inputs
+B=script/benchmark-class-width
+```
+
+## 1. Build candidates
+
+Same keys, increasingly narrow classes:
+
+```bash
+for n in 1 2 4 8;  do python3 $B/reshape-blns.py $n           > $BENCH/inputs/blns-$n.json;  done
+for n in 1 2 5 10; do python3 $B/reshape-keyword-unions.py $n > $BENCH/inputs/ku-$n.schema; done
+```
+
+## 2. Tier 0 — structural check (no toolchain, ~5 seconds)
+
+Confirms the candidates isolate the right variable: total code volume should
+stay roughly flat while the widest class drops sharply.
+
+```bash
+python3 $B/measure.py --lang rust --src-lang json   $BENCH/inputs/blns-{1,2,4,8}.json
+python3 $B/measure.py --lang rust --src-lang schema $BENCH/inputs/ku-{1,2,5,10}.schema
+```
+
+Reference output:
+
+```
+candidate                  lines   types  widest  widest type
+--------------------------------------------------------------------
+blns-1.json                 1776       4     230  A
+blns-2.json                 1801       7     116  A2
+blns-4.json                 1844      13      59  A4
+blns-8.json                 1941      25      30  A2
+
+candidate                  lines   types  widest  widest type
+--------------------------------------------------------------------
+ku-1.schema                 3952     554     276  Group1
+ku-2.schema                 3957     555     138  Group1
+ku-5.schema                 3972     558      56  Group5
+ku-10.schema                3997     563      28  Group10
+```
+
+Lines move under 10% while widest class drops 8-10×.
+
+## 3. Tier 1 — the actual timing
+
+```bash
+REPS=10 bash $B/bench-rust.sh --src-lang json   $BENCH/inputs/blns-{1,2,4,8}.json
+REPS=10 bash $B/bench-rust.sh --src-lang schema $BENCH/inputs/ku-{1,2,5,10}.schema
+```
+
+`bench-rust.sh` pins a shared `CARGO_TARGET_DIR` and does a throwaway warm-up
+build first, so serde compiles once and every timed build recompiles only the
+crate under test. Without that, the per-sample serde rebuild the real fixture
+harness pays (~8s) would swamp the ~1-2s signal being measured. `touch` before
+each build forces a genuine recompile. If a build fails, the script aborts
+loudly rather than reporting a fast, meaningless number — a failed compile
+looks like a great result if you don't check for it.
+
+`REPS=10` takes ~15-25s per candidate; use `REPS=3` for a quick check, `REPS=10`
+for a number worth quoting — single-digit rep counts on the `ku` schema were
+noisy enough in early runs to be misleading.
+
+Reference output:
+
+```
+candidate                widest    lines   median_s
+------------------------------------------------------
+blns-1.json                 230     1776       0.73
+blns-2.json                 116     1801       0.74
+blns-4.json                  59     1844       0.78
+blns-8.json                  30     1941       0.82
+
+candidate                widest    lines   median_s
+------------------------------------------------------
+ku-1.schema                 276     3952       1.98
+ku-2.schema                 138     3957       1.97
+ku-5.schema                  56     3972       2.00
+ku-10.schema                 28     3997       2.02
+```
+
+## Reading the result
+
+- **Steep drop as width falls** → the premise holds.
+- **Flat, or a slight rise** → the premise doesn't hold on this rustc.
+
+Measured outcome: flat on `ku`, and `blns` gets **monotonically slower** as
+classes narrow (0.73s → 0.82s, +12%) — more types cost more than the wide
+struct saves. See `RESULTS.md` for the full write-up and what's worth doing
+instead.
+
+## Troubleshooting
+
+- `cargo not found` → `export PATH="$HOME/.cargo/bin:$PATH"` (not persisted by
+  the rustup install used here).
+- Glob like `$BENCH/inputs/ku-{1,2,5,10}.schema` passed through literally →
+  `$BENCH` isn't set in this shell, or step 1 wasn't run in it, so the files
+  don't exist and the brace expansion has nothing to match.
+- `BUILD FAILED for ... -- timings would be meaningless` → the script caught a
+  real compile error; the tail of `rustc`'s output is printed above the
+  message.
diff --git a/script/benchmark-class-width/RESULTS.md b/script/benchmark-class-width/RESULTS.md
@@ -0,0 +1,141 @@
+# Issue #559 — measured on Rust, and the premise doesn't hold
+
+**Recommendation: close #559 as obsolete.** Narrowing the widest generated Rust
+struct by 8-10× produces no compile-time improvement. If anything, it's
+slightly slower.
+
+Measured 2026-08-12/13, macOS / Apple Silicon, rustc + cargo 1.97.1.
+
+## The claim under test
+
+#559 says `blns-object.json`, `keywords.json`, and `keyword-unions.schema` have
+objects big enough to slow down compilation, "particularly Rust." It comes from
+[#516](https://github.com/glideapps/quicktype/issues/516), where profiling with
+`cargo-expand` and `-Ztime-passes` found 40k expanded lines, 30k in a single
+method, dominated by rustc's `translation` pass.
+
+The claimed mechanism: for an N-field struct, serde's `#[derive(Deserialize)]`
+expands into one `visit_map` function with N locals, an N-arm match, and N
+missing-field checks. Cost concentrates in **one function**, not spread across
+the file. Rust is the language actually named, the only one profiled in #516,
+and the only one this repo's CI compiles for these fixtures
+(`.github/workflows/test-pr.yaml`, `fixture: rust,schema-rust`) — so it's the
+toolchain this benchmark measures directly rather than by inference.
+
+## Method
+
+Candidates hold the same keys spread across more, narrower classes — **identical
+total field count**, so a linear cost model predicts identical compile times.
+Any drop would be the superlinear component the issue describes.
+
+| candidate | lines | types | widest class |
+|---|---|---|---|
+| `blns-1` *(= the checked-in file)* | 1,776 | 4 | **230** |
+| `blns-2` | 1,801 | 7 | 116 |
+| `blns-4` | 1,844 | 13 | 59 |
+| `blns-8` | 1,941 | 25 | 30 |
+
+| candidate | lines | types | widest class |
+|---|---|---|---|
+| `ku-1` *(≈ the checked-in file)* | 3,952 | 554 | **276** |
+| `ku-2` | 3,957 | 555 | 138 |
+| `ku-5` | 3,972 | 558 | 56 |
+| `ku-10` | 3,997 | 563 | 28 |
+
+Total lines move under 10% in both cases while the widest class drops 8-10×.
+Three guards: `n=1` is a control that must reproduce the checked-in file (it
+does); every candidate is asserted lossless (identical key/value multiset); and
+`bench-rust.sh` aborts if a build fails, since a failed build is fast and would
+otherwise fake a win.
+
+**What's timed:** `cargo build` only — compile time, not `cargo run`, not the
+JSON round-trip. serde is prebuilt once in a shared `CARGO_TARGET_DIR`, so each
+timed build recompiles only `module_under_test.rs`. This isolates the specific
+thing #559 is about (rustc compiling a wide struct), separate from the
+per-sample fixture overhead covered below.
+
+## Results
+
+Median of 10 builds per candidate:
+
+| widest class | `blns` compile | `keyword-unions` compile |
+|---|---|---|
+| 276 / 230 *(control)* | 0.73s | 1.98s |
+| 138 / 116 | 0.74s | 1.97s |
+| 56 / 59 | 0.78s | 2.00s |
+| 28 / 30 | **0.82s** | 2.02s |
+
+`keyword-unions` is flat to within 3%. `blns` is **monotonically slower** as
+classes narrow — 0.73s → 0.82s, +12% — tracking the 9% growth in emitted lines:
+splitting one wide struct into several narrow ones adds more code than it
+removes cost. Neither result supports the issue's premise on the toolchain it
+actually names.
+
+rustc has also changed substantially since the 2018 profile (rustc ~1.24 era);
+whatever superlinear cost `-Ztime-passes` found in `translation` back then does
+not reproduce on 1.97.
+
+## Where fixture time actually goes
+
+For context on the ~9% number below and why it matters:
+
+```
+one rust sample through the harness:  8,656 ms
+of which the crate compile is:          790 ms   (9%)
+```
+
+The other ~91% is dependency compilation: `fixtures.ts:281` copies
+`test/fixtures/rust` into a fresh directory per sample, `RustLanguage` has no
+`setupCommand`, and no `CARGO_TARGET_DIR` is set anywhere in the repo, so cargo
+rebuilds serde/serde_derive/serde_json from scratch every time.
+
+Two things follow from this, and they cut in opposite directions from #559:
+
+- Splitting `blns-object.json` into 4 files, as #559 suggests, adds 3 more
+  samples — roughly 3 × 8.66s ≈ **26 seconds** per Rust JSON run — for a
+  compile-time change measured at ±0.1s. That's a net loss.
+- The 91% figure suggests the real lever is elsewhere: **setting a shared
+  `CARGO_TARGET_DIR` for the Rust fixture is worth investigating separately.**
+  This is a plausible optimization based on where the time currently goes in a
+  single measured sample, not something this benchmark proves out end-to-end.
+  It wasn't tested against the full 52-sample suite, doesn't account for cache
+  contention if the harness's forked workers (`test/lib/multicore.ts`) write to
+  the same target directory concurrently, and CI's cold-start-per-job model may
+  behave differently than this local, warm-cache measurement. Worth a
+  follow-up spike, not a claimed fix.
+
+## Scope of the evidence
+
+This result is Rust-only, deliberately — see "The claim under test" above for
+why that's the toolchain that matters for #559. Two earlier passes on other
+toolchains (C++/g++ and Swift/swiftc, both also flat) and a discussion of
+JVM/C# coverage are not reproduced here; ask if that's wanted for a broader
+writeup.
+
+## Worth doing instead
+
+**1. Python cannot run `keyword-unions.schema` at all.** It generates a
+`KeywordUnions(...)` call with **276 arguments**, past CPython's 255 limit —
+hence `test/languages.ts:336`:
+
+```ts
+skipSchema: [
+    "keyword-unions.schema", // Requires more than 255 arguments
+],
+```
+
+Grouping that schema's properties would let Python run it. This is the one
+surviving reason to reshape a fixture, and it is a **coverage** fix, not a
+performance one — independent of everything above.
+
+**2. `keywords.json` has drifted from its generator.** It contains three
+keywords absent from `test/keywords.txt` — `clone`, `equalityContract`,
+`printMembers` (C# record members, hand-inserted). `keyword-unions.schema` and
+`keyword-enum.schema` both regenerate byte-identically; `keywords.json` does
+not. **Running `test/make-keyword-tests.sh` today silently deletes that
+coverage.** Unrelated to #559; worth its own fix.
+
+## Reproducing
+
+See `README.md` in this directory. Everything runs from a scratch `$BENCH`
+directory; no repo files are modified.
diff --git a/script/benchmark-class-width/bench-cpp.sh b/script/benchmark-class-width/bench-cpp.sh
@@ -0,0 +1,63 @@
+#!/usr/bin/env bash
+# Tier 1, C++ -- the strongest non-Rust case for the hypothesis.
+#
+# nlohmann/json instantiates from_json/to_json templates per field, so a wide
+# class means a lot of template instantiation concentrated in one translation
+# unit.  Unlike Rust there is no dependency build to exclude: json.hpp is
+# header-only, so each compile is the whole cost and no shared target dir trick
+# is needed.
+#
+# Usage:  bench-cpp.sh --src-lang json $BENCH/inputs/blns-{1,2,4,8}.json
+set -euo pipefail
+
+REPS="${REPS:-3}"
+SRC_LANG=json
+if [ "${1:-}" = "--src-lang" ]; then SRC_LANG="$2"; shift 2; fi
+
+command -v g++ >/dev/null || { echo "g++ not found"; exit 1; }
+[ -f dist/index.js ] || { echo "dist/index.js missing -- run npm run build"; exit 1; }
+
+BENCH="${BENCH:-${CLAUDE_JOB_DIR:-/tmp}/bench}"
+work="$BENCH/cpp"
+mkdir -p "$work"
+cp test/fixtures/cplusplus/main.cpp test/fixtures/cplusplus/Generators.hpp "$work/"
+
+# Same pinned revision the cplusplus fixture's setupCommand uses.
+if [ ! -f "$work/json.hpp" ]; then
+    echo "fetching json.hpp ..." >&2
+    curl -fsS -o "$work/json.hpp" \
+        https://raw.githubusercontent.com/nlohmann/json/87df1d6708915ffbfa26a051ad7562ecc22e5579/src/json.hpp
+fi
+
+printf "%-22s %8s %8s %10s\n" candidate widest lines "median_s"
+printf -- "------------------------------------------------------\n"
+
+for input in "$@"; do
+    node dist/index.js --lang cplusplus --src-lang "$SRC_LANG" --top-level TopLevel \
+        "$input" > "$work/quicktype.hpp" 2>/dev/null
+
+    widest=$(awk '
+        /^ *(struct|class) [A-Za-z_]/ { inb=1; n=0; next }
+        inb && /^ +[A-Za-z_].*;$/ { n++ }
+        inb && /^ *};/ { inb=0; if (n>max) max=n }
+        END { print max+0 }' "$work/quicktype.hpp")
+    lines=$(wc -l < "$work/quicktype.hpp" | tr -d ' ')
+
+    # A failed compile is fast and would look like a great result.
+    if ! (cd "$work" && g++ -O0 -o quicktype -std=c++17 main.cpp 2>"$work/err"); then
+        echo "COMPILE FAILED for $(basename "$input") -- timings would be meaningless" >&2
+        head -20 "$work/err" >&2
+        exit 1
+    fi
+
+    times=()
+    for _ in $(seq "$REPS"); do
+        start=$(python3 -c 'import time; print(time.time())')
+        (cd "$work" && g++ -O0 -o quicktype -std=c++17 main.cpp 2>/dev/null) \
+            || { echo "compile failed mid-run" >&2; exit 1; }
+        times+=("$(python3 -c "import time; print(f'{time.time()-$start:.2f}')")")
+    done
+    median=$(printf '%s\n' "${times[@]}" | sort -n | awk '{a[NR]=$1} END {print a[int((NR+1)/2)]}')
+
+    printf "%-22s %8s %8s %10s\n" "$(basename "$input")" "$widest" "$lines" "$median"
+done