Playground validation: make a full 720-test sweep possible

[bkaradzic-microsoft]

Two changes that together make a complete run of the 720-test validation suite possible and order-independent. Two commits, reviewable separately.

1. --keep-going, so a sweep can finish

The runner exits on the first failing test, so a run only ever reports one failure and everything after it goes unmeasured. There's no way to answer "how many of the 720 tests pass today?" without repeatedly excluding whatever failed and re-running.

--keep-going / -k records the failure, continues, and lists every failing title in the summary. The run still exits non-zero if anything failed, so CI semantics are unchanged. Without the flag, behaviour is exactly as before.

Run complete. ran=3 passed=1 failed=2 missingRef=0 skipped=0
Failed tests (2):
  - Gaussian Splatting Compressed ply SH
  - Gaussian Splatting Update Data

Verified with two tests pointed at a mismatched reference image plus one control:

	result	exit
without --keep-going	ran=1 passed=0 failed=1	-1
with --keep-going	ran=3 passed=1 failed=2	-1

Framebuffer pool floor 512 -> 2048. Running to completion for the first time immediately exhausted the bgfx framebuffer pool around the FrameGraph tests (bgfx::createFrameBuffer returned invalid handle). 512 was only ever sufficient because the run stopped early. The existing comment in Dependencies/CMakeLists.txt already anticipated this ("combined with V8 GC pacing this can exceed the default limit during long playground sweeps"); this raises the floor to cover a full run.

2. Cross-test state leaks

The suite reuses one engine across all tests, so anything outliving scene.dispose() carries into later tests. The browser harness never sees this because it reloads the page per test.

• Stray scenes. A test can leave extra scenes registered on the engine (an async load that created its own scene, or one whose creation promise resolved after validation). They stay registered and keep their resources alive.
• The engine texture cache. BaseTexture._getFromCache keys on url/noMipmap/isCube only -- not on load-time options. A test that leaves a reference behind (e.g. assigning one texture to both scene.environmentTexture and a material's reflectionTexture) keeps its internal texture cached across scene.dispose(), so a later test loading the same URL silently reuses the previous test's texture along with its prefiltering/irradiance settings.
• SceneLoader.OnPluginActivatedObservable. Global, outlives the scene. Snippets use it to configure the glTF loader (animationStartMode, compileMaterials, ...) and never unregister, so every later glTF test inherits those settings.

Plus a hang guard: a createScene promise that never resolves currently hangs the entire suite, because the existing onReadyTimeout net lives inside processCurrentScene and only applies once the promise has resolved. This converts it to a single failed test. (Documented caveat: it only fires if the JS event loop keeps running.)

This second commit changes no current result -- 302 ran / 299 passed / 3 failed, identical set, before and after. Its value is removing order-dependence that becomes load-bearing as more tests are enabled.

Why it matters

This is what made a complete 720-test sweep possible at all. Measuring the full suite in one pass turned up 47 failures previously hidden behind the first one -- and spot-checking those in isolation shows they pass individually, so they're cross-test state pollution rather than missing functionality. That's a materially different and far more tractable problem than the tally alone suggested, and none of it is visible without this.

[Copilot]

Pull request overview

Adds a --keep-going (-k) mode to the native Playground validation runner so it can continue executing after failures and report a full pass/fail tally, and adjusts bgfx configuration to support long uninterrupted sweeps without exhausting the framebuffer pool.

Changes:

• Add --keep-going / -k CLI flag plumbing through PlaygroundOptions into the JS validation runner.
• Update validation_native.js to accumulate failing test titles and only exit after completing the sweep when --keep-going is enabled.
• Raise the enforced minimum BGFX_CONFIG_MAX_FRAME_BUFFERS floor from 512 to 2048 to avoid pool exhaustion during long runs.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
Dependencies/CMakeLists.txt	Raises the enforced minimum bgfx framebuffer pool size to handle full validation sweeps.
Apps/Playground/Win32/App.cpp	Passes the new KeepGoing option into the JS runtime options object.
Apps/Playground/Shared/CommandLine.h	Adds KeepGoing to PlaygroundOptions.
Apps/Playground/Shared/CommandLine.cpp	Adds --keep-going / -k flag parsing and help text.
Apps/Playground/Scripts/validation_native.js	Implements keep-going behavior and prints a failed-titles summary; exits non-zero if any failures occurred.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.