Skip to content

[codex] add RCL dominance arena scorecards - #57

Draft
xingxuling wants to merge 6 commits into
mainfrom
codex/rcl-dominance-arena-v01
Draft

[codex] add RCL dominance arena scorecards#57
xingxuling wants to merge 6 commits into
mainfrom
codex/rcl-dominance-arena-v01

Conversation

@xingxuling

Copy link
Copy Markdown
Owner

What this changes

RCL's Universal Stress program answers whether a task is expressible and executable across environments. Its legacy nine-gate cell currently lets an independently unverified AI_GENERATE gate collapse an otherwise evidenced compiler result into BLOCKED, while the repository has no executable vertical comparison contract for the question "who is stronger?"

This PR adds an evidence-bound RCL Dominance Arena v0.1 with three independent scorecard axes:

  • Capability: all Universal Stress gates except AI_GENERATE.
  • Dominance: comparable raw candidate/reference metrics with no weighted compensation.
  • Authorability: the independent AI_GENERATE contract.

The first arena executes the production K01 self-host verification path as a real candidate and records optional reference tool probes. It deliberately reports Dominance: UNVERIFIED until a semantically matched reference corpus is available.

Root cause and fix

The K01 adapter counted the historical stage0 proxy boundary as a compiler correctness stage. stage0 explicitly reports that the core runtime is still proxy-hosted and is outside the self-hosting compiler proposition, so a valid 40/40 compiler-stage run was reported as 40/41 and failed K01 correctness. The adapter now excludes only stage0 from K01 compiler correctness while preserving its report and boundary; a run with no compiler stages still fails closed.

The arena command runner also records command exit status, timeout/tool-not-found failure types, output hashes, artifact hashes and the checked-out Git revision. Missing comparable references remain visible rather than becoming a false win.

Validation

  • npm run test:dominance-arena: 10/10 pass.
  • Universal Stress/K01/K02/K03 focused regression: 29/29 pass.
  • Real npm run evidence:dominance-arena on the committed source: candidate execution PASS; K01 capability PASS; legacy K01 BLOCKED only on AI_GENERATE; dominance and authorability UNVERIFIED.
  • Real Arena report root: 30d6f5a13d319982dca87ecb2eec812c198859568c1124c5d82a6a1f6c53cc27.
  • Full npm test: 793 pass, 8 unrelated existing Frontier/environment failures, 1 skipped. The failures are outside this PR's changed modules and include Windows temporary-path composition, host interaction detection, continuous-field contract classification and self-record count assumptions.

The PR does not claim RCL is already the strongest compiler language. It supplies the executable measurement boundary needed to make that claim falsifiable.

@vercel

vercel Bot commented Aug 15, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
rcl-rncs-mcp Ready Ready Preview Aug 15, 2026 9:43pm

@xingxuling

Copy link
Copy Markdown
Owner Author

CI note: the three required workflows were rejected before any step started. GitHub reports runner_id=0 and steps=[], and recent main runs show the same condition. This is the repository/account billing or spending-limit infrastructure blocker documented in the existing evidence ledgers, not a test assertion failure. Local verification remains recorded in the PR body; do not treat this PR as CI-passed until a real hosted runner executes the workflows.

@xingxuling

Copy link
Copy Markdown
Owner Author

继续推进已完成并推送到 commit 7501bef。\n\n新增:\n- provider-evidence comparison contract:同一 inputRoot 才允许比较,缺失/不匹配保持 UNVERIFIED。\n- compiler-microbench-v0.1:同一算术链语料的真实 RCL compileRealityToBytecode + native VM,以及真实 rustc + executable reference。\n- 原始指标:correctness、compileBuildSpeed、runtimeMs、artifactFootprintBytes;无加权平均。\n- 2 个真实执行回归测试和 Arena 动态契约测试。\n\n最终本地证据:\n- reportRoot: f358014806b02bcef135d50b8f2b0a401cca3222706e8f21af3b30754bc69017\n- inputRoot: b6a5b9512d04d884eae20d61c543255252c377c79307261ee204de714f9a7363\n- candidate/reference: PASS/PASS\n- narrow microbenchmark dominance: PASS\n- RCL compile ms: 6.251; rustc compile ms: 494.236\n- RCL runtime ms: 47.803; reference runtime ms: 89.279\n- RCL artifact: 2948 bytes; reference artifact: 147968 bytes\n- RCL RBC SHA-256: CBF8B4BBF38921EBEC8B7A63E1344A96E68135BA4903E5D4F35204C4B313688E\n- reference EXE SHA-256: 7E41A38CE3A942F6846258A4A4A678FD418D027E550F3E50853057B8E5FB9BAB\n\n验证:\n- Arena 11/11\n- microbench execution 2/2\n- K01/K02/K03 focused 8/8\n- full npm test: 805 total, 796 pass, 8 pre-existing Windows/frontier failures, 1 skip\n\n边界:该 PASS 只证明固定 arithmetic-chain microbenchmark 的 raw metrics,不证明 whole-language/ecosystem/memory/authorability/commercial superiority。\n\nCI:latest runs for this SHA still fail before steps due the repository account billing/spending-limit; Vercel passes. PR remains open/draft until CI can actually execute.

@xingxuling

Copy link
Copy Markdown
Owner Author

三轮重复基准已完成(commit 7501bef)。\n\nWorkload: arithmetic-chain-24\ninputRoot: b6a5b9512d04d884eae20d61c543255252c377c79307261ee204de714f9a7363\n所有 RCL/rustc runs: PASS,correctness=1。\n\n三轮平均:\n- RCL compileBuildSpeed: 7.294 ms;Rust: 547.248 ms\n- RCL runtime: 53.736 ms;Rust: 95.596 ms\n- RCL artifact: 2948 bytes;Rust: 147968 bytes\n- RCL/Rust ratios: compile 0.0133x,runtime 0.5621x,artifact 0.0199x\n\nRCL RBC 三轮 SHA-256 均为 CBF8B4B...313688E,说明该产物在重复构建中稳定;rustc EXE 每轮哈希不同,但输出和指标均通过。\n\n汇总证据:\noutput/benchmark-runs/2026-08-16/benchmark-summary.json\nsummaryRoot: 05bf033c50bc5a7ffa5123dabf55d44d0e1a13b74d4a0147977b8ea720c5bd71\n\n边界仍然是固定 arithmetic-chain 微基准;不等同于 whole-language、生态、内存或商业产品优势。

@xingxuling

Copy link
Copy Markdown
Owner Author

Cross-language benchmark update (2026-08-16)\n\nSource revision: 196f718\nWorkload: �rithmetic-chain-24; identical input root �6a5b9512d04d884eae20d61c543255252c377c79307261ee204de714f9a7363; 3 repetitions; all correctness checks PASS.\n\n| Provider | Build/prep mean | Runtime mean | Artifact |\n| --- | ---: | ---: | ---: |\n| RCL native VM | 5.687 ms | 49.330 ms | 2,948 B |\n| rustc 1.97.1 | 510.740 ms | 88.401 ms | 147,968 B |\n| CPython 3.11.6 | 228.922 ms* | 181.773 ms | 1,204 B |\n\nRCL is faster than both references on this workload. RCL loses CPython on artifact footprint (2.4485x), so the non-compensatory overall dominance result is FAIL, not a whole-language superiority claim. * CPython build/prep is py_compile preparation, not machine-code compilation.\n\nEvidence: output/benchmark-runs/2026-08-16/cross-language/benchmark-summary.json; summaryRoot

@xingxuling

Copy link
Copy Markdown
Owner Author

Multi-language workload matrix update\n\nSource revision: 3588c3c\nCommand:

pm run evidence:dominance-arena:matrix\nWorkloads: 4; providers: RCL native VM, rustc 1.97.1, CPython 3.11.6; 12/12 provider executions passed correctness.\n\n| Workload | RCL vs rustc | RCL vs CPython |\n| --- | --- | --- |\n| text normalization | PASS | FAIL: artifact 622 B vs 532 B |\n| sequence aggregation | PASS | FAIL: artifact 1,091 B vs 374 B |\n| UTF-8 byte length | PASS | PASS |\n| JSON-shaped text scan | PASS | FAIL: artifact 735 B vs 712 B |\n\nThe matrix uses per-workload non-compensatory raw comparisons. RCL therefore has 4/4 against rustc and 1/4 against CPython; overall matrix status is FAIL, not a whole-language superiority claim. Build/prep and cold runtime were wins in the listed comparisons, but CPython wins artifact footprint on three workloads.\n\nSeparate capability gaps remain explicitly BLOCKED: native JSON parsing, declared file I/O, and declared concurrency. The JSON-shaped workload is only textual scanning and does not count as parser parity.\n\nEvidence: output/dominance-arena/compiler-workload-matrix-v0.1/compiler-workload-matrix-report.json; reportRoot 18c821046f51279c52c6abe4c71ca1c09dc09b9a68b2f47918e2cd491d293921. Focused matrix regression: 1/1 PASS. CI remains failing on the current SHA, so no merge was performed.

@xingxuling

Copy link
Copy Markdown
Owner Author

Native JSON compact slice

Implemented and verified the first real RCL JSON parse-and-compact path.

  • Added json_compact(Text) -> Text to the JS runtime, type system, bytecode ABI (JSON_COMPACT=71) and Native VM.
  • Native parser validates strings, escapes, literals, numbers, arrays, objects, nesting and trailing data; it removes structural whitespace while preserving object key order and number lexemes.
  • Added a Windows Native output-capture fallback using the Windows temporary-file API because MinGW tmpfile() returned Permission denied in this environment.
  • Added a scoped JSON workload comparing real RCL Native VM and CPython; Rust is explicitly excluded because this repository has no matching standard-library parser reference.
  • Rebuilt and recorded the Windows Native artifacts with HermesUbuntu MinGW; npm run build:native returns PREBUILT_NATIVE_ARTIFACTS_VERIFIED.

Evidence

  • Commit: 40ae1524ec5ef47f49cfd1369d4ba4cff54eca08
  • Matrix source revision: 40ae1524ec5ef47f49cfd1369d4ba4cff54eca08
  • Matrix report root: 5c35b68abda4897c683643acee56e41f9bdcf5d5f1ee9df1de82d5b1d6e3a23
  • 5 workloads, 14 provider executions, 14/14 correctness passes.
  • RCL vs Rust: 4/4 raw comparisons PASS.
  • RCL vs CPython: 2/5 raw comparisons PASS; 3 failures remain emitted-artifact-footprint losses, so matrix status remains FAIL.
  • JSON compact workload: RCL and CPython PASS; file I/O and concurrency remain explicit BLOCKED gaps.

Local verification

  • node --test --test-concurrency=1 tests/language.test.mjs: 73/73 PASS
  • npm run test:dominance-arena: 11/11 PASS
  • npm run test:dominance-arena:microbench: 3/3 PASS
  • npm run test:dominance-arena:matrix: 1/1 PASS
  • npm run build:native: PASS

GitHub Actions currently reports verify-authority, verify, and focused-verification as failed, but the run API exposes no job steps and --log-failed returns log not found; no merge is requested or performed until an actionable CI failure is available.

@xingxuling

Copy link
Copy Markdown
Owner Author

修复已推送:77d9a5f (fix: close native selfhost and Windows verification gaps)

本地 Windows 验证:

  • npm test: 809 tests, 808 pass, 0 fail, 1 skipped
  • verify:selfhost-examples: artifactParity=true, eligible=22, unsupported=36, failure=0
  • verify:selfhost-fixedpoint: 9/9 pass
  • verify:version-contract: pass
  • verify:native-boundary: NATIVE_WINDOWS_VERIFIED
  • Frontier independent-file suite: 5/5 pass, repeated twice
  • 工作树已清洁

远端 CI 当前不是代码失败:RCL Canonical Verification、RCL Authority Contract、RCL Universal Program Stress 三个 job 都在 runner 启动前被 GitHub 账户 Billing & plans / spending limit 阻断,annotation 原文为:
"The job was not started because recent account payments have failed or your spending limit needs to be increased."

因此本 PR 暂不宣称 CI 通过,也不执行合并。请账户侧修复 Billing & plans 后重新运行必需检查,再进行最终差异审阅和合并。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant