Proof, not promises. This repo continuously audits whether the javis.bot relay serves the real Claude / GPT models it advertises — not quantized, downgraded, or substituted ones.
🔍 Live dashboard: https://javisbot-service.github.io/javis-transparency/ · 🌐 Get access: javis.bot · 🇹🇼🇨🇳 中文: README.zh.md
Everything behind javis.bot's model-substitution ("掉包") audit is public here — the code, the runs, and the results. The goal: turn "is this model real?" from "trust me" into "check it yourself, and let third parties check it too."
| You might doubt… | …and here's how you check it yourself |
|---|---|
| "Is this really the code that runs?" | audit/probe_public.py in this repo, with full commit history. |
| "Did it actually run?" | Actions run logs — the raw output of every scheduled run is public. |
| "Were the results hand-edited?" | Results are committed by github-actions[bot] (not the operator's account); git's hash-chained history is tamper-evident and any force-push is visible. |
| "Is the page a separate fake?" | The page is generated by GitHub Pages straight from this repo's docs/ — it never touches Javis servers. |
The audit does not run on Javis's servers — it runs public code on GitHub's machines. That separation is the core of the trust model.
- Cai, Shi, Zhao, Song — Are You Getting What You Pay For? Auditing Model Substitution in LLM APIs — arXiv:2504.04715
- Zhang et al. (CISPA) — Real Money, Fake Models: Deceptive Model Claims in Shadow APIs — arXiv:2603.01919 — found 45.83% of shadow APIs failed model-identity verification
- Zhu et al. — Auditing Black-Box LLM APIs with a Rank-Based Uniformity Test — arXiv:2506.06975
- Lin et al. — Behavioral Consistency and Transparency Analysis on LLM API Gateways (IMC'26) — arXiv:2604.21083
The dashboard lists deterministic probes (temperature=0) and a copy-paste curl. Run it against https://api.javis.bot with your own key and compare: do the completions match, and does the response model field report the advertised model? The endpoint the audit hits is the same one you call day to day.
This audit is probabilistic consistency detection, not cryptographic proof. In theory a provider could return authentic responses to known audit sources while substituting on real traffic (see arXiv:2506.06975) — a limitation shared by all black-box auditing. We mitigate it with "you can reproduce it."
audit/probe_public.py Pure-stdlib probe: probe → judge → write results (no secrets, no internal info)
audit/probes.json Public probe set
audit/config.json Public config
audit/baselines/ Versioned reference baselines (from capture-baseline)
docs/index.html Transparency page (hosted on GitHub Pages)
docs/data/history.json Audit history (committed by the bot)
.github/workflows/audit.yml Every-6h schedule + manual trigger
License: MIT.