Rank tiers by capability, verify every rung is callable, route scout and vision

[atyrode]

Driven by Opus 5 shipping. The id swap was the smallest part.

1. Price stopped predicting capability

pickLadder ranked by input cost. Against today's live catalog that produced:

plan/slow/designer/reviewer  claude-opus-4-1:high   ($15, 200k, xhigh)
                  fallback   claude-opus-5:high     ($5,  1M,   max)

The smart rung led by a fossil its own fallback dominates on every axis, capped a thinking rung below what the model behind it offers. Providers now reprice successors below predecessors they never delist, so cost inverts capability.

Now: supersession first (newest per family, version components compared numerically so a future 5.10 beats 5.9), then ranked by thinking ceiling → context → price. Price is the last word, not the first. loadCatalog asserts the result: a dearer rung offering less context or less thinking headroom is rejected by name.

2. A model your account cannot call could lead the smart rung

This is the part worth reviewing closely.

omp models --json lists claude-mythos-5. It 404s on this account. It reports claude-fable-5's exact price, context window, thinking range, modalities and maxTokens — I diffed every field; only id/name/selector differ. Nothing in the metadata says which one you can call.

The only thing keeping it off the ladder was a shorter-id tiebreak. Rename it claude-myth-5 and it wins the smart rung, and every profile leading with it fails at launch.

So reachability is no longer a flag. init probes every candidate; only a model that answers may become a rung. The file is stamped probed: true and generate refuses one that is not — that marker is the gate, so --from-json can still scaffold offline (stamped false) without an unverified scaffold quietly becoming live routing. --bench is gone: measurement is now a side effect of a probe that always runs, which also retires the placeholder 50/2.0 pair that made the speed meter identical for every model.

The probe found a false positive that mattered. claude-fable-5 failed — not unreachable, but refusing omp's bundled bench prompt:

Refusal (cyber): This request triggered restrictions on violative cyber
content and was blocked under Anthropic's Usage Policy.

A rule of "any failure means unreachable" would have silently deleted a perfectly callable elite from the ladder. omp bench is not a reachability oracle on its default prompt. So the probe asks something innocuous (--prompt), and sorts three ways:

outcome	evidence	action
reachable	answered, zero failures, non-nil average	eligible, measured
not found	provider says it does not exist	dropped
unresolved	refusal, rate limit, missing row	nothing inferred

Unresolved is a refusal to certify, not a quiet drop: init names each model and its reason and writes nothing. A probed: true file built from a partial probe is worse than no file.

3. The elite was crowned as the everyday smart rung

Tier 0/4 came only from omp's tier-scoped quota buckets. That report turns out to depend on the ambient auth environment — the anthropic:fable bucket is visible from one environment on this machine and absent from another, minutes apart, deterministically (3/3 each way). Root cause: OMP_AUTH_BROKER_TOKEN/_URL present or not.

When absent, the elite didn't merely go undetected — it fell onto the ordinary ladder and became tier 3, so every routine "smart" request would drain the scarce $10 bucket that exists to be spent deliberately. Observed live:

A: haiku(1)  opus-5(2)  fable-5(3)     <- no tier 4, sonnet-5 gone

Price now confirms what the bucket used to assert alone: a pool's top model priced at twice the next is its elite, bucket or no bucket. Price says nothing about entitlement — that is the probe's job — but a model in its own price class is not the everyday workhorse.

4. scout and vision

omp bundles six agents; the grid routed five. scout — the one omp's own prompt marks "MUST be used for exploratory codebase research" — silently inherited @smol, invisible in the preview and unselectable. Now agent-backed, so genConfigYAML mirrors it into task.agentModelOverrides. vision is one of omp's ten real role ids (confirmed against omp's docs) and backs image description when the active model is text-only; it always leads on an image-capable model, since the codex spark variants are not.

Both needed roleWeight entries — weightedModels silently skips a role it cannot weigh, dropping it from both meters with no trace. A test asserts every emitted role is weighted.

Routing scout also desynced the catalog header, which carried a hand-typed copy of the agent list beside genAgentRoles: the grid marked 414 ● scout rows while the legend named five agents and denied scout existed. The legend is now derived from genAgentRoles, and a test asserts it names exactly the roles the rows mark — nothing covered that line before, which is how it went stale unnoticed.

Enter no longer launches a combo the catalog lacks. It walked a nil block and handed omp an empty modelRoles: map — reproducible on a stock scaffold, where shipped spark: on pointed at a combo no tier-0-less catalog contained. Dials are now derived from the catalog: a toggle with no combos is forced off and hidden.

Result

O: spark(0)  mini(1)  terra(2)  sol(3)
A: haiku(1)  sonnet-5(2)  opus-5(3)  fable-5(4)

The hand-curated dotfiles ladder, derived automatically. 414 combos (was 91, none with spark or fable).

Verification

• go build, go vet, gofmt, go test ./... clean — 9.4s, down from 564s: the suite had been making real API calls through obScan. Now fully hermetic.
• Live code generate init: 21 models probed in ~47s → mythos dropped, fable-5 retained and correctly tier 4, probed: true written.
• New binary against the dotfiles curated catalog: 415 combos, 8 models. Delta vs the deployed catalog is exactly 414 x 2 lines — the new scout and vision rows, nothing else.
• The gate rejects an unattested copy of that same file with an actionable message.
• Regression tests for each finding, including TestScaffoldRefusalSurfacesNotDropped — a refusal must surface as unresolved naming the model, while a genuine 404 is dropped and must not appear in that list. It fails under the previous two-state rule.
• Rebased onto main after code ls/session reap (feat(session): add code ls and code session reap #36) and the v0.4.8 pin (chore(nix): pin code v0.4.8 #38) landed — clean, no conflicts. Re-verified on the combined tree: build/vet/fmt/tests green, code ls works, and the generator still renders the dotfiles catalog to 415 combos.

Not in this PR

No release tag — per AGENTS.md a v* tag is a deployment, so that waits for review. Companion: atyrode/dotfiles#341.

One loose end, evidenced but unfixed: retry.fallbackChains keyed by agent name never fire. Verified by forcing a retry — a sonic: chain specifying haiku was ignored and the agent landed on sonnet-5 instead, because reviewer/librarian/sonic/scout are agent names, not role ids. Role-keyed chains do work (a default: chain fired exactly as written). That affects defaults.yml, not this repo, and keying by model selector shadows the working designer: chain, so it needs a decision rather than a patch.