Skip to content

fix(coding-agent): make rlm.find_models represent every provider and ignore word order - #855

Open
sashankh wants to merge 1 commit into
PrimeIntellect-ai:mainfrom
sashankh:fix/799-find-models-coverage
Open

fix(coding-agent): make rlm.find_models represent every provider and ignore word order#855
sashankh wants to merge 1 commit into
PrimeIntellect-ai:mainfrom
sashankh:fix/799-find-models-coverage

Conversation

@sashankh

@sashankh sashankh commented Aug 7, 2026

Copy link
Copy Markdown

Summary

With no query, findRlmModelMatches scored every candidate 0, so ordering fell through to an alphabetical tiebreak on the selector before .slice(0, limit). With 155 models across 8 providers, a bare rlm.find_models() returned 8 results from a single provider, and nothing in the return value marked it as a page.

The failure mode is confident wrongness rather than an error. An orienting agent calls this bare as its first move before delegating, and can reasonably conclude the providers it cannot see are unreachable. Issue #799 records exactly that: an orchestrator routed three delegations through shell-CLI fallbacks that were unnecessary.

While in the same matcher, this also fixes the token-order half of #810: the query was normalized into a single string, so "5.6 gpt sol" returned [] while "gpt 5.6 sol" matched. The openai-codex reachability half of #810 is out of scope here.

Changes

  • Round-robin ranked candidates by provider when the query is empty, so a bounded page reaches every provider instead of the alphabetical head of one.
  • Return total, truncated, and per-provider match counts for the whole match set, so a page cannot be mistaken for the full catalog.
  • Score a multi-word query token-wise as a final tier, keeping exact, prefix, and partial ranking ahead of it, so word order no longer decides whether a model matches.
  • Surface the page summary to Python as RLMModelPage, a list subclass, so existing code that treats the result as a list keeps working while total / truncated / providers are available.

Non-empty-query ranking is otherwise unchanged.

Tests

New regression packages/coding-agent/test/suite/regressions/799-find-models-provider-coverage.test.ts covers an unqualified page sampling every provider and reporting what it was cut from, a reordered query matching the same models as the original, and query ranking still taking precedence over provider coverage.

$ npx tsx ../../node_modules/vitest/dist/cli.js --run \
    test/suite/regressions/799-find-models-provider-coverage.test.ts \
    test/suite/regressions/4649-subagent-model-selection.test.ts
Test Files  2 passed (2)
$ npm run check
Checked 899 files. No fixes applied.
Installer render check passed.
$ cd prime-agent-runtime && uv run python -m unittest discover -s test -p 'test_subagent_registry.py'
Ran 11 tests - OK

Two pre-existing ModuleNotFoundError: No module named 'mcp' errors in the Python runtime suite, and 4 pre-existing failures in test/resource-loader.test.ts, reproduce identically on unmodified origin/main and are unrelated to this change.

closes #799
refs #810

Note

Fix rlm.find_models to cover every provider and match query tokens in any order

  • findRlmModelMatches adds an order-independent token-matching tier after exact/prefix/partial ranking, so queries like gpt 4o and 4o gpt return the same results.
  • Unqualified (empty) queries now interleave results across providers via round-robin rather than sorting alphabetically, ensuring every provider appears in the page.
  • find_models returns RLMModelPage (a list subclass) with total, truncated, and providers metadata instead of a plain list, in both Python (prime-agent-runtime) and TypeScript (rlm-runtime.ts).
  • The runtime readiness check in bootstrap.ts now requires RLMModelPage to be importable from rlm.
  • Behavioral Change: callers of find_models / findRlmModels that expected a plain list or { models } object now receive a richer result type with page metadata fields.

Macroscope summarized cca59fd.

…ignore word order

With no query every candidate scored 0, so ordering fell through to an alphabetical
tiebreak on the selector before truncating to `limit`. With 155 models across 8
providers a bare `rlm.find_models()` returned 8 results from a single provider and
nothing in the response marked it as a page, so an orienting agent could reasonably
conclude the other providers were unreachable and route delegations through
unnecessary shell-CLI fallbacks.

Model matching also normalized the whole query into one string, so a reordered query
such as "5.6 gpt sol" matched nothing while "gpt 5.6 sol" matched.

- Round-robin ranked candidates by provider when the query is empty, so a bounded
  page reaches every provider instead of the alphabetical head of one
- Return `total`, `truncated`, and per-provider match counts for the whole match set
  so a page cannot be mistaken for the full catalog
- Score a multi-word query token-wise as a final tier, keeping exact, prefix, and
  partial ranking ahead of it, so word order no longer decides whether a model matches
- Surface the page summary through the kernel bridge as `RLMModelPage`, a list
  subclass, so existing code that treats the result as a list is unaffected

closes PrimeIntellect-ai#799
refs PrimeIntellect-ai#810
zhengr pushed a commit to zhengr/prime-agent that referenced this pull request Aug 8, 2026
…rimeIntellect-ai#855)

Add markdown.codeBlockIndent setting to customize indentation prefix for
rendered code blocks. Default remains 2 spaces for visual clarity, but
setting to empty string removes indentation for easier copy/paste of
code snippets to scripts, editors, or other tools.

Changes:
- tui: add optional codeBlockIndent to MarkdownTheme interface
- coding-agent: add MarkdownSettings with codeBlockIndent property
- coding-agent: compose theme with settings at call sites (no global state)
- coding-agent: update message components to accept optional MarkdownTheme

Co-authored-by: Mario Zechner <badlogicgames@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

rlm.find_models() returns an alphabetical head slice that reads as the full reachable set

1 participant