From a0c11cc2650785f21e4d010e28b7075fe03b97cc Mon Sep 17 00:00:00 2001 From: openhands Date: Tue, 4 Aug 2026 12:28:50 -0300 Subject: [PATCH 1/2] Document Agent Canvas Model Router --- docs.json | 1 + openhands/usage/agent-canvas/llm-profiles.mdx | 3 + openhands/usage/agent-canvas/model-router.mdx | 72 +++++++++++++++++++ 3 files changed, 76 insertions(+) create mode 100644 openhands/usage/agent-canvas/model-router.mdx diff --git a/docs.json b/docs.json index d9a0a4ab9..7faadb88a 100644 --- a/docs.json +++ b/docs.json @@ -262,6 +262,7 @@ "openhands/usage/agent-canvas/setup", "openhands/usage/agent-canvas/first-time-setup", "openhands/usage/agent-canvas/llm-profiles", + "openhands/usage/agent-canvas/model-router", "openhands/usage/agent-canvas/conversations", "openhands/usage/agent-canvas/agent-profiles", "openhands/usage/agent-canvas/plugins", diff --git a/openhands/usage/agent-canvas/llm-profiles.mdx b/openhands/usage/agent-canvas/llm-profiles.mdx index 7fb327c9c..99f3fb738 100644 --- a/openhands/usage/agent-canvas/llm-profiles.mdx +++ b/openhands/usage/agent-canvas/llm-profiles.mdx @@ -70,6 +70,8 @@ LLM profiles are useful when you want different model setups for different tasks LLM profiles are separate from [Agent Profiles](/openhands/usage/agent-canvas/agent-profiles). Agent Profiles choose which agent runs a new conversation. OpenHands Agent Profiles reference an LLM profile to decide which model configuration that agent uses. +LLM profiles are also used by [Model Router](/openhands/usage/agent-canvas/model-router). A router classifies a task, then switches the conversation to the saved profile whose name matches the selected model. + ### Manage Saved Profiles The available profiles list shows each profile's name, configured model, and whether it is active. Use a profile's menu to edit or rename it, set it as the active profile for new conversations, or delete it when you no longer need it. @@ -113,4 +115,5 @@ For error-specific steps, see [Troubleshooting](/openhands/usage/agent-canvas/tr - [Setup](/openhands/usage/agent-canvas/setup) - [Customize and Settings](/openhands/usage/agent-canvas/customize-and-settings) - [Agent Profiles](/openhands/usage/agent-canvas/agent-profiles) +- [Model Router](/openhands/usage/agent-canvas/model-router) - [LLM Settings](/openhands/usage/settings/llm-settings) diff --git a/openhands/usage/agent-canvas/model-router.mdx b/openhands/usage/agent-canvas/model-router.mdx new file mode 100644 index 000000000..6bf805c1f --- /dev/null +++ b/openhands/usage/agent-canvas/model-router.mdx @@ -0,0 +1,72 @@ +--- +title: Model Router +description: Route each Agent Canvas task to a saved LLM profile using a classifier prompt. +--- + +Model Router lets an OpenHands agent choose an LLM profile for each task before the main work starts. This is useful when you have several saved models and want the agent to trade off quality, latency, and cost automatically. + +Model Router uses saved [LLM profiles](/openhands/usage/agent-canvas/llm-profiles). The classifier returns a model/profile name, and the backend switches the conversation to the matching saved profile. + + +Model Router works with OpenHands Agent Profiles. ACP agents such as Claude Code, Codex, and Gemini CLI manage model selection in their own runtime. + + +## Create a Model Router + +1. Open `Settings > Model Router`. +2. Select `Add Model Router`. +3. Choose one of the templates: + - `Add Default Pareto Max Score` — prefilled with the default Pareto prompt that favors the highest expected task score. + - `Add Default Pareto Min Cost` — prefilled with a placeholder min-cost Pareto prompt. + - `Custom` — starts with an empty router form. +4. Review the classifier model, default model, prompt template, and model table. +5. Save the router. + +If no router is active yet, the first router you create becomes active automatically for new OpenHands conversations. + +## Default Pareto templates + +The default Pareto templates use: + +- `minimax-m3` as the classifier profile +- `minimax-m3` as the fallback/default profile +- a model table with the target model names and benchmark score/cost data +- a direct prompt template that must include `{{ instance_text }}` and may include `{{ model_table }}` + +The Max Score template is ready to use. The Min Cost template is currently a placeholder so you can edit it while keeping the same model table and profile-generation workflow. + +## Create the target LLM profiles + +When you create a default Pareto router, Agent Canvas can create any missing target LLM profiles for you. Leave `Create missing LLM profiles for this router` checked to reuse your current provider settings for all router models. + +This copies the active LLM profile's provider settings and changes only the model identifier and usage id for each router target. It is the quickest way to try a router with an existing LiteLLM proxy or compatible provider configuration. + + +Make sure your provider or proxy actually serves the target model IDs. The generated profiles reuse credentials and base URL, but they do not verify that every model is available. + + +## Custom routers + +Choose `Custom` when you want to write your own router prompt or route between a smaller set of saved profiles. + +A custom router starts blank and leaves automatic target-profile creation unchecked. Fill in: + +- **Classifier model** — the saved LLM profile that classifies the task. +- **Default model** — the saved LLM profile used if routing fails or no task class matches. +- **Prompt template** — the prompt sent to the classifier. It must include `{{ instance_text }}`. +- **Model table** — optional text injected wherever the prompt uses `{{ model_table }}`. + +The classifier should return JSON with a `model` value that matches a saved LLM profile name. Matching is case-insensitive, so `GPT-5.4` and `gpt-5.4` resolve to the same profile name. + +## Use the router in a conversation + +After a router is active, start a new OpenHands conversation. The agent receives a routing tool and can call it before doing the task. Agent Canvas shows the selected profile in the conversation list and updates the model selector below the chat after a successful switch. + +You can still switch models manually with the chat model selector or the `/model` command. See [Manage LLM Profiles](/openhands/usage/agent-canvas/llm-profiles#switching-profiles-in-a-conversation). + +## Related guides + +- [Manage LLM Profiles](/openhands/usage/agent-canvas/llm-profiles) +- [Agent Profiles](/openhands/usage/agent-canvas/agent-profiles) +- [SDK Model Routing](/sdk/guides/llm-routing) +- [SDK LLM Profile Store](/sdk/guides/llm-profile-store) From 1b146a7d145fa3f15f73f701b93deca2f2164390 Mon Sep 17 00:00:00 2001 From: openhands Date: Tue, 4 Aug 2026 12:33:54 -0300 Subject: [PATCH 2/2] Add model router usage prompt example --- openhands/usage/agent-canvas/model-router.mdx | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/openhands/usage/agent-canvas/model-router.mdx b/openhands/usage/agent-canvas/model-router.mdx index 6bf805c1f..8b016c214 100644 --- a/openhands/usage/agent-canvas/model-router.mdx +++ b/openhands/usage/agent-canvas/model-router.mdx @@ -62,6 +62,14 @@ The classifier should return JSON with a `model` value that matches a saved LLM After a router is active, start a new OpenHands conversation. The agent receives a routing tool and can call it before doing the task. Agent Canvas shows the selected profile in the conversation list and updates the model selector below the chat after a successful switch. +You can ask the agent to call the router the same way you would ask it to use any other tool. For example: + +```text +hey classify and route the following task, then work on it: + +fix a tricky Python test failure in a large codebase +``` + You can still switch models manually with the chat model selector or the `/model` command. See [Manage LLM Profiles](/openhands/usage/agent-canvas/llm-profiles#switching-profiles-in-a-conversation). ## Related guides