[Docs] Add self-hosted inference provider guides

apps/docs/docs.json

@@ -43,7 +43,12 @@
           {
             "group": "Models and Inference",
             "root": "models",
-            "pages": []
+            "expanded": true,
+            "pages": [
+              "providers/inference/litellm",
+              "providers/inference/ollama",
+              "providers/inference/vllm"
+            ]
           },
           {
             "group": "Communications",

apps/docs/models.mdx

@@ -17,7 +17,13 @@ Configure models from **Settings > Models**.
 An inference provider is the service that hosts or routes model calls. Roomote
 supports providers such as OpenRouter, Vercel AI Gateway, Baseten,
 Together AI, OpenAI, Anthropic, Moonshot AI, MiniMax, OpenCode, Amazon Bedrock,
-Google Gemini, xAI, and ChatGPT subscriptions.
+Google Gemini, xAI, ChatGPT subscriptions, and self-hosted OpenAI-compatible
+endpoints.
+
+For self-hosted inference, see [Ollama](/providers/inference/ollama),
+[vLLM](/providers/inference/vllm), and [LiteLLM](/providers/inference/litellm).
+The Roomote services and the sandbox that runs a Roomote task must both be able
+to reach the endpoint when the model provider is used for task work.
 
 You can connect more than one inference provider in the same deployment. That
 lets you mix and match models by provider instead of betting the whole

apps/docs/providers/inference/litellm.mdx

@@ -0,0 +1,63 @@
+---
+title: LiteLLM
+icon: waypoints
+description: Route Roomote model traffic through a LiteLLM Proxy for central credentials, policy, and cost controls.
+---
+
+LiteLLM Proxy provides one OpenAI-compatible endpoint in front of multiple
+model providers. Use it when you want Roomote to use centrally managed provider
+credentials, model aliases, fallback policies, budget controls, or usage
+reporting without giving every Roomote deployment direct provider keys.
+
+## Configure LiteLLM
+
+1. deploy LiteLLM Proxy on a private network or behind an authenticated TLS reverse proxy
+2. define the models, aliases, provider credentials, and any budget policies in LiteLLM
+3. create a scoped virtual key for Roomote
+4. add an OpenAI-compatible inference provider in **Settings > Models**
+5. enter the LiteLLM Proxy base URL, typically ending in `/v1`, and the virtual key
+6. add the LiteLLM model alias or model ID that Roomote should use
+
+Use a dedicated LiteLLM virtual key for Roomote rather than a provider root key.
+Give it access only to the models Roomote needs and rotate it through your
+secret manager when team membership or deployment access changes.
+
+## Keys and model names
+
+LiteLLM's proxy key is the credential Roomote sends to the proxy; it is not the
+underlying OpenAI, Anthropic, or other provider key. Store the proxy key as a
+deployment secret or save it through **Settings > Models**. Do not put it in a
+task environment or repository configuration.
+
+The model ID configured in Roomote must be a model or alias accepted by your
+LiteLLM Proxy. Aliases are often useful because you can change the underlying
+provider or model in LiteLLM without changing Roomote's model-role mappings.
+
+## Cost controls
+
+LiteLLM can apply spend limits, rate limits, model access rules, and fallback
+policies before a request reaches an upstream provider. Set those controls in
+LiteLLM and keep Roomote's enabled model list small enough that users can make
+intentional choices.
+
+Start with explicit per-key or team budgets, model allowlists, and rate limits.
+Then use LiteLLM usage reporting alongside Roomote [Cost Analytics](/cost-analytics)
+to compare task outcomes with upstream cost. Configure fallbacks carefully:
+they improve availability, but a fallback can change capability, latency, data
+handling, or price.
+
+## Reachability and Docker
+
+The Roomote deployment and task workers must be able to reach the LiteLLM
+endpoint. In Docker, `localhost` is container-local, so use a shared private
+network, service DNS name, host-gateway address, or internal reverse proxy
+instead of a host-only loopback URL. Test the configured URL and scoped key from
+a Roomote service container and a task worker.
+
+## Security
+
+- keep the proxy endpoint private or require TLS and authentication
+- use a unique, least-privilege virtual key for Roomote
+- store upstream provider credentials only in LiteLLM's secret management, not in task environments
+- restrict which model IDs and providers the Roomote key can invoke
+- review LiteLLM logs and usage data as potentially sensitive because they can include request metadata

apps/docs/providers/inference/ollama.mdx

@@ -0,0 +1,71 @@
+---
+title: Ollama
+icon: server
+description: Run locally hosted Ollama models with Roomote while keeping model traffic in your own infrastructure.
+---
+
+Ollama is a practical option when you want to run supported models on your own
+machine or network. It is best suited to local development, trusted internal
+deployments, and workloads where the host has enough CPU, memory, or GPU
+capacity for the selected model.
+
+## Before you connect
+
+Roomote connects to Ollama over HTTP. The Roomote API and controller need a
+network route to the Ollama server, and task workers need that same route when
+they receive the model configuration. Test reachability from the actual
+Roomote deployment and from a task worker, not only from your laptop.
+
+For a self-hosted deployment, expose Ollama only on a private network or behind
+an authenticated reverse proxy. Do not publish an unauthenticated Ollama port
+directly to the internet.
+
+## Configure Ollama
+
+1. install Ollama and pull the model you plan to use
+2. make the Ollama API reachable from the Roomote deployment
+3. add Ollama in **Settings > Models** and enter its base URL
+4. add the installed model by its Ollama model name and enable it
+5. select it as the default coding model or assign it to a specialized role
+
+For example, pull a model on the host that runs Ollama:
+
+```sh
+ollama pull qwen3-coder
+```
+
+The exact model name in Roomote must match the model served by Ollama. Choose a
+model with sufficient context length and tool-use quality for coding tasks;
+small local models can be useful for helper work but may be less reliable as a
+default coding model.
+
+## Docker deployments
+
+`localhost` inside a Roomote container means that container, not the Docker
+host. When Ollama runs directly on the Docker host, configure a host-reachable
+address instead of `http://localhost:11434`.
+
+On Docker Desktop, `http://host.docker.internal:11434` usually reaches a host
+service. On Linux, add a host-gateway mapping to the Roomote Compose service or
+place Ollama and Roomote on a shared private Docker network, then use the host
+gateway address or the Ollama service name. Confirm that task worker containers
+can resolve and connect to the same address.
+
+If Ollama is bound only to `127.0.0.1`, containers cannot reach it. Bind it to a
+private interface that your Roomote services can reach, and restrict access with
+firewall rules or network policy.
+
+## Verify setup
+
+1. from the Roomote host, request the Ollama tags endpoint
+2. from a Roomote container or task worker, repeat the request against the same base URL
+3. start a small Roomote task with the Ollama model selected
+4. confirm the task starts and model usage appears in the task details
+
+## Security and capacity
+
+- keep Ollama on a private network; model endpoints can expose prompts and responses
+- restrict network access to Roomote services and trusted operators
+- use a reverse proxy with authentication when the endpoint crosses network boundaries
+- size the host for model memory, concurrent tasks, and context windows; queue or limit task concurrency before the host becomes memory-bound
+- review model licenses and data-handling requirements before serving a model to a team

apps/docs/providers/inference/vllm.mdx

@@ -0,0 +1,73 @@
+---
+title: vLLM
+icon: server
+description: Serve high-throughput OpenAI-compatible models with vLLM for Roomote deployments.
+---
+
+vLLM provides an OpenAI-compatible API for models you operate yourself. Use it
+when you need higher-throughput GPU inference, want to serve a supported model
+behind your own endpoint, or need more control over batching and GPU use than a
+desktop-focused local runner provides.
+
+## Before you connect
+
+Roomote must reach the vLLM endpoint from its services and from task workers
+that receive the model configuration. Use a stable private DNS name or service
+address rather than a laptop-only address, then test it from the same network
+as the deployment.
+
+Keep the vLLM API private. The OpenAI-compatible endpoint handles prompts and
+responses and can accept credentials, so it should sit behind private
+networking or an authenticated TLS reverse proxy.
+
+## Start an OpenAI-compatible server
+
+Start vLLM with the OpenAI-compatible API server and the model you want to
+serve. This example exposes the service on all interfaces for a private network:
+
+```sh
+vllm serve <model-id> \
+  --host 0.0.0.0 \
+  --port 8000 \
+  --api-key <long-random-secret> \
+  --max-model-len <context-tokens> \
+  --gpu-memory-utilization 0.9
+```
+
+Use `--api-key` whenever the service is reachable beyond a tightly controlled
+local network. Set `--max-model-len` to a value that fits your GPU memory and
+task context needs; an unnecessarily large value can reduce concurrency or
+prevent the model from loading. Adjust `--gpu-memory-utilization` conservatively
+when other workloads share the GPU.
+
+For tensor-parallel deployments, add `--tensor-parallel-size <gpu-count>`. For
+models that require an explicit trust decision, use `--trust-remote-code` only
+after reviewing the model repository and its revision. Do not enable it merely
+to make an unfamiliar model start.
+
+## Configure Roomote
+
+1. add an OpenAI-compatible inference provider in **Settings > Models**
+2. enter the vLLM base URL, typically ending in `/v1`
+3. provide the same API key configured for vLLM
+4. add the served model ID and enable it
+5. select it for the default coding model or a specialized role
+
+The model ID in Roomote must match the ID vLLM exposes. If you deploy a model
+under a custom served name, use that served name rather than assuming the
+repository name.
+
+## Docker deployments
+
+As with any host service, `localhost` from a Roomote container is not the
+Docker host. Run vLLM on a private shared network, use a host-gateway address,
+or route through an internal reverse proxy. Verify the URL and API key from a
+Roomote service container and a task worker before assigning the model to users.
+
+## Security and operations
+
+- terminate TLS and require authentication before traffic leaves a trusted network
+- restrict the endpoint to Roomote services and approved operators
+- pin model revisions and review remote model code before using `--trust-remote-code`
+- monitor GPU memory, queue depth, latency, and error rates as task concurrency grows
+- set request and concurrency limits at the proxy or vLLM layer to protect other workloads on the GPU host