Skip to content

[Docs] Add self-hosted inference provider guides#480

Draft
roomote-roomote[bot] wants to merge 1 commit into
developfrom
feature/self-hosted-inference-docs-39c6dsctvve43
Draft

[Docs] Add self-hosted inference provider guides#480
roomote-roomote[bot] wants to merge 1 commit into
developfrom
feature/self-hosted-inference-docs-39c6dsctvve43

Conversation

@roomote-roomote

Copy link
Copy Markdown
Contributor

Opened on behalf of Bruno Bergher. Follow up by mentioning @roomote-roomote, in the web UI, or in Slack.

Related issue

No linked issue.

Why this PR exists

  • I am a maintainer / this is internal Roomote work

What changed

Added public inference-provider documentation for Ollama, vLLM, and LiteLLM.
The guides cover self-hosted endpoint reachability, Docker host networking,
vLLM serving and security flags, LiteLLM proxy keys and cost controls, and
provider-specific security practices. The Models and Inference navigation now
links to each guide.

How it was tested

Ran mise exec -- pnpm --filter @roomote/docs check successfully.

Checklist

  • The PR title follows the repo convention: [Fix], [Feat], [Improve], [Refactor], [Docs], or [Chore] followed by a user-facing description
  • This PR is small and scoped to one change
  • pnpm lint and pnpm check-types pass locally
  • I added tests or included a clear manual validation note above
  • I removed secrets, tokens, private keys, and customer data from code, logs, and screenshots
  • If this change should appear in the changelog, I ran pnpm changeset

@roomote-roomote

roomote-roomote Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor Author

1 issue outstanding. See task

  • apps/docs/providers/inference/litellm.mdx:17-19 (also ollama.mdx:25-29 and vllm.mdx:50-54): the guides direct operators to add Ollama or an OpenAI-compatible provider and enter a base URL in Settings > Models, but that UI only exposes SETUP_MODEL_PROVIDER_CATALOG; it has no Ollama/generic provider entry and no endpoint field. These setup flows therefore cannot be completed on the documented product. Add the required custom-provider support or document a configuration path that exists in the current release.

Reviewed 612a385

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant