Self-hosted RAG platform for AI document search across PDFs, DOCX, XLSX, local files, and web sources — written in Rust
English | 繁體中文 | 📚 Documentation
OpenDocuments was originally inspired by and built as a TypeScript / Node.js monorepo server using Hono and Turborepo. While that architecture served as an excellent proof of concept, we undertook a comprehensive ground-up rewrite in modern Rust to address critical technical debt and fulfill zero-trust, high-efficiency requirements:
- True Single-Binary Distribution: The complete Axum API router and React WebUI static assets are compiled directly into the binary memory using
rust-embed. Runs as a standalone file with zero external dependencies. - Deterministic Memory Footprint: Rather than spawning multiple heavy JS runtimes (each consuming 150MB+ overhead), the Rust runtime encapsulates all subsystems within a single, highly-optimized OS thread pool with microsecond-level scheduling.
- Rust-Native Embedded Storage: Metadata indexing via SQLite and vector similarity + full-text search via LanceDB are embedded natively into the binary process—eliminating any IPC crossing or slow C-binding bridges.
- Performance Gains: Text extraction, semantic chunking, and Reciprocal Rank Fusion (RRF) query planning perform 5x to 15x faster under the Rust-native execution graph, unlocking real-time responsiveness even on constrained homelab hardware.
OpenDocuments has been completely rewritten in Rust to clear technical debt and optimize for resource-constrained environments (like legacy government/school PCs).
Here is a quick comparison between the legacy TypeScript/Node.js implementation and the new Rust core, measured using hyperfine on a 10,000-row messy administrative Excel sheet:
| Metric | Legacy (Node.js) | Modern (Rust Core) | Improvement |
|---|---|---|---|
| Cold Start / Idle Memory | ~180 MB | ~18 MB | 90% RAM Saved |
| Parsing & Chunking Latency | ~14.25 seconds | 0.83 seconds | 17x Faster |
| Binary Size / Dependencies | Thick node_modules |
Single Binary (with WebUI embedded) | Zero External Dependency |
🔍 Click to view hyperfine benchmark command & output log
# Environment: AMD Ryzen 5 5600GT, 64GB RAM, Linux (CachyOS)
# Tool used: hyperfine --warmup 3
Benchmark 1: opendoc document index admin_heavy.xlsx
Time (mean ± σ): 827.0 ms ± 6.2 ms [User: 22.2 ms, System: 12.2 ms]
Range (min … max): 819.6 ms … 835.5 ms 10 runsOpenDocuments is an open-source, self-hosted RAG (Retrieval-Augmented Generation) platform that turns scattered documents into an AI-searchable knowledge base. It parses format-complex documents, indexes them with hybrid vector + keyword search, and answers natural-language questions with cited sources.
Use OpenDocuments when you want:
- A self-hosted alternative to enterprise AI search and proprietary knowledge-base search tools.
- AI document search with citations for PDFs, DOCX, XLSX, local files, and web sources.
- A local-first RAG stack that can run entirely with Ollama so sensitive documents stay on your own infrastructure.
- A knowledge base for AI coding assistants through MCP, including Claude Code, Cursor, Windsurf, and other MCP clients.
- A high-performance Rust-native core that compiles into a single binary, serving both the backend and embedded WebUI from memory.
Install with a single command and launch:
# Option 1: One-line install script (Pre-built binary)
curl -fsSL https://raw.githubusercontent.com/cawa0505/OpenDocuments/main/install.sh | sh
# Option 2: Install via Cargo directly from GitHub (Rust developers)
# Note: Requires system-installed 'protoc' (protobuf compiler). We use RUSTC_BOOTSTRAP=1 to raise the compiler's recursion limit for the heavy Lance/Arrow dependencies, and force the rustix libc backend to avoid nightly attribute errors.
RUSTC_BOOTSTRAP=1 RUSTFLAGS="-Z min-recursion-limit=512 --cfg=rustix_use_libc" cargo install --git https://github.com/cawa0505/OpenDocuments opendoc --force
# Start OpenDocuments
opendoc start --port 3000Open http://localhost:3000, index your documents, and ask questions with source citations.
All technical specifications and architecture maps are located in docs/en/ and openspec/:
- 🗂️ System Architecture & Directory Map: Single-binary Rust workspace layout, BYOK secret boundaries, and dual-retrieval data flow.
- 🗺️ R&D Roadmap & Milestones: Multi-phase engineering roadmap from single-binary MVP to Tauri 2.0 control cabin and MCP publisher.
- ⚡ Open Source RAG Engine Spec: Complete Data Plane technical documentation for LanceDB integration and SQL-like pre-filtering.
- 📐 OpenSpec Specifications: Formal system behavior contracts (Tags, BYOK, Workspace resolution).
- 📖 Official Documentation Site: Hosted guides, API references, and deployment tutorials.
OpenDocuments acts as a Token-Efficient RAG Gateway designed to connect private document knowledge with frontier AI models and LLM providers.
By combining LanceDB dense vectors, LanceDB full-text sparse keyword search, and Reciprocal Rank Fusion (RRF) reranking, OpenDocuments filters out irrelevant content before constructing the prompt context. This reduces token overhead by up to 70%+, ensuring high-precision context delivery to API endpoints like Claude 3.7 Sonnet, GPT-4o/o3-mini, Google Gemini 1.5 Pro, Grok 3, and Ollama.
- BYOK (Bring Your Own Key): API keys are encrypted and stored in a local SQLite table (
600permission) with zero telemetry, never leaking to frontend or remote servers. - OpenAI & Anthropic Compatible: Progressive SSE streaming and unified model router out-of-the-box.
- Model Context Protocol (MCP): Acts as a standard MCP server over Stdio/IPC, allowing developer tools like Claude Code, Cursor, and Windsurf to securely search local documents.
We actively welcome AI model vendors, API aggregators, and Cloud infrastructure providers (such as Anthropic, OpenAI, Groq, Together AI, Google Cloud, and AWS) to collaborate through API Grants / Test Credits. Grants directly support:
- Continuous CI/CD automated benchmarking of new LLM capabilities and prompt alignment.
- Testing context recall accuracy across multi-modal and long-context models.
- Maintaining the 100% open-source, vendor-neutral core for developers worldwide.
| Feature | What it means |
|---|---|
| Self-hosted RAG | Run the full document search stack on your own secure infrastructure. |
| Cited AI answers | Ask natural-language questions and see exactly which documents support the answer. |
| Hybrid retrieval | Combine dense vector search, LanceDB full-text keyword search, reranking, and parent-document recall. |
| Single-Binary Package | Axum backend and React WebUI are packaged into a single binary via rust-embed. Zero external asset requirements or port collision. |
| Broad file formats | Native support for Markdown, PDF, DOCX, XLSX, CSV, HTML, and code. |
| Local or cloud models | Use Ollama locally or cloud providers such as OpenAI, Anthropic, Google, and xAI. |
| MCP server | Let Claude Code, Cursor, Windsurf, and other MCP clients search your internal knowledge base. |
| Workspace isolation | Role-based workspace and collection logical isolation for secure multi-context data boundaries. |
OpenDocuments is designed as a modular Rust Cargo Workspace:
apps/
webui/ - React SPA (Vite + Tailwind CSS) frontend
crates/
opendoc-cli - Main CLI and terminal interface (opendoc)
opendoc-mcp - Axum API server, SSE streaming, and MCP protocol core
opendoc-storage - SQLite metadata and LanceDB vector mixed retrieval store
opendoc-llm - OpenAI-compatible LLM client and progressively-parsed streaming
opendoc-types - Shared strong types (DocumentChunk, Tag, etc.)
opendoc-parser-* - Standalone sandboxed document format parsers (PDF, DOCX, XLSX, etc.)
OpenDocuments is configured via a standard TOML file located at ~/.config/opendocuments/config.toml.
The configuration is automatically initialized with default values the first time you run opendoc.
[server]
url = "http://127.0.0.1:3000"
[database]
path = "~/.opendocuments" # Base directory for database files
[model]
default_workspace = "default" # Default workspace created on system startup
active_workspace = "MyWorkspace" # Active workspace
score_threshold = 0.60 # RAG retrieval similarity cutoff threshold
local_reranker_path = "~/.opendocuments/models/bge-reranker-base.onnx"This is the fastest way to run a local AI document search engine with the OpenDocuments CLI.
Option A: One-line Install (Recommended)
Download and install the pre-compiled single binary (Linux / macOS):
curl -fsSL https://raw.githubusercontent.com/cawa0505/OpenDocuments/main/install.sh | shOption B: Build from Source
# Clone the repository and install the unified binary to ~/.cargo/bin/opendoc
make installopendoc start --port 3000Open http://localhost:3000 to access the Web UI and start indexing!
You can also use the CLI to directly query and index local documents:
# Switch to a specific workspace
opendoc workspace switch "MyWorkspace"
# Index local files/folders
opendoc document index /path/to/docs
# Quick CLI query
opendoc ask "How does our auth system work?"OpenDocuments is 100% open-source, vendor-neutral, and community-driven. If OpenDocuments saves you hardware costs, protects your document privacy, or streamlines your daily administrative workflow, consider supporting its ongoing development:
- Solana (SOL): pay in wallet — one tap opens your wallet app
4pb8p2cTHdQb9WmU68n6AtQ3rrEHEzkQoESAXADzwKSF
- Core Infrastructure: Maintaining zero-dependency, ultra-fast single binary builds across Linux, macOS, and Windows.
- Local Model Optimization: Enhancing embedded ONNX / WASM local reranking and vector quantization for constrained devices.
- Open-Core Guarantee: Ensuring core RAG and MCP server capabilities remain 100% free and open-source forever.
To maintain production reliability and strict open-source privacy, all code contributions MUST adhere to these mandatory quality gates:
- Strict Clean Builds: All Rust builds (
cargo check,cargo build) MUST compile with 0 errors and 0 warnings (e.g.unused_imports,unused_variables). Any compilation warnings must be immediately cleaned prior to pull requests.
- Test-Driven Fixes: Before making any code change or bug fix, write or update a corresponding unit test to confirm the gap, verify the fix passes all unit tests, and perform end-to-end verification.
- Core RAG retrieval (
search_and_rerank) MUST operate dynamically against physical SQLite and LanceDB databases. Hardcoded static mock documents or dummy responses are strictly prohibited.
- Zero Local Topology Leakage: Hardcoding internal network topologies, private IP addresses, or internal development hostnames is strictly forbidden. Always use
127.0.0.1,localhost, or RFC 5737 test addresses. - Closed-Source / Dev Environment Privacy Guardrail: Before any
git commit, inspect the diff to ensure no local development paths, private environment secrets, or unannounced closed-source product links are committed until official public sites are launched.
This project is licensed under the MIT License - see the LICENSE file for details.