OpenGateway

🚀 The High-Performance LLM Data Plane with a CLI-only Control Plane

OpenGateway is a lightweight, high-performance LLM gateway designed for production environments. It provides a unified OpenAI-compatible data plane and a strictly CLI-first control plane, eliminating unnecessary web UI overhead and focusing on reliability, speed, and seamless integration.

✨ Key Features

• 🔌 Unified Data Plane: Fully OpenAI-compatible /v1/chat/completions and /v1/models interface.
• 🧭 Direct Provider Routing: API keys bind directly to provider pools with robust fallback chains.
• ⚖️ Advanced Load Balancing: Support for Priority, RoundRobin, and LeastConnections strategies.
• 🚦 Fine-grained Flow Control: QPS, concurrency limits, and bounded queuing per API key.
• 🛠️ CLI-First Control Plane: Manage providers, API keys, and system state exclusively via the CLI.
• 📊 Production Observability: Built-in request logging, metrics, and distributed tracing.
• ✅ Health & Failover: Active health monitoring with automatic, transparent provider failover.

📚 Documentation

• Documentation Index
• Architecture Guide
• AI Agent Guide

🚀 Quick Start

Installation

# Clone and build from source
git clone https://github.com/EeroEternal/OpenGateway.git
cd OpenGateway
cargo build --release

📚 Documentation Index →

Basic Usage

# Start a local gateway instance backed by one upstream provider
./target/release/opengateway \
  --provider zhipu \
  --model glm-4.6 \
  --api-key "your-zhipu-key" \
  --host 127.0.0.1 \
  --port 3105

# Point OpenAI-compatible clients at the data plane
curl -X POST http://127.0.0.1:3105/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"glm-4.6","messages":[{"role":"user","content":"hello"}]}'

OpenGateway no longer ships client-specific presets. If you are integrating Zed, Aider, OpenHands, Codex CLI, or another tool, point that client at the gateway's compatible OpenAI-compatible HTTP interface.

📋 CLI Control Plane

OpenGateway moves all management logic to the CLI. Use the global --database-url flag to specify your backing SQLite or PostgreSQL instance.

# General status check
cargo run -- gateway status --format json

# Managing Providers
cargo run -- provider list --format json
cargo run -- provider create --name deepseek --type openai --config '{"api_key":"sk-xxx","model":"deepseek-chat"}' --format json

# Managing API Keys
cargo run -- api-key list --format json
cargo run -- api-key create --name prod-key --provider-ids 1,2 --strategy Priority --qps-limit 10 --format json

🏗️ Architecture & Operations

The HTTP admin control plane and web UI have been removed. Runtime management is now CLI-first, providing higher security and lower operational overhead.

For full architectural details, request paths, and load balancing strategies, see docs/DEVELOPMENT.md. architecture.md](docs/architecture

Development Setup

# Set your backing database
export DATABASE_URL="sqlite://target/opengateway.db"
export OPENGATEWAY_TRACE_ENABLED="true"

# Start the gateway data plane
cargo run --bin opengateway -- --host 127.0.0.1 --port 3105

The data plane will be available at http://localhost:3105/v1.

🦿 Agent Integration

OpenGateway is designed to be managed by AI Agents. For specific operating rules, tool usage, and CLI constraints, see AGENTS.md.

🔧 Advanced Configuration

OpenGateway can be tuned via CLI flags or environment variables.

# Provider API keys for direct startup
export ZHIPU_API_KEY="your-zhipu-api-key"
export OPENAI_API_KEY="sk-xxx"

# Internal trace stats (optional)
export OPENGATEWAY_TRACE_ENABLED="true"
export OPENGATEWAY_TRACE_NAME="opengateway.chat"

For more details on persistent configuration using opengateway.yaml, see the Configuration Guide.

🧪 Testing

# Test health endpoint (if enabled)
curl http://localhost:3105/health

# Test OpenAI-compatible Chat Completions
curl -X POST http://localhost:3105/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-api-key" \
  -d '{"model": "gpt-4", "messages": [{"role": "user", "content": "Hello!"}]}'

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.