CodeEngine 🚀

CodeEngine is a lightweight, high-performance, and secure online code execution system. It allows users to write, submit, and run untrusted code in real time inside isolated, resource-constrained Docker sandboxes. The frontend communicates with a Go backend gateway via Server-Sent Events (SSE) to stream execution progress and output back to the user instantly.

---

⚙️ Local Setup Instructions

Prerequisites

Make sure you have the following installed on your system:

• Docker Desktop
• git (for downloading and updating the codebase)
• macOS / Linux / WSL / Git Bash: bash shell and curl (for checking API gateway readiness)
• Windows CMD / PowerShell: Windows 10/11 built-in command prompt or PowerShell

---

🛡️ gVisor (runsc) Sandbox Security & Setup

CodeEngine uses gVisor (runsc) as its sandbox container runtime for strong kernel-level isolation when running untrusted code.

1. Installing runsc on Linux / ARM64 / x86_64

Run the following script to download the binary for your host architecture (ARM64 aarch64 or x86_64 x86_64), verify its checksum, and register it with Docker:

(
  set -e
  ARCH=$(uname -m)
  URL=https://storage.googleapis.com/gvisor/releases/release/latest/${ARCH}
  wget ${URL}/runsc ${URL}/runsc.sha512
  sha512sum -c runsc.sha512
  chmod +x runsc
  sudo mv runsc /usr/local/bin/
  sudo runsc install
  sudo systemctl restart docker
)

2. Registering with Docker (/etc/docker/daemon.json)

Running sudo runsc install automatically updates /etc/docker/daemon.json to register the runtime:

{
    "runtimes": {
        "runsc": {
            "path": "/usr/local/bin/runsc"
        }
    }
}

3. Performance Trade-off & Syscall Overhead

• gVisor intercepts and handles Linux system calls in user space rather than passing them directly to the host kernel.
• This introduces a ~10–30% syscall performance overhead, which is especially noticeable during compiler-heavy phases (such as g++ invocation).
• Given CodeEngine's strict 7-second execution timeout, this overhead is a highly acceptable security trade-off to prevent container escape and host kernel exploitation.

4. Local Development Fallback (SANDBOX_RUNTIME=runc)

If you are developing locally on Docker Desktop without runsc registered in your daemon, override the sandbox runtime using the SANDBOX_RUNTIME environment variable:

SANDBOX_RUNTIME=runc ./start.sh

---

Step 1: Clone, Pull, and Open Project Folder 📁

Open your terminal and run the following commands to download the repository (or pull latest changes) and open the folder:

# Clone the repository (if you do not have it locally yet)
git clone https://github.com/Soham-Donode/code-engine-v1.git

# Navigate into the project directory
cd code-engine-v1

# Pull the latest changes (if you already cloned the project earlier)
git pull origin main

---

Step 2: Ensure Docker Desktop is Running 🐳

Before launching the services, you must ensure Docker Desktop is open and active on your system:

• Mac / Windows: Open the Docker Desktop application from your Applications folder or system tray and wait for it to show a green "Running" status.
• Linux: Verify the Docker daemon is running by starting it in the terminal:

  sudo systemctl start docker

---

Step 3: Run the Application 🚀

Choose the section below corresponding to your operating system to launch the application using our automated setup scripts:

🍎 macOS & 🐧 Linux

We provide automated helper scripts (start.sh and stop.sh) to manage the services.

1. Set File Permissions

First, make both the start and stop scripts executable on your system:

chmod +x start.sh stop.sh

2. Start Services

Run the startup script from the root directory to spin up the entire application stack:

./start.sh

• What this does:
  1. Verifies Docker and the Docker daemon are running.
  2. Spins up the services (nginx, engine, redis, postgres) in the background.
  3. Polls the API gateway (http://localhost/submit) until it is online.
  4. Opens frontend/index.html in your default browser.

3. Stop Services

To stop and clean up all running services, containers, and networks:

./stop.sh

• What this does:
  1. Runs docker compose down to cleanly stop and remove all services.

---

🪟 Windows (Command Prompt / PowerShell / Git Bash / WSL)

We provide Windows-native batch scripts (start.bat and stop.bat) for CMD/PowerShell, and shell scripts for Git Bash/WSL.

Option A: Native Command Prompt / PowerShell

• Start Services: Double-click start.bat in File Explorer, or run:

  start.bat

• Stop Services: Double-click stop.bat in File Explorer, or run:

  stop.bat

Option B: Git Bash / WSL (Windows Subsystem for Linux)

• Set File Permissions (only needed once):

  chmod +x start.sh stop.sh

• Start Services:

  ./start.sh

• Stop Services:

  ./stop.sh

• What these scripts do:

  1. Verify Docker and the Docker daemon are running.
  2. Spin up the services in the background using Docker Compose.
  3. Poll the API gateway until it is online (if curl is available).
  4. Open frontend/index.html in your default web browser.

---

🛠️ Method B: Manual Commands (All Platforms)

If you prefer to orchestrate and manage the infrastructure manually, use the commands below:

1. Spin Up Services

Build the container images and run them in background detached mode:

docker compose up -d --build

2. Monitor Container Logs

To stream logs from all running containers:

docker compose logs -f

Or view logs from a specific container:

docker compose logs -f engine
docker compose logs -f nginx

3. Open the Frontend

Since the startup script is bypassed, manually double-click or run:

open frontend/index.html

4. Tear Down Services

To stop the services and release all network ports:

docker compose down

---

Preview

---

🔑 API Key Management & Rate Limits

CodeEngine implements SHA-256 hashed API key authentication with a flat rate limit of 100 requests per key per 24 hours.

API Key Endpoints

1. Generate a New API Key

• Method: POST /api/keys
• Request Body (optional):

  {
    "label": "my laptop"
  }

• Response (201 Created):

  {
    "api_key": "ce_a1b2c3d4e5f6...",
    "key_prefix": "ce_a1b2c",
    "daily_limit": 100,
    "created_at": "2026-07-25T14:00:00Z"
  }

  [!IMPORTANT] The plaintext api_key is returned only once in this response and cannot be retrieved again. Only the SHA-256 hash is stored.

2. Check Key Usage & Remaining Quota

• Method: GET /api/keys/:prefix/usage
• Response (200 OK):

  {
    "requests_today": 12,
    "daily_limit": 100,
    "remaining": 88,
    "resets_at": "2026-07-26T14:00:00Z"
  }

3. Revoke an API Key

• Method: DELETE /api/keys/:prefix
• Response (200 OK):

  {
    "message": "API key revoked successfully",
    "key_prefix": "ce_a1b2c"
  }

---

Using API Keys (curl Quickstart)

Step 1: Generate an API Key

curl -X POST http://localhost/api/keys \
  -H "Content-Type: application/json" \
  -d '{"label": "dev-key"}'

Step 2: Submit Code for Sandboxed Execution

Pass your generated API key using the X-API-Key header:

curl -X POST http://localhost/submit \
  -H "Content-Type: application/json" \
  -H "X-API-Key: ce_YOUR_GENERATED_API_KEY" \
  -d '{
    "language": "python",
    "code": "print(\"Hello from CodeEngine!\")",
    "input": ""
  }'

Step 3: Stream Real-Time Execution Results

curl -N -H "X-API-Key: ce_YOUR_GENERATED_API_KEY" \
  http://localhost/stream/YOUR_SUBMISSION_ID

---

🏗️ System Architecture

CodeEngine follows a decoupled, distributed architecture designed for safety, speed, and real-time execution feedback.

Component Relationship Diagram

The diagram below shows how the user client, the rate-limiting gateway, the Go execution engine, the Docker sandbox runner, and the databases/cache interact:

graph TD
    subgraph Client [Client Space]
        FE[Frontend UI: HTML/CSS/JS]
    end

    subgraph Gateway [Gateway Layer]
        NG[Nginx Reverse Proxy: Port 80]
        RL[IP-based Rate Limiter: 2 req/s]
    end

    subgraph App [Go Backend Engine]
        API[Gin API Server: Port 8080]
        WK[Background Worker Go-routine]
        RN[Docker Runner Package]
    end

    subgraph Storage [Storage & Caching]
        RD[(Redis 7 Cache / Streams)]
        PG[(PostgreSQL 15 Submissions DB)]
    end

    subgraph Sandbox [Container Isolation]
        DS[Host Docker Daemon]
        SB[Isolated Containers: Python / C++ / JavaScript]
    end

    FE -->|1. Submit Code| NG
    FE -->|4. SSE Stream| NG
    NG -->|Rate-Limited Proxy| API
    NG -->|Unbuffered Proxy| API

    API -->|2. Ingest & Queue| RD
    WK -->|5. Fetch Job| RD
    WK -->|6. Run Task| RN
    RN -->|7. Docker API| DS
    DS -->|8. Spawn Ephemeral Sandbox| SB
    WK -->|9. Update Status & Output| RD
    
    %% Storage connections (store.go prepared architecture)
    API -.->|Optional Persistence| PG
    WK -.->|Optional Persistence| PG

Loading

The Code Submission Lifecycle

The sequence diagram below displays the step-by-step lifecycle of a code compilation/execution job:

sequenceDiagram
    autonumber
    actor User as Frontend Client
    participant Nginx as Nginx (Port 80)
    participant API as Go API (Port 8080)
    participant Redis as Redis Cache & Streams
    participant Worker as Go Worker Go-routine
    participant Docker as Docker Daemon / Sandbox

    User->>Nginx: POST /submit (Language, Code, Input)
    Note over Nginx: Rate limit check applied<br/>(2 req/sec, burst 5)
    Nginx->>API: Forward request
    API->>Redis: Initialize cache state ("status:queued")
    API->>Redis: Push job to stream "code_queue"
    API-->>User: Respond with generated submission_id
    
    Note over User, API: SSE Stream Connection Established
    User->>Nginx: GET /stream/:id
    Nginx->>API: Forward stream connection (No buffering)
    
    par Stream Status Polling
        loop Every 500ms (until terminal state)
            API->>Redis: Get status:<submission_id>
            Redis-->>API: Return state payload
            API-->>User: Push SSE Message (stdout/stderr/status)
        end
    and Background Task Processing
        Worker->>Redis: Read next message from "code_queue"
        Redis-->>Worker: Return job details
        Worker->>Redis: Update cache state ("status:running")
        Worker->>Docker: Execute code in sandbox
        Note over Docker: Run docker container with:<br/>--network none, -m 256m, --rm
        Docker-->>Worker: Return stdout/stderr/time duration
        Worker->>Redis: Update cache state ("status:completed/error/timeout")
        Worker->>Redis: XAck (Acknowledge) stream message
    end

Loading

---

🔍 Core Component Breakdown

1. Frontend Client (frontend/)

An interactive IDE interface styled using Vanilla CSS:

• Submits code to /submit.
• Connects to /stream/:id via an EventSource (SSE).
• Dynamically handles compilation errors, standard output, and process timeouts.

2. Nginx API Gateway (nginx.conf)

Serves as the frontend-facing gateway (Port 80) and performs reverse-proxy routing and rate-limiting:

• Rate Limiting: Protects backend services from load surges by limiting requests to 2 requests/sec with a burst buffer of 5 using a remote IP limit zone.
• SSE Streaming Support: Disables proxy buffering (proxy_buffering off), connection caching, and chunked transfer encoding for /stream/ requests, allowing real-time status updates to flow without lag.

3. Go Engine & Worker (main.go)

Runs a Gin router and serves the execution flow:

• API Router:
  • POST /submit: Ingests code/language payload, creates a temporary status in Redis, and pushes the job to the Redis Stream.
  • GET /stream/:id: Delivers SSE events by polling Redis status every 500ms. Automatically closes the stream on terminal states (completed, error, timeout).
  • CORS Preflight: Handles cross-origin requests and OPTIONS preflight checks seamlessly (returning 204 status).
• Async Worker: An asynchronous worker routine that polls the Redis Stream code_queue within a consumer group, processing one job at a time to distribute execution load.

4. Sandbox Runner (runner/docker.go)

Interacts with the Docker daemon via the mounted socket to spin up isolated container instances:

• Security Restrictions & Platform Support:
  • --runtime=runsc: Uses gVisor user-space kernel isolation (configurable via SANDBOX_RUNTIME env var, defaulting to runsc).
  • --network none: Isolated sandboxing prevents external network calls.
  • -m 256m: Caps container memory usage to 256MB.
  • --pids-limit 64: Prevents fork bomb attacks inside the sandbox.
  • --cap-drop ALL: Drops all Linux kernel capabilities.
  • --platform linux/arm64: Native Apple Silicon (ARM64) and multi-architecture runtime support.
  • --rm: Removes the container automatically upon completion.
• Supported Languages (Lightweight Alpine Runtimes):
  • Python: Runs code inline using python:3.10-alpine.
  • C++: Compiles and executes code inside frolvlad/alpine-gxx via standard gcc compilation (g++ -O0).
  • JavaScript / Node.js: Executes code inline using node:20-alpine.
• Timeout Handler: Enforces a strict 7-second time limit per run.

5. Datastore & Storage (store/store.go, db/init.sql)

Prepared for persistence integrations:

• PostgreSQL: Configured with a schema for storing historical submissions persistently.
• Redis: Serves as a fast, in-memory cache for status logs and as the queue broker for processing stream tasks.

---

🗄️ Repository Directory Layout

.
├── Dockerfile              # Configures Go runner container environment
├── db/
│   └── init.sql            # Schema definitions for PostgreSQL database
├── docker-compose.yml      # Service definitions (Nginx, Engine, Redis, Postgres)
├── frontend/
│   ├── index.html          # Interactive IDE Frontend
│   ├── style.css           # Modern IDE aesthetics & typography
│   └── app.js              # State management & SSE event handler
├── go.mod                  # Go module definition file
├── go.sum                  # Package checksum registry
├── main.go                 # Go API entrypoint & Stream Worker implementation
├── nginx.conf              # Reverse proxy, rate limiter & buffering configuration
├── runner/
│   └── docker.go           # Dynamic Docker sandbox container driver
├── start.sh                # Helper script to launch services
├── stop.sh                 # Helper script to stop services
└── store/
    └── store.go            # DB & Redis integration storage module