Web Server Benchmarks

A benchmark framework for measuring HTTP and WebSocket server performance and energy in Docker containers.

This repository benchmarks multiple stacks, including:

• Static HTTP servers (Apache, Nginx, Cowboy, Yaws, Erlang variants)
• Dynamic HTTP servers (Apache, Nginx+Python, Cowboy, Yaws, Erlang variants)
• WebSocket servers (Apache, Nginx+Java, Nginx+Python websockets, Nginx+Tornado, Cowboy, Yaws)

Overview

• Auto-discovers benchmarks from benchmarks/static, benchmarks/dynamic, benchmarks/websocket
• Builds and health-checks containers before running load
• Captures runtime metrics and energy (via Scaphandre when available)
• Stores CSV results by run timestamp
• Includes a GUI graph generator for analysis/export

Typical flow:

make setup
make build
make check-health
make run-quick
make graph

Prerequisites

• Linux (Debian/Ubuntu recommended)
• Python 3 + venv support (python3-venv)
• Docker
• Make
• Optional: Scaphandre for energy metrics

Verify:

make check-tools

Directory Structure

benchmarks/
  static/      # static HTTP benchmark containers
  dynamic/     # dynamic HTTP benchmark containers
  websocket/   # websocket benchmark containers
scripts/
  install_benchmarks.sh
  run_benchmarks.sh
  check_health.sh
  test_containers.sh
tools/
  measure_docker.py
  measure_websocket.py
  gui_graph_generator.py
  gen_usage_scenarios.py  # writes the GMT usage_scenario.yml files
  loadgen/                # client container used by the GMT scenarios
results/
logs/
docs/

Every benchmark directory also holds a usage_scenario.yml for the Green Metrics Tool next to its Dockerfile.

Main Commands

• make setup - create/repair venv and install dependencies
• make build - build all benchmark images
• make check-health - validate built images are runnable/healthy
• make test - build + health check all images
• make run - run full benchmark suite
• make run-quick - quick run with reduced parameters
• make run-super-quick - smoke run
• make run-single SERVER=<image> - run only one image
• make graph - launch graph GUI

You can also run specific suites with auto pattern targets:

• make run-static
• make run-dynamic
• make run-websocket

Configuration

Common environment variables:

• BENCH_DIR (default benchmarks) - benchmark root
• HOST_PORT (default 8001) - host port used for tests
• HTTP_MAX_WORKERS - HTTP worker count override
• BENCH_MEASURE_QUIET - compact vs verbose measure logs
• MEASURE_HEARTBEAT_SEC - quiet-mode heartbeat interval

Examples:

make run BENCH_DIR=benchmarks
HOST_PORT=9001 make check-health
HTTP_MAX_WORKERS=70 make run-static

Green Metrics Tool

Besides the Scaphandre-based path above, every benchmark ships a Green Metrics Tool usage_scenario.yml. GMT orchestrates the containers itself and stores energy, CPU, memory, network and the load generator's own metrics (requests, throughput, latency) in its dashboard, including carbon per request via the SCI.

The scenarios are run by GMT itself - one cluster job per benchmark, with the job's filename pointing at that benchmark's usage_scenario.yml. To change the load levels:

make gmt-scenarios                  # regenerate with the default profile
make gmt-scenarios GMT_PROFILE=full # the full sweep of scripts/run_benchmarks.sh

The load pattern is the same as make run, so both paths describe the same workload. See docs/GREEN_METRICS_TOOL.md for prerequisites (GMT needs an energy metric provider enabled), the phase layout and the caveats.

Results

Each run writes timestamped outputs under results/<timestamp>/ with CSV files grouped by benchmark type.

See:

• docs/RESULTS.md
• docs/CONFIGURATION_AUDIT.md
• docs/BENCHMARKS_AUDIT.md
• docs/GREEN_METRICS_TOOL.md

Notes

• Benchmark container names currently follow legacy prefixes (st-, dy-, ws-).
• The framework is generic and can support additional benchmark types if new runners/tools are added.