Skip to content

feat(genhttp-11): Enable limited WS - #1081

Merged
MDA2AV merged 2 commits into
MDA2AV:mainfrom
Kaliumhexacyanoferrat:genhttp-11-limited-ws
Aug 7, 2026
Merged

feat(genhttp-11): Enable limited WS#1081
MDA2AV merged 2 commits into
MDA2AV:mainfrom
Kaliumhexacyanoferrat:genhttp-11-limited-ws

Conversation

@Kaliumhexacyanoferrat

Copy link
Copy Markdown
Collaborator

Description


PR Commands — comment on this PR to trigger (requires collaborator approval):

Command Description
/benchmark -f <framework> Run every test the framework subscribes to
/benchmark -f <framework> -t <test> Run one test only
/benchmark -f <framework> --save Run and save results (updates the leaderboard on merge)
/benchmark -f <framework> -t <test> --save Run one test and save results
/benchmark -f <framework> --compare <other> Measure the deltas against another framework instead of this one

Always specify -f <framework>; the flags combine in any order. Results come back as a comment with a per-profile table of RPS, p99, CPU and memory.

What the deltas are measured against. By default, this framework's own results published on main - answering "did this change help?". When you are tuning a variant or a successor entry, --compare re-bases them on another entry instead:

/benchmark -f genhttp-11 --compare genhttp

The reply states which baseline it used, and profiles the other framework does not run show n/a rather than a delta.


Run benchmarks locally

You can validate and benchmark your framework locally with the lite script — no CPU pinning, fixed connection counts, all load generators run in Docker.

./scripts/validate.sh <framework>
./scripts/benchmark-lite.sh <framework> baseline
./scripts/benchmark-lite.sh --load-threads 4 <framework>

Requirements: Docker Engine on Linux. Load generators (gcannon, h2load, h2load-h3, wrk, ghz) are built as self-contained Docker images on first run.

@Kaliumhexacyanoferrat

Copy link
Copy Markdown
Collaborator Author

/benchmark -f genhttp-11 -t echo-ws-limited --save

@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@MDA2AV

MDA2AV commented Aug 7, 2026

Copy link
Copy Markdown
Owner

/benchmark -f genhttp-11 -t echo-ws-limited --save

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: genhttp-11 | Test: echo-ws-limited

Test Conn RPS CPU Mem Δ RPS Δ Mem
echo-ws-limited 512 1,031,076 5891.2% 351MiB NEW NEW
echo-ws-limited 4096 999,151 5891.1% 491MiB NEW NEW
Full log
#12 4.505 /source/Tests/Crud.cs(125,31): warning CS8602: Dereference of a possibly null reference. [/source/genhttp.csproj]
#12 4.596   genhttp -> /source/bin/Release/net10.0/genhttp.dll
#12 4.649   genhttp -> /app/
#12 DONE 4.7s

#13 [stage-1 2/3] WORKDIR /app
#13 CACHED

#14 [stage-1 3/3] COPY --from=build /app .
#14 CACHED

#15 exporting to image
#15 exporting layers done
#15 exporting manifest sha256:765950d1607be14b73e042fb85b996dd17df0024c15dccdbc813c4b56b579220 done
#15 exporting config sha256:26292d00fe6e688bc172b561ac29d49364e7d74af9f65b530638ef2d38ad4e4d done
#15 exporting attestation manifest sha256:9a1938282ed6d6938fff7de1e982de4427919f178bd85359fc6568a151cde0fb 0.0s done
#15 exporting manifest list sha256:c2fdbb427b0360817ff4edf4561ab3263aa9c35a3cce30fd81db729fbb2c3493 0.0s done
#15 naming to docker.io/library/httparena-genhttp-11:latest
#15 naming to docker.io/library/httparena-genhttp-11:latest done
#15 unpacking to docker.io/library/httparena-genhttp-11:latest 0.0s done
#15 DONE 0.1s
[info] tuning host for benchmark runs
[info] CPU governor → performance
[info] setting kernel socket limits
[info] setting UDP buffer sizes for QUIC
[info] setting loopback MTU to 1500 (realistic Ethernet)
[info] restarting docker daemon
[info] dropping kernel caches
[info] starting postgres sidecar
[info] postgres ready (seeded)
[info] starting redis sidecar (cpuset=0,64)
[info] redis ready

==============================================
=== genhttp-11 / echo-ws-limited / 512c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     512 (8/thread)
  Pipeline:  1
  Req/conn:  10
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    278us    123us    508us   2.76ms   5.33ms

  3564164 frames sent in 5.00s, 3563866 frames received
  Throughput: 712.60K req/s
  Bandwidth:  17.81MB/s
  WS upgrades: 358726
  WS frames:   3563866
  Latency samples: 3563866 / 3563866 responses (100.0%)
  Reconnects: 356280
[info] CPU 5578.4% | Mem 347MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     512 (8/thread)
  Pipeline:  1
  Req/conn:  10
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    190us    140us    302us   1.32ms   2.76ms

  5137859 frames sent in 5.00s, 5137772 frames received
  Throughput: 1.03M req/s
  Bandwidth:  25.67MB/s
  WS upgrades: 522703
  WS frames:   5137772
  Latency samples: 5137753 / 5137772 responses (100.0%)
  Reconnects: 513770
[info] CPU 5926.3% | Mem 341MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     512 (8/thread)
  Pipeline:  1
  Req/conn:  10
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    167us    123us    270us   1.06ms   2.66ms

  5154772 frames sent in 5.00s, 5155381 frames received
  Throughput: 1.03M req/s
  Bandwidth:  25.76MB/s
  WS upgrades: 524129
  WS frames:   5155381
  Latency samples: 5154678 / 5155381 responses (100.0%)
  Reconnects: 515439
[info] CPU 5891.2% | Mem 351MiB

=== Best: 1031076 req/s (CPU: 5891.2%, Mem: 351MiB) ===
[info] saved results/echo-ws-limited/512/genhttp-11.json
httparena-bench-genhttp-11
httparena-bench-genhttp-11

==============================================
=== genhttp-11 / echo-ws-limited / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  10
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    584us    131us   1.48ms   6.00ms   14.70ms

  3447014 frames sent in 5.00s, 3446984 frames received
  Throughput: 689.18K req/s
  Bandwidth:  17.22MB/s
  WS upgrades: 345692
  WS frames:   3446984
  Latency samples: 3446981 / 3446984 responses (100.0%)
  Reconnects: 344681
[info] CPU 5582.8% | Mem 466MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  10
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    337us    164us    605us   3.25ms   11.60ms

  4995444 frames sent in 5.00s, 4995757 frames received
  Throughput: 998.91K req/s
  Bandwidth:  24.95MB/s
  WS upgrades: 501345
  WS frames:   4995757
  Latency samples: 4995726 / 4995757 responses (100.0%)
  Reconnects: 499701
[info] CPU 5891.1% | Mem 491MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  10
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    301us    159us    487us   3.31ms   8.17ms

  4985325 frames sent in 5.00s, 4985556 frames received
  Throughput: 996.87K req/s
  Bandwidth:  24.91MB/s
  WS upgrades: 500913
  WS frames:   4985556
  Latency samples: 4985551 / 4985556 responses (100.0%)
  Reconnects: 498703
[info] CPU 5844.7% | Mem 521MiB

=== Best: 999151 req/s (CPU: 5891.1%, Mem: 491MiB) ===
[info] saved results/echo-ws-limited/4096/genhttp-11.json
httparena-bench-genhttp-11
httparena-bench-genhttp-11
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/genhttp-11.json - 2 new, 28 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
httparena-redis
[info] restoring loopback MTU to 65536

@MDA2AV
MDA2AV merged commit 6958800 into MDA2AV:main Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants