Skip to content

update mq-bridge, add crud, fortunes - #1312

Open
marcomq wants to merge 4 commits into
MDA2AV:mainfrom
marcomq:update/mq-bridge
Open

update mq-bridge, add crud, fortunes#1312
marcomq wants to merge 4 commits into
MDA2AV:mainfrom
marcomq:update/mq-bridge

Conversation

@marcomq

@marcomq marcomq commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Description

Updating mq-bridge to latest version.

  • add crud
  • add fortunes
  • optimize postgres
  • use mimalloc allocater

PR Commands — comment on this PR to trigger (requires collaborator approval):

Command Description
/benchmark -f <framework> Run every test the framework subscribes to
/benchmark -f <framework> -t <test> Run one test only
/benchmark -f <framework> --save Run and save results (updates the leaderboard on merge)
/benchmark -f <framework> -t <test> --save Run one test and save results
/benchmark -f <framework> --compare <other> Measure the deltas against another framework instead of this one
/benchmark-multiple -f <fw1>,<fw2>,... Benchmark several frameworks in one run — takes -t and --save too; saved results land in a single commit
/benchmark-multiple --save No -f needed: benchmark and save every framework the PR touches
/benchmark-test -t <test> Benchmark all enabled frameworks subscribed to <test> and save the results

For /benchmark, always specify -f <framework>; the flags combine in any order. Results come back as a comment with a per-profile table of RPS, p99, CPU and memory — one table per framework on multi runs. A new benchmark comment while a run is in flight queues behind it (one deep) instead of cancelling it. For multi-framework PRs (dependency bumps, same-language refactors) prefer /benchmark-multiple, which runs everything in a single job and commits all saved results together, so no run overwrites another. --compare works on single-framework runs only.

What the deltas are measured against. By default, this framework's own results published on main - answering "did this change help?". When you are tuning a variant or a successor entry, --compare re-bases them on another entry instead:

/benchmark -f genhttp-11 --compare genhttp-11-kestrel

The reply states which baseline it used, and profiles the other framework does not run show n/a rather than a delta.


Run benchmarks locally

You can validate and benchmark your framework locally with the lite script — no CPU pinning, fixed connection counts, all load generators run in Docker.

./scripts/validate.sh <framework>
./scripts/benchmark-lite.sh <framework> baseline
./scripts/benchmark-lite.sh --load-threads 4 <framework>

Requirements: Docker Engine on Linux. Load generators (gcannon, h2load, h2load-h3, wrk) are built as self-contained Docker images on first run.

@marcomq

marcomq commented Aug 25, 2026

Copy link
Copy Markdown
Contributor Author

/benchmark -f mq-bridge

@github-actions

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: mq-bridge | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 1,132,385 3558.7% 167MiB +5.0% +70.4%
baseline 4096 1,221,306 4104.7% 288MiB +3.8% +59.1%
pipelined 512 5,558,624 6056.6% 139MiB +12.7% +178.0%
pipelined 4096 5,481,331 6497.9% 256MiB +9.7% +52.4%
limited-conn 512 746,840 4584.0% 180MiB +4.3% +122.2%
limited-conn 4096 724,086 6263.2% 338MiB +1.4% +31.0%
json 4096 943,150 6010.9% 379MiB +5.8% +43.6%
json-comp 512 525,681 5690.8% 434MiB +566.0% +456.4%
json-comp 4096 568,162 6261.3% 605MiB +39.4% +155.3%
json-comp 16384 558,406 6472.2% 1.2GiB +45.2% +67.9%
json-tls 4096 979,780 5362.2% 362MiB +4.0% +47.2%
upload 32 1,397 2841.9% 1.3GiB -0.5% -18.8%
upload 256 1,540 6224.2% 4.7GiB +4.7% -2.1%
api-4 256 51,685 399.1% 66MiB +75.0% +43.5%
api-16 1024 164,948 1595.2% 134MiB +80.4% +31.4%
static 1024 112,405 3155.7% 2.9GiB NEW NEW
static 4096 112,485 4126.2% 4.0GiB NEW NEW
static 6800 105,939 4603.1% 4.4GiB NEW NEW
static-tls 1024 109,992 2920.4% 3.0GiB NEW NEW
static-tls 4096 65,023 1630.0% 3.4GiB NEW NEW
static-tls 6800 109,695 3285.0% 3.5GiB NEW NEW
async-db 1024 142,658 3145.9% 298MiB +24.3% +125.8%
crud 4096 235,210 3046.4% 537MiB NEW NEW
fortunes 1024 130,380 4736.1% 404MiB NEW NEW
baseline-h2 256 688,984 4980.6% 403MiB +26.9% +37.5%
baseline-h2 1024 611,368 5028.6% 1.0GiB +18.9% +29.8%
static-h2 256 110,346 3429.5% 3.1GiB -67.6% +2030.5%
static-h2 1024 106,583 3410.5% 3.4GiB -56.7% +891.9%
baseline-h2c 256 710,885 5133.0% 474MiB +25.9% +81.6%
baseline-h2c 1024 637,821 5231.4% 1.0GiB +27.5% +23.4%
baseline-h2c 4096 561,168 5378.2% 3.3GiB +24.5% +32.0%
json-h2c 1024 732,816 4525.0% 528MiB +43.3% +40.1%
json-h2c 4096 679,217 4470.3% 1.5GiB +44.6% +36.4%
Full log
40. Stopping all clients.Stopped all clients for thread #50


20. Stopping all clients.

finished in 5.06s, 741559.40 req/s, 2.47GB/s
requests: 3707797 total, 3740565 started, 3707797 done, 3707797 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 3708052 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 12.33GB (13240837391) total, 124.53MB (130580645) headers (space savings 76.48%), 12.15GB (13043457785) data
                     min         max         mean         sd        +/- sd
time for request:       56us    152.96ms     42.72ms     24.17ms    64.66%
time for connect:       18us       860us       355us       197us    63.09%
time to 1st byte:     1.00ms    107.50ms     18.90ms     19.12ms    86.43%
req/s           :     633.95      877.45      724.08       39.44    68.65%
[info] CPU 4525.0% | Mem 528MiB

[run 3/3]
starting benchmark...
Application protocol: h2c
35.
42Main benchmark duration is started for thread #41.
.
39.
11.
57.
22.
34..


52Main benchmark duration is started for thread #34.
.
33.

.

.
.Main benchmark duration is started for thread #26.

35. Stopping all clients.
30Main benchmark duration is over for thread #29. Stopping all clients.
. Stopping all clients.
6. Stopping all clients.
30
10
. Stopping all clients.Stopped all clients for thread #46


finished in 5.06s, 734378.40 req/s, 2.44GB/s
requests: 3671892 total, 3704660 started, 3671892 done, 3671892 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 3672098 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 12.21GB (13112819026) total, 123.48MB (129483107) headers (space savings 76.45%), 12.03GB (12917183689) data
                     min         max         mean         sd        +/- sd
time for request:       54us    150.61ms     43.41ms     24.16ms    64.76%
time for connect:       18us      1.10ms       449us       239us    63.87%
time to 1st byte:     1.23ms     99.79ms     17.17ms     17.42ms    85.74%
req/s           :     619.18      863.61      717.06       37.60    69.73%
[info] CPU 4557.8% | Mem 560MiB

=== Best: 732816 req/s (CPU: 4525.0%, Mem: 528MiB) ===
[info] saved results/json-h2c/1024/mq-bridge.json
httparena-bench-mq-bridge
httparena-bench-mq-bridge

==============================================
=== mq-bridge / json-h2c / 4096c (tool=h2load) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
starting benchmark...
16.

.Warm-up phase is over for thread #17
.Warm-up phase is over for thread #.20
.Main benchmark duration is started for thread #Warm-up phase is over for thread #25.
.Main benchmark duration is started for thread #
48.
.
progress: Main benchmark duration is started for thread #20.84
% of clients started
2151.
.
13.

22.
56Warm-up phase is over for thread #Main benchmark duration is started for thread #10.27
28.
..


Application protocol: h2c

52. Stopping all clients.Stopped all clients for thread #
32
3. Stopping all clients.

3
. Stopping all clients.

28. Stopping all clients.

finished in 5.11s, 673277.60 req/s, 2.24GB/s
requests: 3366388 total, 3497460 started, 3366388 done, 3366388 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 3366734 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 11.18GB (12003855447) total, 113.19MB (118684191) headers (space savings 76.45%), 11.01GB (11824347815) data
                     min         max         mean         sd        +/- sd
time for request:       57us    823.71ms    185.97ms    136.78ms    56.52%
time for connect:       18us      6.23ms      2.39ms      1.69ms    56.81%
time to 1st byte:     5.85ms    573.20ms     67.98ms     69.59ms    89.43%
req/s           :      98.95      257.27      164.30       19.81    69.73%
[info] CPU 4478.4% | Mem 1.1GiB

[run 2/3]
starting benchmark...
Application protocol: h2c
.Main benchmark duration is started for thread #56.


35.

29.
57.
0. Stopping all clients.
59Stopped all clients for thread #4
. Stopping all clients.Stopped all clients for thread #
15
33. Stopping all clients.

40
45
. Stopping all clients.

finished in 5.10s, 692741.40 req/s, 2.30GB/s
requests: 3463707 total, 3594779 started, 3463707 done, 3463707 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 3464011 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 11.50GB (12352643538) total, 116.82MB (122499671) headers (space savings 76.38%), 11.33GB (12167569125) data
                     min         max         mean         sd        +/- sd
time for request:       51us    779.30ms    178.71ms    140.15ms    57.74%
time for connect:       17us      7.92ms      3.04ms      2.33ms    54.05%
time to 1st byte:     3.42ms    571.94ms     37.58ms     42.57ms    91.41%
req/s           :      92.77      262.68      169.05       21.80    68.58%
[info] CPU 4470.3% | Mem 1.5GiB

[run 3/3]
starting benchmark...
Application protocol: h2c
18.

18.
.Main benchmark duration is started for thread #57
.Main benchmark duration is started for thread #31.


30.Warm-up phase is over for thread #
59.

6.
15
. Stopping all clients.
36Main benchmark duration is over for thread #
46. Stopping all clients.
. Stopping all clients.
. Stopping all clients.
35. Stopping all clients.


finished in 5.09s, 672300.20 req/s, 2.23GB/s
requests: 3361501 total, 3492573 started, 3361501 done, 3361501 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 3361764 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 11.16GB (11987462475) total, 113.64MB (119162308) headers (space savings 76.32%), 11.00GB (11807565502) data
                     min         max         mean         sd        +/- sd
time for request:       60us    768.42ms    185.11ms    139.27ms    56.09%
time for connect:       17us      8.18ms      2.97ms      2.16ms    53.74%
time to 1st byte:     4.31ms    545.18ms     32.08ms     33.35ms    94.97%
req/s           :     103.58      245.82      164.05       20.15    68.70%
[info] CPU 4475.0% | Mem 1.6GiB

=== Best: 679217 req/s (CPU: 4470.3%, Mem: 1.5GiB) ===
[info] saved results/json-h2c/4096/mq-bridge.json
httparena-bench-mq-bridge
httparena-bench-mq-bridge
[info] skip: mq-bridge does not subscribe to baseline-h3
[info] skip: mq-bridge does not subscribe to static-h3
[info] skip: mq-bridge does not subscribe to gateway-64
[info] skip: mq-bridge does not subscribe to gateway-h3
[info] skip: mq-bridge does not subscribe to production-stack
[info] skip: mq-bridge does not subscribe to unary-grpc
[info] skip: mq-bridge does not subscribe to unary-grpc-tls
[info] skip: mq-bridge does not subscribe to echo-ws
[info] skip: mq-bridge does not subscribe to echo-ws-pipeline
[info] skip: mq-bridge does not subscribe to echo-ws-limited
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/mq-bridge.json - 33 new, 39 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
httparena-redis
[info] restoring loopback MTU to 65536

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant