WebSocket benchmark · Linode (Akamai Cloud) · Dedicated CPU · 14 October 2026
4 vCPU, 8 GB RAM : 795,888 WebSocket clients connected at once.
API Maker 4.0.0 on a Linode Dedicated 8 GB (g6-dedicated-4) : 4 dedicated vCPUs and 8 GB of RAM at $72 a month, its MongoDB and Redis on the same machine. A second server opened WebSocket clients, each with both tokens and a subscription of its own, until the memory of the server was full, while autocannon made the calls which notify them : every run 60 seconds, the memory and CPU of every part sampled each second.
Clients connected
795,888
the memory was full : 1,024 MB left
Memory per client
8.7KB
API Maker 3.3 · kernel 4.4 · Redis 1.1
New clients / second
6,627
both tokens and a subscription · 12,535 without one
Notification delivered in
1.04ms
p50 at 1,000 a second · p99 2.03 ms
The server¶
| Server | Linode (Akamai Cloud), Dedicated 8 GB (g6-dedicated-4), dedicated cpu, $72 / month |
| CPU | AMD EPYC 7642 48-Core Processor, 4 dedicated vCPUs (KVM) KVM guest, 1 thread per core · L1d 256 KiB (4 instances) · L2 2 MiB (4 instances) · L3 64 MiB (4 instances) · BogoMIPS 4599.99 |
| Memory | 8 GB (7,941 MB usable), swap 8.3 GB (496 MB of Linode + 7.8 GB made by the installer) |
| Disk, region | 160 GB SSD, Mumbai, India (ap-west) |
| System | Ubuntu 24.04.4 LTS, Linux 6.8.0-134, Docker 29.9.0 |
| API Maker | 4.0.0 : release build of api_maker_be_rust 423a34c with the new WebSocket core of 14 October 2026 (opt-level 3, fat LTO), installed by its installer (https://apimaker.dev/v4/install.sh), no reverse proxy |
| Settings | the defaults of the installer : one worker thread per vCPU, up to 2,032,896 open files for the service (LimitNOFILE), and the sandboxes kept running for the whole run (removeSandboxInactiveSinceMinutes 1440) |
| Network limits | as the installer sets them : 2,032,896 connections tracked by the firewall of Linux (net.netfilter.nf_conntrack_max), 16,384 waiting to be accepted (net.core.somaxconn). The memory of all its TCP connections together : raised by API Maker as the clients came, to 1.9 GB at the most (net.ipv4.tcp_mem, 25 % of the memory ; Linux gives 9 %) |
| Databases | MongoDB 8.0.10 (the MongoDB of API Maker, a replica set of one member, in Docker) · Redis 7.0.5 (the Redis of API Maker, in Docker) : on the same server. No other database runs |
| Sandboxes | TypeScript on Node.js 24.21.0 (an image of 404 MB built by API Maker), 2 sandboxes of the account (sandboxCountForAdmin 2, the default) |
| Load generator | Dedicated 16 GB (g6-dedicated-8) : AMD EPYC 7642 48-Core Processor, 8 dedicated vCPUs, Ubuntu 24.04.4 LTS. The WebSocket clients : am_ws_load, a program in Rust made for these benchmarks, a socket and a task per client, from 41 addresses of the load generator. The HTTP calls : Node.js 24.21.0, autocannon 8.0.0 |
| Network | A VLAN of the same data centre, round trip 0.38 ms (ping) |
| Date | 14 October 2026, 03:18 to 03:58 UTC |
Only what the run needs was running : API Maker and its two sandboxes, its MongoDB and its Redis. API Maker was restarted and the swap emptied before the run. How every number was taken : How the WebSocket benchmarks are made. Every server side by side : All results. The APIs of this server : 4 vCPU · 8 GB, API benchmark.
The data¶
- The clients connect to
ws://<server>:38246/?x-am-authorization=<token>&x-am-user-authorization=<token>: the token of an API user, and the token of a person, a row of thecustomerstable of the benchmark dataset (https://apimaker.dev/json/benchmark_dataset.json) made by a DB token generator. API Maker checks both. - A subscription is one
REGISTERfor the custom WebSocket eventorder-updateswith a condition of its own :{ conditionType: 'RESPONSE', criteria: { recipientId: '<the number of the client>' } }. It is the way one notification reaches one person. - A notification for one person is 141 bytes ; a row of the dataset told to the clients of a table is 321 bytes.
- The tables are the ones of the API benchmarks, in MongoDB : ten rows of customers, ten columns each.
Results at a glance¶
Subscribe to an event with a can user connect codethe code ran in the server
Subscribe : the code reads the databasethe code ran in the sandbox
Subscribe : the code reads the clockthe code ran in the sandbox
100 clients at a time connect, are let in and leave, for 60 seconds.
A notification for one person13,049 calls / second
A broadcast120 calls / second
Rows inserted in a table779 calls / second
As many calls as the server takes : what the clients received each second.
How many clients fit¶
Clients connect 4,000 a second, level after level, each with both tokens and a subscription of its own ; at each level they wait 30 seconds while the server is sampled (the level the run stops at, 60). The run stopped adding clients at 795,888 : the memory of the server was full. 922 MB of its 7,941 MB were still counted as available, and Linux had moved 537 MB to the swap since the clients began to connect, to make that room : 385 MB were left (the run stops when what is available, less what Linux moved to the swap since the clients began to connect, is under 5 % of the memory of the server).
API MakerMongoDB of API MakerRedisSandboxesLinux : its sockets, Docker, the restAvailable
No client
10,000 clients
50,000 clients
100,000 clients
200,000 clients
300,000 clients
400,000 clients
500,000 clients
600,000 clients
700,000 clients
795,888 clients
| Clients | API Maker | Redis | Kernel | MongoDB | Sandboxes | Server, used | Available | Swap | CPU busy |
|---|---|---|---|---|---|---|---|---|---|
| None (at rest) | 118 | 1 | 224 | 123 | 55 | 852 | 7,090 | 0 | 0.8 % |
| 10,000 | 156 | 12 | 322 | 158 | 480 | 1,457 | 6,485 | 0 | 1.6 % |
| 50,000 | 286 | 52 | 457 | 206 | 479 | 1,775 | 6,166 | 0 | 1.4 % |
| 100,000 | 440 | 103 | 681 | 203 | 479 | 2,164 | 5,777 | 0 | 2.8 % |
| 200,000 | 766 | 203 | 1,129 | 203 | 479 | 3,001 | 4,940 | 0 | 4.1 % |
| 300,000 | 1,086 | 313 | 1,579 | 203 | 479 | 3,798 | 4,143 | 0 | 5.7 % |
| 400,000 | 1,389 | 406 | 2,007 | 203 | 479 | 4,499 | 3,442 | 0 | 10.1 % |
| 500,000 | 1,732 | 498 | 2,430 | 203 | 479 | 5,299 | 2,643 | 0 | 13.0 % |
| 600,000 | 2,031 | 625 | 2,848 | 203 | 479 | 6,135 | 1,806 | 0 | 16.6 % |
| 700,000 | 2,300 | 716 | 3,276 | 183 | 386 | 6,758 | 1,184 | 124 | 18.8 % |
| 795,888 | 2,474 | 806 | 3,691 | 66 | 203 | 6,917 | 1,024 | 594 | 19.3 % |
Memory in MB. API Maker is the resident memory of its process ; Redis is the memory of its data (used_memory) ; the kernel of Linux keeps every socket and every tracked connection in its slab ; a container is counted the way docker stats counts it. CPU busy is the whole server while the clients wait : the server pings every client once a minute, and each answers.
Memory when the server is full¶
With 795,888 clients connected and waiting, of the 7,941 MB of the server :
| Part | At rest, no client | 795,888 clients | More |
|---|---|---|---|
| API Maker (process) | 118 MB | 2,474 MB | + 2,356 MB |
| Redis (container) | 3 MB | 802 MB | + 799 MB |
| Linux kernel : sockets, tracked connections (slab) | 224 MB | 3,691 MB | + 3,467 MB |
| MongoDB of API Maker (container) | 123 MB | 66 MB | − 57 MB |
| Sandboxes (2 containers) | 55 MB | 203 MB | + 148 MB |
| Whole server, used | 852 MB | 6,917 MB | + 6,066 MB |
| Available | 7,090 MB | 1,024 MB | |
| Swap used | 0 MB | 594 MB | |
| Open files of API Maker | 28 | 795,920 | |
| Keys of Redis · channels it listens on | 13 · 5 | 1,591,792 · 795,893 |
MongoDB and the sandboxes grew during the runs before this one (the inserts, the code of the events), not with the clients.
What a client costs¶
Between 200,000 and 600,000 clients (5 levels), each client more cost the server 8.7 KB :
| Where | Per client | What it is |
|---|---|---|
| API Maker | 3.3 KB | The client itself : its socket and the one task which reads it, its tokens and what they said, its subscription and the channel it is told on. |
| Linux kernel | 4.4 KB | The TCP socket, and the entry of the connection in the table of the firewall (Docker makes Linux track every connection). |
| Redis | 1.1 KB | The subscription : its key, its place in the list of its condition, and the channel API Maker listens on. |
| In all | 8.7 KB | 120,250 clients per GB of memory. |
Measured over the levels at which nothing was in the swap. The memory the whole server had available went down by 8.0 KB per client : MongoDB, the sandboxes and the cache of files give some back while the clients take theirs.
One more subscription, and a broadcast to everybody¶
With 100,000 clients connected, every client sent a second REGISTER, for an event without a condition (announcements), 4,000 of them a second : 100,000 were taken in 25.0 s. They waited 30 seconds with two subscriptions each, were told 3 broadcasts, one call of the emit event WS API each, and took the subscription back with UNREGISTER.
One subscription more
1.2KB
per client : API Maker 0.5 KB · Redis 0.8 KB
A broadcast to 100k clients
1.39s
until the last client read it
Half of them told after
889ms
the first after 376 ms
Clients told / second
72,124
one call, every client
| Broadcast | Clients told | The call answered after | The first client | Half of them | 99 % of them | The last one | Clients told a second |
|---|---|---|---|---|---|---|---|
| 1 | 100,000 of 100,000 | 1.45 s | 453 ms | 942 ms | 1,435 ms | 1,447 ms | 69,113 |
| 2 | 100,000 of 100,000 | 1.39 s | 376 ms | 889 ms | 1,373 ms | 1,387 ms | 72,124 |
| 3 | 100,000 of 100,000 | 1.40 s | 365 ms | 861 ms | 1,390 ms | 1,402 ms | 71,334 |
One call of the emit event WS API tells every client : API Maker asks Redis for the clients of the event, publishes the notification for each of them, and the server which holds a client sends it. The times are counted from the moment the call is written to the moment a client reads its notification.
| Memory, 100,000 clients | One subscription each | Two subscriptions each | A subscription more, per client |
|---|---|---|---|
| API Maker | 440 MB | 484 MB | 0.5 KB |
| Redis (its data) | 103 MB | 175 MB | 0.8 KB |
| Keys of Redis · channels | 200,016 · 100,005 | 300,017 · 200,005 | |
| Whole server, less available | 1.2 KB |
New clients per second¶
How many clients a second the server lets in : 100 clients at a time connect, are let in and leave, one after the other, for 60 seconds (after 5 seconds which are not counted). It is what a server faces when its clients all come back, after a deployment or a network cut. The times are counted from the start of the TCP connection.
Connect : the token of the API user¶
A client connects with x-am-authorization alone, is told CONNECTED, and leaves. API Maker checks the token of the API user and finds its groups : what every WebSocket client costs before it asks for anything.
Clients / second
13,849
831,768 clients in 60 s, 100 at a time
Let in after, p50
4.28ms
connected 4.28 ms · TCP 0.28 ms
Let in after, p99
8.27ms
max 30.5 ms
Refused or lost
0
not connected, not subscribed, no answer
Server during this run API Maker 3.8 vCPU · MongoDB 0.0 vCPU · whole server busy 97 % · load generator busy 59 %
$ am_ws_load churn -c 100 -d 60 "ws://<server>:38246/?x-am-authorization=<token>"
Running 60s test @ ws://<server>:38246
100 clients at a time : connect, CONNECTED, leave
┌─────────────┬─────────┬─────────┬─────────┬──────────┬─────────┬─────────┬──────────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├─────────────┼─────────┼─────────┼─────────┼──────────┼─────────┼─────────┼──────────┤
│ TCP connect │ 0.19 ms │ 0.28 ms │ 1.02 ms │ 1.96 ms │ 0.36 ms │ 0.44 ms │ 20.85 ms │
├─────────────┼─────────┼─────────┼─────────┼──────────┼─────────┼─────────┼──────────┤
│ CONNECTED │ 1.5 ms │ 4.28 ms │ 7.49 ms │ 8.27 ms │ 4.31 ms │ 1.61 ms │ 30.5 ms │
├─────────────┼─────────┼─────────┼─────────┼──────────┼─────────┼─────────┼──────────┤
│ Left │ 3.82 ms │ 7.06 ms │ 11.2 ms │ 12.21 ms │ 7.18 ms │ 1.95 ms │ 38.56 ms │
└─────────────┴─────────┴─────────┴─────────┴──────────┴─────────┴─────────┴──────────┘
┌─────────────┬────────┬────────┬────────┬────────┬──────────┬────────┬────────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├─────────────┼────────┼────────┼────────┼────────┼──────────┼────────┼────────┤
│ Clients/Sec │ 13,166 │ 13,341 │ 13,865 │ 14,464 │ 13,862.8 │ 261.17 │ 13,166 │
└─────────────┴────────┴────────┴────────┴────────┴──────────┴────────┴────────┘
Time from the start of the connection to each step. Clients counted once per second.
# of samples: 60
832k clients let in in 60.06s, 0 refused or lost
Connect : both tokens¶
A client connects with x-am-authorization and x-am-user-authorization, is told CONNECTED, and leaves. Both tokens are checked : the one of the API user, and the one of a person, made by the DB token generator of the account.
Clients / second
12,535
752,875 clients in 60 s, 100 at a time
Let in after, p50
4.73ms
connected 4.73 ms · TCP 0.28 ms
Let in after, p99
9.16ms
max 26.4 ms
Refused or lost
0
not connected, not subscribed, no answer
Server during this run API Maker 3.8 vCPU · MongoDB 0.0 vCPU · whole server busy 98 % · load generator busy 54 %
$ am_ws_load churn -c 100 -d 60 "ws://<server>:38246/?x-am-authorization=<token>&x-am-user-authorization=<token>"
Running 60s test @ ws://<server>:38246
100 clients at a time : connect, CONNECTED, leave
┌─────────────┬─────────┬─────────┬──────────┬──────────┬─────────┬─────────┬──────────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├─────────────┼─────────┼─────────┼──────────┼──────────┼─────────┼─────────┼──────────┤
│ TCP connect │ 0.19 ms │ 0.28 ms │ 0.86 ms │ 1.58 ms │ 0.34 ms │ 0.36 ms │ 14.77 ms │
├─────────────┼─────────┼─────────┼──────────┼──────────┼─────────┼─────────┼──────────┤
│ CONNECTED │ 1.61 ms │ 4.73 ms │ 8.38 ms │ 9.16 ms │ 4.75 ms │ 1.8 ms │ 26.37 ms │
├─────────────┼─────────┼─────────┼──────────┼──────────┼─────────┼─────────┼──────────┤
│ Left │ 4.15 ms │ 7.83 ms │ 12.35 ms │ 13.34 ms │ 7.94 ms │ 2.13 ms │ 27.84 ms │
└─────────────┴─────────┴─────────┴──────────┴──────────┴─────────┴─────────┴──────────┘
┌─────────────┬────────┬────────┬────────┬────────┬──────────┬────────┬────────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├─────────────┼────────┼────────┼────────┼────────┼──────────┼────────┼────────┤
│ Clients/Sec │ 12,056 │ 12,143 │ 12,502 │ 13,387 │ 12,547.9 │ 364.66 │ 12,056 │
└─────────────┴────────┴────────┴────────┴────────┴──────────┴────────┴────────┘
Time from the start of the connection to each step. Clients counted once per second.
# of samples: 60
753k clients let in in 60.06s, 0 refused or lost
Connect with both tokens and subscribe¶
The whole of what an application does : connect with both tokens, CONNECTED, one REGISTER for a custom WebSocket event with a condition of its own (recipientId : its user id), the answer, then leave. API Maker checks the groups and the auth provider of the event, keeps the subscription in Redis and listens on its channel ; when the client leaves, all of it is taken back.
Clients / second
6,627
397,988 clients in 60 s, 100 at a time
Let in after, p50
13.4ms
connected 3.13 ms · TCP 0.25 ms
Let in after, p99
19.3ms
max 27.6 ms
Refused or lost
0
not connected, not subscribed, no answer
Server during this run API Maker 3.0 vCPU · Redis 0.4 vCPU · MongoDB 0.0 vCPU · whole server busy 96 % · load generator busy 34 %
$ am_ws_load churn -c 100 -d 60 "ws://<server>:38246/?x-am-authorization=<token>&x-am-user-authorization=<token>" --register 'order-updates'
Running 60s test @ ws://<server>:38246
100 clients at a time : connect, CONNECTED, REGISTER, leave
┌─────────────┬──────────┬──────────┬──────────┬──────────┬──────────┬─────────┬──────────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├─────────────┼──────────┼──────────┼──────────┼──────────┼──────────┼─────────┼──────────┤
│ TCP connect │ 0.18 ms │ 0.25 ms │ 0.49 ms │ 0.65 ms │ 0.27 ms │ 0.17 ms │ 10.91 ms │
├─────────────┼──────────┼──────────┼──────────┼──────────┼──────────┼─────────┼──────────┤
│ CONNECTED │ 0.93 ms │ 3.13 ms │ 6.56 ms │ 7.28 ms │ 3.3 ms │ 1.5 ms │ 16.78 ms │
├─────────────┼──────────┼──────────┼──────────┼──────────┼──────────┼─────────┼──────────┤
│ REGISTER │ 8.33 ms │ 13.37 ms │ 18.22 ms │ 19.34 ms │ 13.33 ms │ 2.46 ms │ 27.58 ms │
├─────────────┼──────────┼──────────┼──────────┼──────────┼──────────┼─────────┼──────────┤
│ Left │ 10.11 ms │ 14.97 ms │ 20.46 ms │ 21.66 ms │ 15.06 ms │ 2.59 ms │ 32.61 ms │
└─────────────┴──────────┴──────────┴──────────┴──────────┴──────────┴─────────┴──────────┘
┌─────────────┬───────┬───────┬───────┬───────┬─────────┬────────┬───────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├─────────────┼───────┼───────┼───────┼───────┼─────────┼────────┼───────┤
│ Clients/Sec │ 6,382 │ 6,399 │ 6,618 │ 6,907 │ 6,633.1 │ 126.97 │ 6,382 │
└─────────────┴───────┴───────┴───────┴───────┴─────────┴────────┴───────┘
Time from the start of the connection to each step. Clients counted once per second.
# of samples: 60
398k clients let in in 60.06s, 0 refused or lost
Subscribe to an event with a can user connect code¶
The same, on an event whose can user connect code decides who may subscribe : it reads the person of x-am-user-authorization from g.req.auth. The code ran in the server itself (lean code, QuickJS).
Clients / second
5,533
332,290 clients in 60 s, 100 at a time
Let in after, p50
16.0ms
connected 3.53 ms · TCP 0.24 ms
Let in after, p99
23.1ms
max 37.8 ms
Refused or lost
0
not connected, not subscribed, no answer
Server during this run API Maker 3.1 vCPU · Redis 0.3 vCPU · MongoDB 0.0 vCPU · whole server busy 98 % · load generator busy 30 %
$ am_ws_load churn -c 100 -d 60 "ws://<server>:38246/?x-am-authorization=<token>&x-am-user-authorization=<token>" --register 'checked-token'
Running 60s test @ ws://<server>:38246
100 clients at a time : connect, CONNECTED, REGISTER, leave
┌─────────────┬──────────┬──────────┬──────────┬──────────┬──────────┬─────────┬──────────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├─────────────┼──────────┼──────────┼──────────┼──────────┼──────────┼─────────┼──────────┤
│ TCP connect │ 0.18 ms │ 0.24 ms │ 0.78 ms │ 2.11 ms │ 0.31 ms │ 0.46 ms │ 18.22 ms │
├─────────────┼──────────┼──────────┼──────────┼──────────┼──────────┼─────────┼──────────┤
│ CONNECTED │ 1.1 ms │ 3.53 ms │ 7.36 ms │ 8.39 ms │ 3.71 ms │ 1.69 ms │ 28.42 ms │
├─────────────┼──────────┼──────────┼──────────┼──────────┼──────────┼─────────┼──────────┤
│ REGISTER │ 9.98 ms │ 16.03 ms │ 21.78 ms │ 23.12 ms │ 15.98 ms │ 2.96 ms │ 37.76 ms │
├─────────────┼──────────┼──────────┼──────────┼──────────┼──────────┼─────────┼──────────┤
│ Left │ 12.13 ms │ 17.98 ms │ 24.29 ms │ 25.78 ms │ 18.05 ms │ 3.06 ms │ 43.42 ms │
└─────────────┴──────────┴──────────┴──────────┴──────────┴──────────┴─────────┴──────────┘
┌─────────────┬───────┬───────┬───────┬───────┬─────────┬───────┬───────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├─────────────┼───────┼───────┼───────┼───────┼─────────┼───────┼───────┤
│ Clients/Sec │ 5,375 │ 5,376 │ 5,524 │ 5,724 │ 5,538.2 │ 94.33 │ 5,375 │
└─────────────┴───────┴───────┴───────┴───────┴─────────┴───────┴───────┘
Time from the start of the connection to each step. Clients counted once per second.
# of samples: 60
332k clients let in in 60.06s, 0 refused or lost
Subscribe : the code reads the database¶
The same, with a can user connect code which also reads the person from the database (g.sys.db.getAll with a find) before it answers : one read of MongoDB for every subscription. The code ran in a sandbox of the account.
Clients / second
2,046
122,868 clients in 60 s, 100 at a time
Let in after, p50
45.0ms
connected 6.05 ms · TCP 0.23 ms
Let in after, p99
69.3ms
max 109 ms
Refused or lost
0
not connected, not subscribed, no answer
Server during this run API Maker 1.4 vCPU · Redis 0.2 vCPU · MongoDB 0.6 vCPU · sandboxes 1.6 vCPU · whole server busy 99 % · load generator busy 11 %
$ am_ws_load churn -c 100 -d 60 "ws://<server>:38246/?x-am-authorization=<token>&x-am-user-authorization=<token>" --register 'checked-database'
Running 60s test @ ws://<server>:38246
100 clients at a time : connect, CONNECTED, REGISTER, leave
┌─────────────┬──────────┬──────────┬──────────┬──────────┬──────────┬─────────┬───────────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├─────────────┼──────────┼──────────┼──────────┼──────────┼──────────┼─────────┼───────────┤
│ TCP connect │ 0.17 ms │ 0.23 ms │ 0.46 ms │ 0.57 ms │ 0.26 ms │ 0.3 ms │ 35.62 ms │
├─────────────┼──────────┼──────────┼──────────┼──────────┼──────────┼─────────┼───────────┤
│ CONNECTED │ 1.78 ms │ 6.05 ms │ 13.03 ms │ 14.74 ms │ 6.4 ms │ 2.95 ms │ 44.93 ms │
├─────────────┼──────────┼──────────┼──────────┼──────────┼──────────┼─────────┼───────────┤
│ REGISTER │ 29.15 ms │ 45.02 ms │ 64.86 ms │ 69.31 ms │ 45.53 ms │ 9.11 ms │ 108.86 ms │
├─────────────┼──────────┼──────────┼──────────┼──────────┼──────────┼─────────┼───────────┤
│ Left │ 32.11 ms │ 48.35 ms │ 68.54 ms │ 73.28 ms │ 48.86 ms │ 9.32 ms │ 120.7 ms │
└─────────────┴──────────┴──────────┴──────────┴──────────┴──────────┴─────────┴───────────┘
┌─────────────┬───────┬───────┬───────┬───────┬─────────┬───────┬───────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├─────────────┼───────┼───────┼───────┼───────┼─────────┼───────┼───────┤
│ Clients/Sec │ 1,940 │ 1,946 │ 2,049 │ 2,157 │ 2,047.8 │ 50.61 │ 1,940 │
└─────────────┴───────┴───────┴───────┴───────┴─────────┴───────┴───────┘
Time from the start of the connection to each step. Clients counted once per second.
# of samples: 60
123k clients let in in 60.06s, 0 refused or lost
Subscribe : the code reads the clock¶
The same, with a can user connect code which reads the time of the machine (Date.now()) : code which is not lean, so API Maker runs it in a sandbox of the account. The code ran in a sandbox of the account.
Clients / second
4,505
270,567 clients in 60 s, 100 at a time
Let in after, p50
20.1ms
connected 3.48 ms · TCP 0.23 ms
Let in after, p99
30.7ms
max 45.7 ms
Refused or lost
0
not connected, not subscribed, no answer
Server during this run API Maker 2.5 vCPU · Redis 0.3 vCPU · MongoDB 0.0 vCPU · sandboxes 0.8 vCPU · whole server busy 98 % · load generator busy 23 %
$ am_ws_load churn -c 100 -d 60 "ws://<server>:38246/?x-am-authorization=<token>&x-am-user-authorization=<token>" --register 'checked-sandbox'
Running 60s test @ ws://<server>:38246
100 clients at a time : connect, CONNECTED, REGISTER, leave
┌─────────────┬──────────┬──────────┬──────────┬──────────┬──────────┬─────────┬──────────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├─────────────┼──────────┼──────────┼──────────┼──────────┼──────────┼─────────┼──────────┤
│ TCP connect │ 0.17 ms │ 0.23 ms │ 0.5 ms │ 0.82 ms │ 0.27 ms │ 0.25 ms │ 12.62 ms │
├─────────────┼──────────┼──────────┼──────────┼──────────┼──────────┼─────────┼──────────┤
│ CONNECTED │ 1.19 ms │ 3.48 ms │ 7.08 ms │ 8.01 ms │ 3.64 ms │ 1.55 ms │ 21.76 ms │
├─────────────┼──────────┼──────────┼──────────┼──────────┼──────────┼─────────┼──────────┤
│ REGISTER │ 12.63 ms │ 20.13 ms │ 28.72 ms │ 30.69 ms │ 20.26 ms │ 4.05 ms │ 45.7 ms │
├─────────────┼──────────┼──────────┼──────────┼──────────┼──────────┼─────────┼──────────┤
│ Left │ 14.56 ms │ 21.97 ms │ 30.88 ms │ 32.77 ms │ 22.17 ms │ 4.13 ms │ 48.45 ms │
└─────────────┴──────────┴──────────┴──────────┴──────────┴──────────┴─────────┴──────────┘
┌─────────────┬───────┬───────┬───────┬───────┬─────────┬────────┬───────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├─────────────┼───────┼───────┼───────┼───────┼─────────┼────────┼───────┤
│ Clients/Sec │ 4,270 │ 4,281 │ 4,533 │ 4,746 │ 4,509.4 │ 136.13 │ 4,270 │
└─────────────┴───────┴───────┴───────┴───────┴─────────┴────────┴───────┘
Time from the start of the connection to each step. Clients counted once per second.
# of samples: 60
271k clients let in in 60.06s, 0 refused or lost
Can user connect code¶
An event of the WebSocket Events page can carry a can user connect code : API Maker runs it for every subscription to the event, with the tokens of the client in g.req.auth and the subscription in g.req.eventData, and lets the client subscribe when it returns { canConnect: true }. Three codes were measured : one which reads the person of the token, one which also reads the database, one which reads the clock.
Can user connect code of the event checked-token : ran in the server
import * as T from 'types';
async function main(g: T.IAMGlobal) {
// The person of x-am-user-authorization : only somebody who signed in subscribes, from a country we serve.
const person = g.req.auth.authAMDB;
if (!person) return { canConnect: false, errorText: 'Sign in first.' };
return { canConnect: person.country === 'India', errorText: 'Not available in your country.' };
};
module.exports = main;
Can user connect code of the event checked-database : ran in the sandbox
import * as T from 'types';
async function main(g: T.IAMGlobal) {
const person = g.req.auth.authAMDB;
if (!person) return { canConnect: false, errorText: 'Sign in first.' };
// Still a customer ? One read of the database for every subscription.
const rows = await g.sys.db.getAll({ instance: 'mongodb', database: 'am_bench', collection: 'customers', queryParams: { find: { customer_id: person.customer_id } } });
return { canConnect: rows.length === 1 && rows[0].country === 'India', errorText: 'Not a customer any more.' };
};
module.exports = main;
Can user connect code of the event checked-sandbox : ran in the sandbox
import * as T from 'types';
async function main(g: T.IAMGlobal) {
const person = g.req.auth.authAMDB;
if (!person) return { canConnect: false, errorText: 'Sign in first.' };
// The time of the machine : this code runs in the sandbox of the account.
const hour = new Date(Date.now()).getUTCHours();
return { canConnect: person.country === 'India' && hour >= 0, errorText: 'Not available in your country.' };
};
module.exports = main;
| Can user connect code | Runs in | Clients / second | Subscribed after, p50 | p99 | Against no code |
|---|---|---|---|---|---|
| No code | – | 6,627 | 13.4 ms | 19.3 ms | – |
| Reads the person of the token | the server (QuickJS) | 5,533 | 16.0 ms | 23.1 ms | 17 % fewer |
| Reads the person from the database | a sandbox of the account | 2,046 | 45.0 ms | 69.3 ms | 69 % fewer |
| Reads the clock | a sandbox of the account | 4,505 | 20.1 ms | 30.7 ms | 32 % fewer |
Lean code (it only reads what it is handed) runs in the server itself, with QuickJS, on the thread which serves the client. Code which needs more runs in a sandbox of the account : every subscription is then a round trip to the sandbox.
Notifications¶
With 10,000 clients connected, autocannon makes the HTTP calls which notify them, for 60 seconds, and the clients count what they receive and how late. Each kind of notification is measured twice : at a pace far under what the server takes, which shows how long a notification takes, and with no pace, which shows how many a second the server tells. Delivery is the time from the moment autocannon writes the request to the moment the client reads the notification : the HTTP call, API Maker, Redis, and the WebSocket.
A notification for one person¶
autocannon calls the emit event WS API with both tokens ; every call names another one of the 10,000 connected clients (recipientId), which is the only one told. The clients measure how long each notification took from the moment its request was written.
Calls / second (paced)
1,000
60,000 calls in 60 s, 10 connections
Notifications / second
991
60,010 received by 10,000 clients
Delivery p50
1.04ms
average 1.09 ms
Delivery p99
2.03ms
max 6.07 ms
Not delivered
0
0 calls failed
Server during this run API Maker 0.3 vCPU · Redis 0.0 vCPU · whole server busy 10 % · load generator busy 8 %
$ autocannon -c 10 -d 60 -R 1000 -l -m POST -b '{"name":"order-updates","eventData":{"recipientId":"<id of a client>","t":<now>,"type":"order-updated","order_no":1001,"status":"SHIPPED"}}' -H "x-am-authorization: <token>" -H "x-am-user-authorization: <token>" -H "content-type: application/json" "http://<server>:38246/api/system-api/bench/emit-event-ws"
Running 60s test @ http://<server>:38246/api/system-api/bench/emit-event-ws
10 connections
┌─────────┬──────┬──────┬───────┬──────┬─────────┬─────────┬──────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├─────────┼──────┼──────┼───────┼──────┼─────────┼─────────┼──────┤
│ Latency │ 0 ms │ 1 ms │ 2 ms │ 2 ms │ 0.84 ms │ 0.47 ms │ 7 ms │
└─────────┴──────┴──────┴───────┴──────┴─────────┴─────────┴──────┘
┌───────────┬────────┬────────┬────────┬────────┬────────┬───────┬────────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼────────┼────────┼────────┼────────┼────────┼───────┼────────┤
│ Req/Sec │ 995 │ 1,000 │ 1,000 │ 1,000 │ 1,000 │ 0.92 │ 995 │
├───────────┼────────┼────────┼────────┼────────┼────────┼───────┼────────┤
│ Bytes/Sec │ 355 kB │ 357 kB │ 357 kB │ 357 kB │ 357 kB │ 327 B │ 355 kB │
└───────────┴────────┴────────┴────────┴────────┴────────┴───────┴────────┘
Req/Bytes counts sampled once per second.
# of samples: 60
┌────────────┬──────────────┐
│ Percentile │ Latency (ms) │
├────────────┼──────────────┤
│ 0.001 │ 0 │
├────────────┼──────────────┤
│ 0.01 │ 0 │
├────────────┼──────────────┤
│ 0.1 │ 0 │
├────────────┼──────────────┤
│ 1 │ 0 │
├────────────┼──────────────┤
│ 2.5 │ 0 │
├────────────┼──────────────┤
│ 10 │ 0 │
├────────────┼──────────────┤
│ 25 │ 1 │
├────────────┼──────────────┤
│ 50 │ 1 │
├────────────┼──────────────┤
│ 75 │ 1 │
├────────────┼──────────────┤
│ 90 │ 1 │
├────────────┼──────────────┤
│ 97.5 │ 2 │
├────────────┼──────────────┤
│ 99 │ 2 │
├────────────┼──────────────┤
│ 99.9 │ 4 │
├────────────┼──────────────┤
│ 99.99 │ 6 │
├────────────┼──────────────┤
│ 99.999 │ 7 │
└────────────┴──────────────┘
61k requests in 60.05s, 21.4 MB read
$ am_ws_load listen "ws://<server>:38246/?x-am-authorization=<token>&x-am-user-authorization=<token>"
10,000 clients subscribed, 63.57s
┌──────────┬─────────┬─────────┬─────────┬─────────┬─────────┬────────┬─────────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├──────────┼─────────┼─────────┼─────────┼─────────┼─────────┼────────┼─────────┤
│ Delivery │ 0.69 ms │ 1.04 ms │ 1.81 ms │ 2.03 ms │ 1.09 ms │ 0.3 ms │ 6.07 ms │
└──────────┴─────────┴─────────┴─────────┴─────────┴─────────┴────────┴─────────┘
┌───────────┬────┬──────┬───────┬───────┬───────┬────────┬─────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼────┼──────┼───────┼───────┼───────┼────────┼─────┤
│ Notif/Sec │ 0 │ 0 │ 1,000 │ 1,000 │ 951.8 │ 213.87 │ 0 │
└───────────┴────┴──────┴───────┴───────┴───────┴────────┴─────┘
Delivery : from the moment the request is written to the moment the client reads the notification.
# of samples: 64
60k notifications received in 63.57s (60,010 of 60,000 expected), 10.7 MB read
Calls / second
13,049
782,930 calls in 60 s, 100 connections
Notifications / second
12,791
783,029 received by 10,000 clients
Delivery p50
7.43ms
average 7.48 ms
Delivery p99
11.6ms
max 25.8 ms
Not delivered
0
0 calls failed
Server during this run API Maker 3.3 vCPU · Redis 0.2 vCPU · MongoDB 0.0 vCPU · whole server busy 96 % · load generator busy 32 %
$ autocannon -c 100 -d 60 -w 5 -l -m POST -b '{"name":"order-updates","eventData":{"recipientId":"<id of a client>","t":<now>,"type":"order-updated","order_no":1001,"status":"SHIPPED"}}' -H "x-am-authorization: <token>" -H "x-am-user-authorization: <token>" -H "content-type: application/json" "http://<server>:38246/api/system-api/bench/emit-event-ws"
Running 60s test @ http://<server>:38246/api/system-api/bench/emit-event-ws
100 connections
5 workers
┌─────────┬──────┬──────┬───────┬───────┬─────────┬─────────┬───────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├─────────┼──────┼──────┼───────┼───────┼─────────┼─────────┼───────┤
│ Latency │ 4 ms │ 7 ms │ 10 ms │ 11 ms │ 7.13 ms │ 1.56 ms │ 25 ms │
└─────────┴──────┴──────┴───────┴───────┴─────────┴─────────┴───────┘
┌───────────┬─────────┬─────────┬─────────┬─────────┬───────────┬────────┬─────────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼─────────┼─────────┼─────────┼─────────┼───────────┼────────┼─────────┤
│ Req/Sec │ 10,983 │ 12,495 │ 13,071 │ 13,567 │ 13,048.94 │ 359.71 │ 10,982 │
├───────────┼─────────┼─────────┼─────────┼─────────┼───────────┼────────┼─────────┤
│ Bytes/Sec │ 3.92 MB │ 4.46 MB │ 4.67 MB │ 4.85 MB │ 4.66 MB │ 128 kB │ 3.92 MB │
└───────────┴─────────┴─────────┴─────────┴─────────┴───────────┴────────┴─────────┘
Req/Bytes counts sampled once per second.
# of samples: 300
┌────────────┬──────────────┐
│ Percentile │ Latency (ms) │
├────────────┼──────────────┤
│ 0.001 │ 1 │
├────────────┼──────────────┤
│ 0.01 │ 1 │
├────────────┼──────────────┤
│ 0.1 │ 3 │
├────────────┼──────────────┤
│ 1 │ 4 │
├────────────┼──────────────┤
│ 2.5 │ 4 │
├────────────┼──────────────┤
│ 10 │ 5 │
├────────────┼──────────────┤
│ 25 │ 6 │
├────────────┼──────────────┤
│ 50 │ 7 │
├────────────┼──────────────┤
│ 75 │ 8 │
├────────────┼──────────────┤
│ 90 │ 9 │
├────────────┼──────────────┤
│ 97.5 │ 10 │
├────────────┼──────────────┤
│ 99 │ 11 │
├────────────┼──────────────┤
│ 99.9 │ 15 │
├────────────┼──────────────┤
│ 99.99 │ 20 │
├────────────┼──────────────┤
│ 99.999 │ 23 │
└────────────┴──────────────┘
783k requests in 60.02s, 280 MB read
$ am_ws_load listen "ws://<server>:38246/?x-am-authorization=<token>&x-am-user-authorization=<token>"
10,000 clients subscribed, 64.22s
┌──────────┬─────────┬─────────┬──────────┬──────────┬─────────┬─────────┬──────────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├──────────┼─────────┼─────────┼──────────┼──────────┼─────────┼─────────┼──────────┤
│ Delivery │ 4.62 ms │ 7.43 ms │ 10.66 ms │ 11.55 ms │ 7.48 ms │ 1.52 ms │ 25.84 ms │
└──────────┴─────────┴─────────┴──────────┴──────────┴─────────┴─────────┴──────────┘
┌───────────┬────┬──────┬────────┬────────┬──────────┬─────────┬─────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼────┼──────┼────────┼────────┼──────────┼─────────┼─────┤
│ Notif/Sec │ 0 │ 0 │ 13,067 │ 14,016 │ 12,508.3 │ 2475.61 │ 0 │
└───────────┴────┴──────┴────────┴────────┴──────────┴─────────┴─────┘
Delivery : from the moment the request is written to the moment the client reads the notification.
# of samples: 64
783k notifications received in 64.22s (783,029 of 782,930 expected), 140.2 MB read
A broadcast¶
1,000 more clients subscribe to an event without a condition : every call of the emit event WS API reaches all 1,000 of them.
Calls / second (paced)
5
300 calls in 60 s, 1 connection
Notifications / second
4,972
301,000 received by 1,000 clients
Delivery p50
8.00ms
average 8.50 ms
Delivery p99
17.3ms
max 35.4 ms
Not delivered
0
0 calls failed
Server during this run API Maker 0.1 vCPU · Redis 0.0 vCPU · whole server busy 5 % · load generator busy 6 %
$ autocannon -c 1 -d 60 -R 5 -l -m POST -b '{"name":"announcements","eventData":{"t":<now>,"type":"announcement","text":"Hello from API Maker"}}' -H "x-am-authorization: <token>" -H "x-am-user-authorization: <token>" -H "content-type: application/json" "http://<server>:38246/api/system-api/bench/emit-event-ws"
Running 60s test @ http://<server>:38246/api/system-api/bench/emit-event-ws
1 connections
┌─────────┬──────┬──────┬───────┬───────┬─────────┬─────────┬───────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├─────────┼──────┼──────┼───────┼───────┼─────────┼─────────┼───────┤
│ Latency │ 1 ms │ 8 ms │ 16 ms │ 18 ms │ 7.74 ms │ 4.46 ms │ 35 ms │
└─────────┴──────┴──────┴───────┴───────┴─────────┴─────────┴───────┘
┌───────────┬────────┬────────┬────────┬────────┬────────┬───────┬────────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼────────┼────────┼────────┼────────┼────────┼───────┼────────┤
│ Req/Sec │ 5 │ 5 │ 5 │ 5 │ 5 │ 0 │ 5 │
├───────────┼────────┼────────┼────────┼────────┼────────┼───────┼────────┤
│ Bytes/Sec │ 1.8 kB │ 1.8 kB │ 1.8 kB │ 1.8 kB │ 1.8 kB │ 0 B │ 1.8 kB │
└───────────┴────────┴────────┴────────┴────────┴────────┴───────┴────────┘
Req/Bytes counts sampled once per second.
# of samples: 60
┌────────────┬──────────────┐
│ Percentile │ Latency (ms) │
├────────────┼──────────────┤
│ 0.001 │ 1 │
├────────────┼──────────────┤
│ 0.01 │ 1 │
├────────────┼──────────────┤
│ 0.1 │ 1 │
├────────────┼──────────────┤
│ 1 │ 1 │
├────────────┼──────────────┤
│ 2.5 │ 1 │
├────────────┼──────────────┤
│ 10 │ 2 │
├────────────┼──────────────┤
│ 25 │ 4 │
├────────────┼──────────────┤
│ 50 │ 8 │
├────────────┼──────────────┤
│ 75 │ 11 │
├────────────┼──────────────┤
│ 90 │ 13 │
├────────────┼──────────────┤
│ 97.5 │ 16 │
├────────────┼──────────────┤
│ 99 │ 18 │
├────────────┼──────────────┤
│ 99.9 │ 31 │
├────────────┼──────────────┤
│ 99.99 │ 35 │
├────────────┼──────────────┤
│ 99.999 │ 35 │
└────────────┴──────────────┘
305 requests in 60.04s, 108 kB read
$ am_ws_load listen "ws://<server>:38246/?x-am-authorization=<token>&x-am-user-authorization=<token>"
1,000 clients subscribed, 63.55s
┌──────────┬────────┬──────┬──────────┬──────────┬────────┬─────────┬──────────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├──────────┼────────┼──────┼──────────┼──────────┼────────┼─────────┼──────────┤
│ Delivery │ 3.9 ms │ 8 ms │ 15.35 ms │ 17.25 ms │ 8.5 ms │ 3.36 ms │ 35.42 ms │
└──────────┴────────┴──────┴──────────┴──────────┴────────┴─────────┴──────────┘
┌───────────┬────┬──────┬───────┬───────┬─────────┬─────────┬─────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼────┼──────┼───────┼───────┼─────────┼─────────┼─────┤
│ Notif/Sec │ 0 │ 0 │ 5,000 │ 5,000 │ 4,774.2 │ 1006.74 │ 0 │
└───────────┴────┴──────┴───────┴───────┴─────────┴─────────┴─────┘
Delivery : from the moment the request is written to the moment the client reads the notification.
# of samples: 64
301k notifications received in 63.55s (301,000 of 300,000 expected), 46.2 MB read
Calls / second
120
7,196 calls in 60 s, 10 connections
Notifications / second
117,698
7,205,000 received by 1,000 clients
Delivery p50
73.4ms
average 72.6 ms
Delivery p99
114ms
max 150 ms
Not delivered
0
0 calls failed
Server during this run API Maker 3.4 vCPU · Redis 0.2 vCPU · MongoDB 0.0 vCPU · whole server busy 94 % · load generator busy 41 %
$ autocannon -c 10 -d 60 -w 5 -l -m POST -b '{"name":"announcements","eventData":{"t":<now>,"type":"announcement","text":"Hello from API Maker"}}' -H "x-am-authorization: <token>" -H "x-am-user-authorization: <token>" -H "content-type: application/json" "http://<server>:38246/api/system-api/bench/emit-event-ws"
Running 60s test @ http://<server>:38246/api/system-api/bench/emit-event-ws
10 connections
5 workers
┌─────────┬───────┬───────┬────────┬────────┬──────────┬──────────┬────────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├─────────┼───────┼───────┼────────┼────────┼──────────┼──────────┼────────┤
│ Latency │ 48 ms │ 83 ms │ 116 ms │ 125 ms │ 82.75 ms │ 16.48 ms │ 278 ms │
└─────────┴───────┴───────┴────────┴────────┴──────────┴──────────┴────────┘
┌───────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Req/Sec │ 109 │ 110 │ 118 │ 143 │ 119.94 │ 9.62 │ 109 │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Bytes/Sec │ 39.3 kB │ 39.6 kB │ 42.5 kB │ 51.5 kB │ 43.2 kB │ 3.46 kB │ 39.2 kB │
└───────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┘
Req/Bytes counts sampled once per second.
# of samples: 300
┌────────────┬──────────────┐
│ Percentile │ Latency (ms) │
├────────────┼──────────────┤
│ 0.001 │ 21 │
├────────────┼──────────────┤
│ 0.01 │ 21 │
├────────────┼──────────────┤
│ 0.1 │ 28 │
├────────────┼──────────────┤
│ 1 │ 42 │
├────────────┼──────────────┤
│ 2.5 │ 48 │
├────────────┼──────────────┤
│ 10 │ 62 │
├────────────┼──────────────┤
│ 25 │ 74 │
├────────────┼──────────────┤
│ 50 │ 83 │
├────────────┼──────────────┤
│ 75 │ 92 │
├────────────┼──────────────┤
│ 90 │ 102 │
├────────────┼──────────────┤
│ 97.5 │ 116 │
├────────────┼──────────────┤
│ 99 │ 125 │
├────────────┼──────────────┤
│ 99.9 │ 145 │
├────────────┼──────────────┤
│ 99.99 │ 278 │
├────────────┼──────────────┤
│ 99.999 │ 278 │
└────────────┴──────────────┘
7k requests in 60.03s, 2.59 MB read
$ am_ws_load listen "ws://<server>:38246/?x-am-authorization=<token>&x-am-user-authorization=<token>"
1,000 clients subscribed, 64.22s
┌──────────┬─────────┬──────────┬───────────┬───────────┬──────────┬──────────┬───────────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├──────────┼─────────┼──────────┼───────────┼───────────┼──────────┼──────────┼───────────┤
│ Delivery │ 38.4 ms │ 73.41 ms │ 106.37 ms │ 114.24 ms │ 72.62 ms │ 16.71 ms │ 150.02 ms │
└──────────┴─────────┴──────────┴───────────┴───────────┴──────────┴──────────┴───────────┘
┌───────────┬────┬──────┬─────────┬─────────┬───────────┬──────────┬─────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼────┼──────┼─────────┼─────────┼───────────┼──────────┼─────┤
│ Notif/Sec │ 0 │ 0 │ 117,405 │ 156,280 │ 114,703.7 │ 25239.61 │ 0 │
└───────────┴────┴──────┴─────────┴─────────┴───────────┴──────────┴─────┘
Delivery : from the moment the request is written to the moment the client reads the notification.
# of samples: 64
7.21M notifications received in 64.22s (7,205,000 of 7,196,000 expected), 1105.5 MB read
Rows inserted in a table¶
100 clients subscribe to the insert API of a table (SCHEMA_POST_BULK_INSERT on inserts_public, getEventData: true) ; autocannon inserts rows of the dataset through the insert API, and every row saved is told to each of the 100 clients, with its columns.
Calls / second (paced)
50
3,000 calls in 60 s, 10 connections
Notifications / second
4,930
301,000 received by 100 clients
Delivery p50
20.7ms
average 20.6 ms
Delivery p99
34.5ms
max 43.2 ms
Not delivered
0
0 calls failed
Server during this run API Maker 0.2 vCPU · Redis 0.0 vCPU · MongoDB 0.1 vCPU · whole server busy 7 % · load generator busy 7 %
$ autocannon -c 10 -d 60 -R 50 -l -m POST -b '{"customer_id":1,"first_name":"BENCH","last_name":"INSERT","phone":<now>,"last_update":"2006-02-15T04:34:00.330Z","pincode":111111,"shipping_id":2,"dob":"2022-10-30T03:25:03.000Z","country":"India"}' -H "content-type: application/json" "http://<server>:38246/api/schema/bench/mongodb/am_bench/inserts_public/save-single-or-multiple"
Running 60s test @ http://<server>:38246/api/schema/bench/mongodb/am_bench/inserts_public/save-single-or-multiple
10 connections
┌─────────┬──────┬──────┬───────┬───────┬──────┬────────┬───────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├─────────┼──────┼──────┼───────┼───────┼──────┼────────┼───────┤
│ Latency │ 1 ms │ 6 ms │ 17 ms │ 19 ms │ 7 ms │ 4.5 ms │ 30 ms │
└─────────┴──────┴──────┴───────┴───────┴──────┴────────┴───────┘
┌───────────┬─────────┬─────────┬─────────┬─────────┬─────────┬────────┬─────────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼────────┼─────────┤
│ Req/Sec │ 50 │ 50 │ 50 │ 50 │ 50 │ 0 │ 50 │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼────────┼─────────┤
│ Bytes/Sec │ 30.2 kB │ 30.2 kB │ 30.3 kB │ 30.3 kB │ 30.2 kB │ 2.88 B │ 30.2 kB │
└───────────┴─────────┴─────────┴─────────┴─────────┴─────────┴────────┴─────────┘
Req/Bytes counts sampled once per second.
# of samples: 60
┌────────────┬──────────────┐
│ Percentile │ Latency (ms) │
├────────────┼──────────────┤
│ 0.001 │ 1 │
├────────────┼──────────────┤
│ 0.01 │ 1 │
├────────────┼──────────────┤
│ 0.1 │ 1 │
├────────────┼──────────────┤
│ 1 │ 1 │
├────────────┼──────────────┤
│ 2.5 │ 1 │
├────────────┼──────────────┤
│ 10 │ 2 │
├────────────┼──────────────┤
│ 25 │ 3 │
├────────────┼──────────────┤
│ 50 │ 6 │
├────────────┼──────────────┤
│ 75 │ 10 │
├────────────┼──────────────┤
│ 90 │ 13 │
├────────────┼──────────────┤
│ 97.5 │ 17 │
├────────────┼──────────────┤
│ 99 │ 19 │
├────────────┼──────────────┤
│ 99.9 │ 24 │
├────────────┼──────────────┤
│ 99.99 │ 29 │
├────────────┼──────────────┤
│ 99.999 │ 30 │
└────────────┴──────────────┘
3k requests in 60.04s, 1.81 MB read
$ am_ws_load listen "ws://<server>:38246/?x-am-authorization=<token>&x-am-user-authorization=<token>"
100 clients subscribed, 64.06s
┌──────────┬─────────┬─────────┬──────────┬──────────┬──────────┬─────────┬──────────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├──────────┼─────────┼─────────┼──────────┼──────────┼──────────┼─────────┼──────────┤
│ Delivery │ 9.04 ms │ 20.7 ms │ 31.92 ms │ 34.53 ms │ 20.58 ms │ 5.76 ms │ 43.23 ms │
└──────────┴─────────┴─────────┴──────────┴──────────┴──────────┴─────────┴──────────┘
┌───────────┬────┬──────┬───────┬───────┬─────────┬─────────┬─────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼────┼──────┼───────┼───────┼─────────┼─────────┼─────┤
│ Notif/Sec │ 0 │ 0 │ 5,000 │ 5,000 │ 4,774.2 │ 1006.74 │ 0 │
└───────────┴────┴──────┴───────┴───────┴─────────┴─────────┴─────┘
Delivery : from the moment the request is written to the moment the client reads the notification.
# of samples: 64
301k notifications received in 64.06s (301,000 of 300,000 expected), 94.4 MB read
Calls / second
779
46,735 calls in 60 s, 100 connections
Notifications / second
76,505
4,683,200 received by 100 clients
Delivery p50
443ms
average 442 ms
Delivery p99
519ms
max 610 ms
Not delivered
0
0 calls failed
Server during this run API Maker 2.7 vCPU · Redis 0.2 vCPU · MongoDB 0.6 vCPU · whole server busy 95 % · load generator busy 28 %
$ autocannon -c 100 -d 60 -w 5 -l -m POST -b '{"customer_id":1,"first_name":"BENCH","last_name":"INSERT","phone":<now>,"last_update":"2006-02-15T04:34:00.330Z","pincode":111111,"shipping_id":2,"dob":"2022-10-30T03:25:03.000Z","country":"India"}' -H "content-type: application/json" "http://<server>:38246/api/schema/bench/mongodb/am_bench/inserts_public/save-single-or-multiple"
Running 60s test @ http://<server>:38246/api/schema/bench/mongodb/am_bench/inserts_public/save-single-or-multiple
100 connections
5 workers
┌─────────┬───────┬────────┬────────┬────────┬───────────┬──────────┬────────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├─────────┼───────┼────────┼────────┼────────┼───────────┼──────────┼────────┤
│ Latency │ 86 ms │ 127 ms │ 172 ms │ 184 ms │ 127.72 ms │ 21.77 ms │ 267 ms │
└─────────┴───────┴────────┴────────┴────────┴───────────┴──────────┴────────┘
┌───────────┬────────┬────────┬────────┬────────┬────────┬─────────┬────────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼────────┼────────┼────────┼────────┼────────┼─────────┼────────┤
│ Req/Sec │ 736 │ 741 │ 777 │ 820 │ 778.92 │ 23.31 │ 736 │
├───────────┼────────┼────────┼────────┼────────┼────────┼─────────┼────────┤
│ Bytes/Sec │ 445 kB │ 448 kB │ 470 kB │ 496 kB │ 471 kB │ 14.1 kB │ 445 kB │
└───────────┴────────┴────────┴────────┴────────┴────────┴─────────┴────────┘
Req/Bytes counts sampled once per second.
# of samples: 300
┌────────────┬──────────────┐
│ Percentile │ Latency (ms) │
├────────────┼──────────────┤
│ 0.001 │ 13 │
├────────────┼──────────────┤
│ 0.01 │ 16 │
├────────────┼──────────────┤
│ 0.1 │ 30 │
├────────────┼──────────────┤
│ 1 │ 72 │
├────────────┼──────────────┤
│ 2.5 │ 86 │
├────────────┼──────────────┤
│ 10 │ 103 │
├────────────┼──────────────┤
│ 25 │ 115 │
├────────────┼──────────────┤
│ 50 │ 127 │
├────────────┼──────────────┤
│ 75 │ 141 │
├────────────┼──────────────┤
│ 90 │ 154 │
├────────────┼──────────────┤
│ 97.5 │ 172 │
├────────────┼──────────────┤
│ 99 │ 184 │
├────────────┼──────────────┤
│ 99.9 │ 210 │
├────────────┼──────────────┤
│ 99.99 │ 236 │
├────────────┼──────────────┤
│ 99.999 │ 267 │
└────────────┴──────────────┘
47k requests in 60.02s, 28.3 MB read
$ am_ws_load listen "ws://<server>:38246/?x-am-authorization=<token>&x-am-user-authorization=<token>"
100 clients subscribed, 64.22s
┌──────────┬───────────┬───────────┬───────────┬───────────┬───────────┬──────────┬───────────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├──────────┼───────────┼───────────┼───────────┼───────────┼───────────┼──────────┼───────────┤
│ Delivery │ 375.81 ms │ 443.39 ms │ 506.11 ms │ 519.17 ms │ 442.17 ms │ 37.29 ms │ 609.79 ms │
└──────────┴───────────┴───────────┴───────────┴───────────┴───────────┴──────────┴───────────┘
┌───────────┬────┬──────┬────────┬────────┬──────────┬──────────┬─────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼────┼──────┼────────┼────────┼──────────┼──────────┼─────┤
│ Notif/Sec │ 0 │ 0 │ 77,587 │ 80,735 │ 74,832.3 │ 13950.32 │ 0 │
└───────────┴────┴──────┴────────┴────────┴──────────┴──────────┴─────┘
Delivery : from the moment the request is written to the moment the client reads the notification.
# of samples: 64
4.68M notifications received in 64.22s (4,683,200 of 4,673,500 expected), 1468.9 MB read
With every client connected¶
The same runs once more, with 795,888 clients connected and waiting : what a server which is full of WebSocket clients still does.
| No client, or 10,000 | 795,888 clients connected | |
|---|---|---|
| A notification for one person, 1,000 a second : delivery p50 | 1.04 ms | 1.03 ms |
| … p99 | 2.03 ms | 1.99 ms |
| A notification for one person, as many as the server takes | 12,791 / s | 9,745 / s |
| Get all with both tokens, 100 connections (autocannon) | 5,268 req/s | 4,185 req/s |
| … latency p50 / p99 | 18 ms / 34 ms | 19 ms / 148 ms |
Calls / second (paced)
1,000
60,000 calls in 60 s, 10 connections
Notifications / second
991
60,006 received by 795,888 clients
Delivery p50
1.03ms
average 1.08 ms
Delivery p99
1.99ms
max 7.35 ms
Not delivered
0
0 calls failed
Server during this run API Maker 0.9 vCPU · Redis 0.0 vCPU · whole server busy 26 % · load generator busy 11 %
$ autocannon -c 10 -d 60 -R 1000 -l -m POST -b '{"name":"order-updates","eventData":{"recipientId":"<id of a client>","t":<now>,"type":"order-updated","order_no":1001,"status":"SHIPPED"}}' -H "x-am-authorization: <token>" -H "x-am-user-authorization: <token>" -H "content-type: application/json" "http://<server>:38246/api/system-api/bench/emit-event-ws"
Running 60s test @ http://<server>:38246/api/system-api/bench/emit-event-ws
10 connections
┌─────────┬──────┬──────┬───────┬──────┬─────────┬─────────┬──────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├─────────┼──────┼──────┼───────┼──────┼─────────┼─────────┼──────┤
│ Latency │ 0 ms │ 1 ms │ 2 ms │ 2 ms │ 0.83 ms │ 0.49 ms │ 7 ms │
└─────────┴──────┴──────┴───────┴──────┴─────────┴─────────┴──────┘
┌───────────┬────────┬────────┬────────┬────────┬────────┬───────┬────────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼────────┼────────┼────────┼────────┼────────┼───────┼────────┤
│ Req/Sec │ 1,000 │ 1,000 │ 1,000 │ 1,000 │ 1,000 │ 0 │ 1,000 │
├───────────┼────────┼────────┼────────┼────────┼────────┼───────┼────────┤
│ Bytes/Sec │ 357 kB │ 357 kB │ 357 kB │ 357 kB │ 357 kB │ 0 B │ 357 kB │
└───────────┴────────┴────────┴────────┴────────┴────────┴───────┴────────┘
Req/Bytes counts sampled once per second.
# of samples: 60
┌────────────┬──────────────┐
│ Percentile │ Latency (ms) │
├────────────┼──────────────┤
│ 0.001 │ 0 │
├────────────┼──────────────┤
│ 0.01 │ 0 │
├────────────┼──────────────┤
│ 0.1 │ 0 │
├────────────┼──────────────┤
│ 1 │ 0 │
├────────────┼──────────────┤
│ 2.5 │ 0 │
├────────────┼──────────────┤
│ 10 │ 0 │
├────────────┼──────────────┤
│ 25 │ 1 │
├────────────┼──────────────┤
│ 50 │ 1 │
├────────────┼──────────────┤
│ 75 │ 1 │
├────────────┼──────────────┤
│ 90 │ 1 │
├────────────┼──────────────┤
│ 97.5 │ 2 │
├────────────┼──────────────┤
│ 99 │ 2 │
├────────────┼──────────────┤
│ 99.9 │ 4 │
├────────────┼──────────────┤
│ 99.99 │ 7 │
├────────────┼──────────────┤
│ 99.999 │ 7 │
└────────────┴──────────────┘
61k requests in 60.06s, 21.4 MB read
$ am_ws_load listen "ws://<server>:38246/?x-am-authorization=<token>&x-am-user-authorization=<token>"
795,888 clients subscribed, 63.57s
┌──────────┬────────┬─────────┬─────────┬─────────┬─────────┬────────┬─────────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├──────────┼────────┼─────────┼─────────┼─────────┼─────────┼────────┼─────────┤
│ Delivery │ 0.7 ms │ 1.03 ms │ 1.77 ms │ 1.99 ms │ 1.08 ms │ 0.3 ms │ 7.35 ms │
└──────────┴────────┴─────────┴─────────┴─────────┴─────────┴────────┴─────────┘
┌───────────┬────┬──────┬───────┬───────┬───────┬────────┬─────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼────┼──────┼───────┼───────┼───────┼────────┼─────┤
│ Notif/Sec │ 0 │ 0 │ 1,000 │ 1,000 │ 951.7 │ 214.15 │ 0 │
└───────────┴────┴──────┴───────┴───────┴───────┴────────┴─────┘
Delivery : from the moment the request is written to the moment the client reads the notification.
# of samples: 64
60k notifications received in 63.57s (60,006 of 60,000 expected), 10.8 MB read
Calls / second
9,875
592,472 calls in 60 s, 100 connections
Notifications / second
9,745
592,572 received by 795,888 clients
Delivery p50
8.11ms
average 9.93 ms
Delivery p99
56.1ms
max 226 ms
Not delivered
0
0 calls failed
Server during this run API Maker 3.4 vCPU · Redis 0.2 vCPU · MongoDB 0.0 vCPU · whole server busy 97 % · load generator busy 39 %
$ autocannon -c 100 -d 60 -w 5 -l -m POST -b '{"name":"order-updates","eventData":{"recipientId":"<id of a client>","t":<now>,"type":"order-updated","order_no":1001,"status":"SHIPPED"}}' -H "x-am-authorization: <token>" -H "x-am-user-authorization: <token>" -H "content-type: application/json" "http://<server>:38246/api/system-api/bench/emit-event-ws"
Running 60s test @ http://<server>:38246/api/system-api/bench/emit-event-ws
100 connections
5 workers
┌─────────┬──────┬──────┬───────┬───────┬─────────┬──────────┬────────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├─────────┼──────┼──────┼───────┼───────┼─────────┼──────────┼────────┤
│ Latency │ 5 ms │ 8 ms │ 35 ms │ 56 ms │ 9.59 ms │ 11.77 ms │ 233 ms │
└─────────┴──────┴──────┴───────┴───────┴─────────┴──────────┴────────┘
┌───────────┬─────────┬─────────┬─────────┬─────────┬──────────┬────────┬─────────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼─────────┼─────────┼─────────┼─────────┼──────────┼────────┼─────────┤
│ Req/Sec │ 7,263 │ 9,111 │ 9,911 │ 10,415 │ 9,874.97 │ 487.03 │ 7,260 │
├───────────┼─────────┼─────────┼─────────┼─────────┼──────────┼────────┼─────────┤
│ Bytes/Sec │ 2.59 MB │ 3.25 MB │ 3.54 MB │ 3.72 MB │ 3.53 MB │ 174 kB │ 2.59 MB │
└───────────┴─────────┴─────────┴─────────┴─────────┴──────────┴────────┴─────────┘
Req/Bytes counts sampled once per second.
# of samples: 300
┌────────────┬──────────────┐
│ Percentile │ Latency (ms) │
├────────────┼──────────────┤
│ 0.001 │ 1 │
├────────────┼──────────────┤
│ 0.01 │ 1 │
├────────────┼──────────────┤
│ 0.1 │ 2 │
├────────────┼──────────────┤
│ 1 │ 4 │
├────────────┼──────────────┤
│ 2.5 │ 5 │
├────────────┼──────────────┤
│ 10 │ 6 │
├────────────┼──────────────┤
│ 25 │ 7 │
├────────────┼──────────────┤
│ 50 │ 8 │
├────────────┼──────────────┤
│ 75 │ 9 │
├────────────┼──────────────┤
│ 90 │ 10 │
├────────────┼──────────────┤
│ 97.5 │ 35 │
├────────────┼──────────────┤
│ 99 │ 56 │
├────────────┼──────────────┤
│ 99.9 │ 161 │
├────────────┼──────────────┤
│ 99.99 │ 194 │
├────────────┼──────────────┤
│ 99.999 │ 222 │
└────────────┴──────────────┘
593k requests in 60.02s, 212 MB read
$ am_ws_load listen "ws://<server>:38246/?x-am-authorization=<token>&x-am-user-authorization=<token>"
795,888 clients subscribed, 63.81s
┌──────────┬─────────┬─────────┬──────────┬──────────┬─────────┬──────────┬───────────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├──────────┼─────────┼─────────┼──────────┼──────────┼─────────┼──────────┼───────────┤
│ Delivery │ 4.98 ms │ 8.11 ms │ 35.52 ms │ 56.06 ms │ 9.93 ms │ 11.69 ms │ 225.79 ms │
└──────────┴─────────┴─────────┴──────────┴──────────┴─────────┴──────────┴───────────┘
┌───────────┬────┬──────┬───────┬────────┬─────────┬─────────┬─────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼────┼──────┼───────┼────────┼─────────┼─────────┼─────┤
│ Notif/Sec │ 0 │ 0 │ 9,928 │ 10,631 │ 9,505.1 │ 1827.11 │ 0 │
└───────────┴────┴──────┴───────┴────────┴─────────┴─────────┴─────┘
Delivery : from the moment the request is written to the moment the client reads the notification.
# of samples: 64
593k notifications received in 63.81s (592,572 of 592,472 expected), 107.1 MB read
Requests / second
4,185
251,102 requests in 60 s
Latency p50
19ms
average 23.4 ms
Latency p99
148ms
max 339 ms
Errors
0
non-2xx, errors, timeouts
Server during this run API Maker 2.4 vCPU · Redis 0.1 vCPU · MongoDB 1.0 vCPU · whole server busy 99 % · load generator busy 23 %
$ autocannon -c 100 -d 60 -w 5 -l -H "x-am-authorization: <token>" -H "x-am-user-authorization: <token>" "http://<server>:38246/api/schema/bench/mongodb/am_bench/customers"
Running 60s test @ http://<server>:38246/api/schema/bench/mongodb/am_bench/customers
100 connections
5 workers
┌─────────┬──────┬───────┬───────┬────────┬─────────┬──────────┬────────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├─────────┼──────┼───────┼───────┼────────┼─────────┼──────────┼────────┤
│ Latency │ 8 ms │ 19 ms │ 67 ms │ 148 ms │ 23.4 ms │ 22.51 ms │ 339 ms │
└─────────┴──────┴───────┴───────┴────────┴─────────┴──────────┴────────┘
┌───────────┬─────────┬─────────┬─────────┬───────┬──────────┬────────┬─────────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼─────────┼─────────┼─────────┼───────┼──────────┼────────┼─────────┤
│ Req/Sec │ 3,523 │ 3,793 │ 4,175 │ 4,403 │ 4,185.45 │ 157.74 │ 3,523 │
├───────────┼─────────┼─────────┼─────────┼───────┼──────────┼────────┼─────────┤
│ Bytes/Sec │ 9.64 MB │ 10.4 MB │ 11.4 MB │ 12 MB │ 11.4 MB │ 430 kB │ 9.64 MB │
└───────────┴─────────┴─────────┴─────────┴───────┴──────────┴────────┴─────────┘
Req/Bytes counts sampled once per second.
# of samples: 300
┌────────────┬──────────────┐
│ Percentile │ Latency (ms) │
├────────────┼──────────────┤
│ 0.001 │ 1 │
├────────────┼──────────────┤
│ 0.01 │ 1 │
├────────────┼──────────────┤
│ 0.1 │ 3 │
├────────────┼──────────────┤
│ 1 │ 6 │
├────────────┼──────────────┤
│ 2.5 │ 8 │
├────────────┼──────────────┤
│ 10 │ 12 │
├────────────┼──────────────┤
│ 25 │ 15 │
├────────────┼──────────────┤
│ 50 │ 19 │
├────────────┼──────────────┤
│ 75 │ 24 │
├────────────┼──────────────┤
│ 90 │ 31 │
├────────────┼──────────────┤
│ 97.5 │ 67 │
├────────────┼──────────────┤
│ 99 │ 148 │
├────────────┼──────────────┤
│ 99.9 │ 270 │
├────────────┼──────────────┤
│ 99.99 │ 309 │
├────────────┼──────────────┤
│ 99.999 │ 326 │
└────────────┴──────────────┘
251k requests in 60.02s, 687 MB read
A restart¶
API Maker was restarted (systemctl restart api_maker_be) with 596,916 clients connected : the newest left first, down to 75 % of the 795,888 it held. Every client of a restarted server comes back within the same seconds and waits for its turn with its request in the kernel, about 4 KB each : a server needs that room. It takes what Redis keeps of them out first, then closes every client with the code 1001 (Server is going away) ; each client waits one to five seconds, as an application does, connects again with the same tokens and sends its REGISTER again. The server which comes back lets 256 clients in at a time and handles the messages of 256 at a time, the others wait for their turn : its memory comes back to what it was, not higher.
Clients dropped
596,916
of 596,916 connected
All back after
189.1s
the restart itself : 46.1 s
A client was away, p50
79.1s
its own wait of 1 to 5 s counted
A client was away, p99
172.5s
the longest : 177 s
After the restart, with the 596,916 clients back : API Maker 2,039 MB (2,474 MB before it), 2,188 MB of memory available. They were then told again, each for itself, 1,000 notifications a second for 60 seconds : all 60,000 arrived, in 1.04 ms (p50) and 2.14 ms (p99). A client whose subscription had not come back with it would have been told nothing.
Every client leaves at once¶
At the end of the run, the 596,916 clients closed their connection in the same second, as when a network or a load balancer in front of the server goes. The server had closed every one of their sockets after 23.4 s, and Redis held nothing of them any more after 29.1 s. A socket is closed at once, and Redis is told behind it.
Questions¶
How many WebSocket connections can API Maker hold on 4 dedicated vCPUs and 8 GB ?¶
795,888 clients were connected at the same time on this server, each with both tokens and a subscription of its own, with MongoDB, Redis and two sandboxes on the same machine. Memory is what ends first : a client costs about 8.7 KB of the server in all.
How much memory does a WebSocket client cost ?¶
About 8.7 KB : 3.3 KB in the API Maker process, 4.4 KB in the Linux kernel (its socket) and 1.1 KB in Redis (its subscription). Measured between 200,000 and 600,000 clients which wait, each connected with both tokens and one subscription.
How many clients can connect each second ?¶
6,627 clients a second connected with both tokens, subscribed and left, 100 at a time ; 12,535 a second without a subscription. With a can user connect code on the event : 5,533 a second.
How fast does a notification reach a client ?¶
At 1,000 notifications a second, each for one person among 10,000 : 1.04 ms from the HTTP call to the client (p50), 2.03 ms for 99 % of them. With as many calls as the server takes : 12,791 notifications a second.
Do connected WebSocket clients slow the APIs down ?¶
Get all with both tokens answered 5,268 requests a second with no WebSocket client, and 4,185 with 795,888 clients connected and waiting. A client which waits costs memory ; its CPU is a ping a minute.