How many WebSocket users can API Maker hold ? As many as the memory has room for.
API Maker installed by its own installer on 7 Linode servers, from 1 to 8 vCPUs, its MongoDB and Redis on the same machine. A second server opened WebSocket clients, each with both tokens and a subscription of its own, until the server was full, and autocannon made the calls which notify them, 60 seconds a run : 50,167,783 clients let in, 214,875,630 notifications delivered. Every page shows the outputs and what each part of the server held in memory.
The answers¶
How many connections one server holds¶
Each client connected with both tokens and one subscription of its own, MongoDB, Redis and two sandboxes on the same server.
| Server | CPU · price / month | Clients connected | What ended first | API Maker | Whole server, used | Per client | Clients per $ a month |
|---|---|---|---|---|---|---|---|
| 1 vCPU · 1 GB | Shared · $5 | 26,484 | memory | 126 MB | 813 MB of 961 MB | 8.6 KB | 5,297 |
| 2 vCPU · 4 GB | Shared · $24 | 338,016 | memory | 1,110 MB | 3,417 MB of 3,916 MB | 8.5 KB | 14,084 |
| 4 vCPU · 8 GB | Shared · $48 | 801,972 | memory | 2,495 MB | 6,937 MB of 7,941 MB | 8.7 KB | 16,708 |
| 8 vCPU · 32 GB | Shared · $192 | 3,662,564 | memory | 10,196 MB | 27,618 MB of 32,096 MB | 8.8 KB | 19,076 |
| 2 vCPU · 4 GB | Dedicated · $36 | 338,006 | memory | 1,108 MB | 3,465 MB of 3,916 MB | 8.5 KB | 9,389 |
| 4 vCPU · 8 GB | Dedicated · $72 | 795,888 | memory | 2,474 MB | 6,917 MB of 7,941 MB | 8.7 KB | 11,054 |
| 8 vCPU · 16 GB | Dedicated · $144 | 1,732,035 | memory | 5,179 MB | 13,942 MB of 15,992 MB | 8.8 KB | 12,028 |
A client which waits costs memory, and almost no CPU : the server pings it once a minute. So the number of clients a server holds follows its memory, shared or dedicated CPU alike. The page of each server gives every level of clients on the way, with what each part held.
Memory when a server is full¶
| Server | Clients | API Maker | Redis | Linux kernel | Per client : API Maker · kernel · Redis | Per client, in all |
|---|---|---|---|---|---|---|
| 1 vCPU · 1 GB | 26,484 | 126 MB | 30 MB | 200 MB | 3.3 · 4.3 · 1.0 KB | 8.6 KB |
| 2 vCPU · 4 GB | 338,016 | 1,110 MB | 356 MB | 1,635 MB | 3.1 · 4.3 · 1.0 KB | 8.5 KB |
| 4 vCPU · 8 GB | 801,972 | 2,495 MB | 808 MB | 3,715 MB | 3.3 · 4.4 · 1.1 KB | 8.7 KB |
| 8 vCPU · 32 GB | 3,662,564 | 10,196 MB | 3,533 MB | 16,443 MB | 3.2 · 4.5 · 1.1 KB | 8.8 KB |
| 2 vCPU · 4 GB | 338,006 | 1,108 MB | 356 MB | 1,627 MB | 3.1 · 4.3 · 1.0 KB | 8.5 KB |
| 4 vCPU · 8 GB | 795,888 | 2,474 MB | 802 MB | 3,691 MB | 3.3 · 4.4 · 1.1 KB | 8.7 KB |
| 8 vCPU · 16 GB | 1,732,035 | 5,179 MB | 1,633 MB | 7,839 MB | 3.2 · 4.5 · 1.1 KB | 8.8 KB |
Scale to N users¶
A WebSocket stays on the server it reached, and what is said to a client can come from any server : the server which answers an API call, or runs emitEventWS, publishes the notification in Redis, and the server which holds the client sends it. So more users are more servers, and nothing else : no sticky session, no broker to add. How it was measured, and what ends first : Scale to N users.
One big server or several small ones¶
The same 8 dedicated vCPUs and 16 GB for API Maker, at the same price, as one server, two, and four ; MongoDB and Redis on a server of their own each time.
| 1 × 8 vCPU · 16 GB | 2 × 4 vCPU · 8 GB | 4 × 2 vCPU · 4 GB | |
|---|---|---|---|
| Clients connected at once | 2,022,026 | 1,915,960 | 1,619,372 |
| … on each server | 2,022,026 | 957,980 | 404,843 |
| Memory per client, in all | 8.8 KB | 8.8 KB | 8.6 KB |
| New clients a second (both tokens, a subscription) | 12,412 | 12,696 | 10,297 |
| … connect only, both tokens | 19,917 | 21,417 | 17,882 |
| A notification for one person : delivery p50 | 1.01 ms | 1.18 ms | 1.37 ms |
| … p99 | 1.72 ms | 1.80 ms | 2.51 ms |
| Notifications a second, one person each | 24,305 | 24,977 | 25,566 |
| Notifications a second, broadcast | 217,711 | 228,910 | 272,609 |
| One server stops : clients dropped | 1,516,519 of 1,516,519 (100 %) | 383,192 of 766,384 (50 %) | 242,906 of 971,623 (25 %) |
| … all back after | 221 s | 68.0 s | 38.0 s |
Several servers hold about what one big server holds, and lose only their share of the clients when one of them stops. The whole comparison.
New clients and notifications¶
100 clients at a time connect, subscribe and leave, for 60 seconds.
autocannon calls the emit event WS API as fast as the server takes ; what the clients received each second.
The servers¶
Shared CPU¶
Dedicated CPU¶
How it is measured¶
- A real installation API Maker is installed by its installer with its default answers, MongoDB and Redis in Docker on the same server. Nothing else runs : no reverse proxy, no other database.
- Real clients, from a second server A dedicated server of the same data centre opens the WebSocket clients, from 11 addresses of its own (one address reaches one port of a server about 64,000 times). A client connects with both tokens, is told
CONNECTED, sends oneREGISTERwith a condition of its own, and answers the pings of the server. - autocannon makes the HTTP calls The calls which notify the clients (the emit event WS API, the insert API of a table) and get all are made by autocannon, as in the API benchmarks. The clients count what they are told, and how late.
- 60 seconds a run Every measured run lasts 60 seconds, after a warm-up. Clients which were refused, notifications which did not arrive and calls which failed are counted.
- Clients are added until the server is full Level after level, each held 30 seconds while the server is sampled (the last one 60). The run stops when what is available, less what Linux moved to the swap since the clients began to connect, is under 5 % of the memory of the server, or when it lets nobody else in : the clients fit in memory, not in the swap.
- Memory and CPU every second The server samples its memory, its CPU, the API Maker process, every container, the sockets of the kernel and the table of its firewall while the clients are there.
The whole method, the events, the can user connect codes and the commands : How the WebSocket benchmarks are made. The data is the API Maker benchmark dataset (https://apimaker.dev/json/benchmark_dataset.json). The APIs themselves : API benchmarks.
Questions¶
How many WebSocket connections can one API Maker server hold ?¶
As many as its memory has room for : 26,484 on 1 shared vCPU and 1 GB, 338,016 on 2 shared vCPUs and 4 GB, 801,972 on 4 shared vCPUs and 8 GB, 3,662,564 on 8 shared vCPUs and 32 GB, 338,006 on 2 dedicated vCPUs and 4 GB, 795,888 on 4 dedicated vCPUs and 8 GB, 1,732,035 on 8 dedicated vCPUs and 16 GB, with MongoDB, Redis and two sandboxes on the same machine. A client which waits costs 8.5 to 8.8 KB of the server in all, and almost no CPU.
How much memory does a WebSocket connection use ?¶
8.5 to 8.8 KB in all on the servers measured, for a client connected with both tokens and one subscription : most of it in the API Maker process, the rest in the Linux kernel (the socket) and in Redis (the subscription).
How do I scale to N WebSocket users ?¶
Add API Maker servers behind a load balancer : each holds the clients which reached it, and they share one Redis, which carries every notification to the server which holds the client. Count the servers from the memory : N users × 8.5 to 8.8 KB, plus one spare server. Measured : 1,915,960 clients on 2 servers, 1,619,372 on 4 servers. See Scale to N users.
One big server or several small ones ?¶
For the same price, they hold about the same number of clients : 2,022,026 on 1 × 8 vCPU · 16 GB, 1,915,960 on 2 × 4 vCPU · 8 GB, 1,619,372 on 4 × 2 vCPU · 4 GB. Several servers lose only a part of the clients when one of them stops, and can be restarted one at a time ; one big server has less to run. See Scale to N users.
How fast are WebSocket notifications ?¶
A notification for one person reached its client in 0.8 to 2.5 ms (p50) from the HTTP call which emitted it, at 1,000 notifications a second. In a broadcast, a server told up to 206,570 clients a second.