Install by hand¶
The install script does all of this for you. Follow these steps when you need to know each piece, or to set up a server your own way.
The pieces are ordinary : one program (api_maker_be, kept running by systemd), MongoDB as a replica set, Redis, and any reverse proxy for TLS. The configuration page lists every setting of .env ; the MongoDB connection string goes in am__mongo_db_connection, Redis in am__redisInternal and am__redisExternal. The blocks below set up each piece the way the installer does, on Ubuntu as root.
API Maker v4
A release of API Maker for one processor is an npm package, read with curl (npm is not needed) : the server, the program of the sandboxes and the admin panel.
apt-get install -y curl jq
ARCH=$([ "$(uname -m)" = aarch64 ] && echo arm64 || echo x64)
PACKAGE=api-maker-engine-linux-$ARCH
VERSION=$(curl -fsSL "https://registry.npmjs.org/@sava-info-systems%2f$PACKAGE" | jq -r '."dist-tags".latest')
mkdir -p ~/projects/api_maker/releases/$VERSION ~/projects/api_maker/uploads
curl -fsSL "https://registry.npmjs.org/@sava-info-systems/$PACKAGE/-/$PACKAGE-$VERSION.tgz" \
| tar -xz --strip-components=1 -C ~/projects/api_maker/releases/$VERSION
ln -sfn ~/projects/api_maker/releases/$VERSION ~/projects/api_maker/current
~/projects/api_maker/current/api_maker_be --version
Docker with Docker Compose
From the official apt repository of Docker, the commands the installer runs. They work on Ubuntu 20.04 to 26.04 ; a release too new for Docker's repository falls back to the one of 24.04 (noble).
for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do apt-get remove -y $pkg; done
apt-get update
apt-get install -y ca-certificates curl gnupg
install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
chmod a+r /etc/apt/keyrings/docker.asc
CODENAME=$(. /etc/os-release && echo "$VERSION_CODENAME")
curl -fsL -o /dev/null "https://download.docker.com/linux/ubuntu/dists/$CODENAME/Release" || CODENAME=noble
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $CODENAME stable" > /etc/apt/sources.list.d/docker.list
apt-get update
apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
docker compose version
Room for many connections
A server holds about 120 WebSocket clients for each MB of its memory (8.4 KB each, with its socket in the kernel and its subscription in Redis : measured). Two limits of Linux end long before that, and the installer raises both :
- The table of the tracked connections. A Linux server which runs Docker tracks every connection in a table (
nf_conntrack), and a connection which closed stays in it for two minutes. Past the size of the table (262,144 connections by default) the kernel drops new connections without a word to the program which listens, SSH included. The installer gives it 256 connections for each MB of memory (between 131,072 and 16,777,216, never under what the server has already) : twice the clients the memory holds, so that clients which all come back after a restart find room while their old connections are still in it. A closed connection leaves it after 30 seconds, and its index gets a place for every four connections, so that a full table is looked through as fast as an empty one. An entry takes about 320 bytes, and only while its connection is there. - The open files. Every connection is an open file of API Maker. The service may have as many as the table tracks (
LimitNOFILE, 1,048,576 at the least), Caddy twice as many (it holds two for every client it passes on), and Linux gives a process no more thanfs.nr_open(1,048,576 as it starts) : the installer raises it with them. With 1,048,576 open files a server of 16 GB took about a million clients, with half of its memory left.
The queue of the connections which wait to be accepted gets longer too : the clients which all come back after a restart wait there.
MEMORY_MB=$(awk '/^MemTotal:/ { printf "%d", $2 / 1024 }' /proc/meminfo)
TRACKED=$((MEMORY_MB * 256))
[ "$TRACKED" -lt 131072 ] && TRACKED=131072
[ "$TRACKED" -gt 16777216 ] && TRACKED=16777216
OPEN_FILES=$(( TRACKED > 1048576 ? TRACKED : 1048576 ))
# The table is there at boot before its limits are read (Docker would load it later), with its index
modprobe nf_conntrack
echo nf_conntrack > /etc/modules-load.d/api-maker.conf
echo "options nf_conntrack hashsize=$((TRACKED / 4))" > /etc/modprobe.d/api-maker.conf
echo $((TRACKED / 4)) > /sys/module/nf_conntrack/parameters/hashsize
cat > /etc/sysctl.d/90-api-maker.conf <<EOF
net.netfilter.nf_conntrack_max = $TRACKED
net.netfilter.nf_conntrack_tcp_timeout_time_wait = 30
net.core.somaxconn = 16384
net.ipv4.tcp_max_syn_backlog = 16384
fs.nr_open = $((OPEN_FILES * 2))
EOF
sysctl -p /etc/sysctl.d/90-api-maker.conf
# The services : LimitNOFILE=$OPEN_FILES in api_maker_be.service, twice that in the one of Caddy
systemctl show api_maker_be -p LimitNOFILE
sysctl net.netfilter.nf_conntrack_count net.netfilter.nf_conntrack_max # how many are tracked now, and at most
dmesg | grep 'nf_conntrack: table full' # nothing : no connection was dropped
API Maker reads the size of the table and its own open files when it starts, says in its log how many WebSockets it takes at the most, and warns when the table holds fewer connections than the server takes and than its memory has room for.
One limit of the network is not set here, because API Maker keeps it itself while it runs : the memory Linux gives all its TCP connections together (net.ipv4.tcp_mem). The service runs as root and raises it as its connections come, two pages for each of them, never above a quarter of the memory of the server (tcpMemoryPercent). Started as another user, it says in its log the line to run as root once it has that many connections :
MongoDB as a one node replica set
API Maker needs MongoDB as a replica set : transactions only work there. A replica set with authentication needs a key file.
mkdir -p ~/config && cd ~/config
openssl rand -base64 756 > keyfile
chmod 600 keyfile && chown 999:999 keyfile
services:
am_mongodb:
image: mongo:8.0.10
restart: always
container_name: mongodb_api_maker
command: [ "--replSet", "rs0", "--bind_ip_all", "--port", "27017", "--keyFile", "/opt/keyfile" ]
ports:
- 38248:27017
environment:
- MONGO_INITDB_ROOT_USERNAME=api_maker_user # 👈 your user
- MONGO_INITDB_ROOT_PASSWORD=CHANGE_ME # 👈 your password
- MONGO_REPLICA_SET_NAME=rs0
healthcheck:
# Starts the replica set on the first run.
test: echo "try { rs.status() } catch (err) { rs.initiate({_id:'rs0',members:[{_id:0,host:'127.0.0.1:27017'}]}) }" | mongosh -u "api_maker_user" -p "CHANGE_ME" --port 27017 --quiet
interval: 10s
timeout: 30s
start_period: 5s
retries: 10
volumes:
- ~/docker-data/mongodb_api_maker/data:/data/db
- ~/docker-data/mongodb_api_maker/configdb:/data/configdb
- ./keyfile:/opt/keyfile
mongodb://api_maker_user:[email protected]:38248/api_maker_db?authSource=admin&replicaSet=rs0&directConnection=true
MongoDB Compass, Studio 3T or any other client connects with the same string.
Redis
services:
redis_api_maker:
image: 'redis:7.0.5-alpine'
restart: always
container_name: redis_api_maker
ports:
- '7479:6379'
command: redis-server --loglevel warning --requirepass CHANGE_ME # 👈 your password
volumes:
- ~/docker-data/redis_api_maker:/data
logging:
driver: "json-file"
options:
max-size: "5m"
max-file: "3"
API Maker as a systemd service
systemd starts API Maker when the server boots, and again whenever it stops. One process uses every core of the server.
BE_HOST_PORT="http://203.0.113.10:38246" # 👈 where the admin panel reaches API Maker, its web sockets too
am__serverName=primary
am__logFilePath=/root/logs/api_maker_be.txt
am__port=38246
am__adminPanel__folder="/root/projects/api_maker/current/admin-panel"
am__adminPanel__port=4626
am__oracleClientPath=/opt/oracle/instantclient
am__mongo_db_connection="mongodb://api_maker_user:[email protected]:38248/api_maker_db?authSource=admin&replicaSet=rs0&directConnection=true"
am__redisInternal='{"nodes": [{port:7479, host:"127.0.0.1", pass: "CHANGE_ME"}]}'
am__passCommunication="…" # 👈 20 characters or more
am__passJWT="…" # 👈 32 characters or more : API Maker warns at every start under 32
am__passDBEncryptDecrypt="…" # 👈 32 characters
[Unit]
Description=API Maker
Wants=network-online.target
After=network-online.target docker.service
# It never gives up : a database that is down for a while must not leave API Maker stopped.
StartLimitIntervalSec=0
[Service]
Type=exec
User=root
WorkingDirectory=/root/projects/api_maker
ExecStart=/root/projects/api_maker/current/api_maker_be --appRoot=/root/projects/api_maker --envPath=/root/config/.env
StandardOutput=append:/root/logs/api_maker_be.txt
StandardError=append:/root/logs/api_maker_be.txt
Restart=always
RestartSec=1s
# SIGTERM : the requests in flight finish, the clients of the web sockets are taken out of Redis and closed, the logs
# are written, the schedulers go to the other servers. The process gives that a minute at the most : it is killed
# after 90 seconds.
KillMode=mixed
TimeoutStopSec=90s
LimitNOFILE=1048576
[Install]
WantedBy=multi-user.target
mkdir -p ~/logs
systemctl daemon-reload
systemctl enable --now api_maker_be
systemctl status api_maker_be # active (running)
tail -f ~/logs/api_maker_be.txt # what API Maker prints
curl -s http://127.0.0.1:38246/ | head -5 # the page of the process
The installer also rotates the log past 50 MB with logrotate, run by the timer api_maker_logrotate.