# Install API Maker by Hand

> Set up API Maker v4 on Ubuntu piece by piece, the way the install script does - the program, Docker, the network limits for many connections, MongoDB as a replica set, Redis, and the systemd service.

Source: https://docs.apimaker.dev/v4/docs/getting-started/install-by-hand.html

The [install script](https://docs.apimaker.dev/v4/docs/getting-started/install-on-server.html) does all of this for you. Follow these steps when you need to know each piece, or to set up a server your own way.

The pieces are ordinary : one program (`api_maker_be`, kept running by systemd), MongoDB as a replica set, Redis, and any reverse proxy for TLS. The [configuration](https://docs.apimaker.dev/v4/docs/am-resources/api-maker-configurations.html) page lists every setting of `.env` ; the MongoDB connection string goes in `am__mongo_db_connection`, Redis in `am__redisInternal` and `am__redisExternal`. The blocks below set up each piece the way the installer does, on Ubuntu as root.

> **Example : API Maker v4**
>
> A release of API Maker for one processor is an npm package, read with `curl` (npm is not needed) : the server, the program of the sandboxes and the admin panel.
>
> ```bash
> apt-get install -y curl jq
> ARCH=$([ "$(uname -m)" = aarch64 ] && echo arm64 || echo x64)
> PACKAGE=api-maker-engine-linux-$ARCH
> VERSION=$(curl -fsSL "https://registry.npmjs.org/@sava-info-systems%2f$PACKAGE" | jq -r '."dist-tags".latest')
>
> mkdir -p ~/projects/api_maker/releases/$VERSION ~/projects/api_maker/uploads
> curl -fsSL "https://registry.npmjs.org/@sava-info-systems/$PACKAGE/-/$PACKAGE-$VERSION.tgz" \
>     | tar -xz --strip-components=1 -C ~/projects/api_maker/releases/$VERSION
> ln -sfn ~/projects/api_maker/releases/$VERSION ~/projects/api_maker/current
> ~/projects/api_maker/current/api_maker_be --version
> ```

> **Example : Docker with Docker Compose**
>
> From the official apt repository of Docker, the commands the installer runs. They work on Ubuntu 20.04 to 26.04 ; a release too new for Docker's repository falls back to the one of 24.04 (`noble`).
>
> ```bash
> for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do apt-get remove -y $pkg; done
> apt-get update
> apt-get install -y ca-certificates curl gnupg
> install -m 0755 -d /etc/apt/keyrings
> curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
> chmod a+r /etc/apt/keyrings/docker.asc
>
> CODENAME=$(. /etc/os-release && echo "$VERSION_CODENAME")
> curl -fsL -o /dev/null "https://download.docker.com/linux/ubuntu/dists/$CODENAME/Release" || CODENAME=noble
> echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $CODENAME stable" > /etc/apt/sources.list.d/docker.list
>
> apt-get update
> apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
> docker compose version
> ```

> **Example : Room for many connections**
>
> A server holds about 120 [WebSocket clients](https://docs.apimaker.dev/v4/docs/pages/web-socket-event-page.html#many-clients) for each MB of its memory (8.4 KB each, with its socket in the kernel and its subscription in Redis : [measured](https://docs.apimaker.dev/v4/docs/benchmarks/websockets/index.html)). Two limits of Linux end long before that, and the installer raises both :
>
> - **The table of the tracked connections.** A Linux server which runs Docker tracks every connection in a table (`nf_conntrack`), and a connection which closed stays in it for two minutes. Past the size of the table (262,144 connections by default) the kernel drops new connections without a word to the program which listens, SSH included. The installer gives it 256 connections for each MB of memory (between 131,072 and 16,777,216, never under what the server has already) : twice the clients the memory holds, so that clients which all come back after a restart find room while their old connections are still in it. A closed connection leaves it after 30 seconds, and its index gets a place for every four connections, so that a full table is looked through as fast as an empty one. An entry takes about 320 bytes, and only while its connection is there.
> - **The open files.** Every connection is an open file of API Maker. The service may have as many as the table tracks (`LimitNOFILE`, 1,048,576 at the least), Caddy twice as many (it holds two for every client it passes on), and Linux gives a process no more than `fs.nr_open` (1,048,576 as it starts) : the installer raises it with them. With 1,048,576 open files a server of 16 GB took about a million clients, with half of its memory left.
>
> The queue of the connections which wait to be accepted gets longer too : the clients which all come back after a restart wait there.
>
> ```bash
> MEMORY_MB=$(awk '/^MemTotal:/ { printf "%d", $2 / 1024 }' /proc/meminfo)
> TRACKED=$((MEMORY_MB * 256))
> [ "$TRACKED" -lt 131072 ] && TRACKED=131072
> [ "$TRACKED" -gt 16777216 ] && TRACKED=16777216
> OPEN_FILES=$(( TRACKED > 1048576 ? TRACKED : 1048576 ))
>
> # The table is there at boot before its limits are read (Docker would load it later), with its index
> modprobe nf_conntrack
> echo nf_conntrack > /etc/modules-load.d/api-maker.conf
> echo "options nf_conntrack hashsize=$((TRACKED / 4))" > /etc/modprobe.d/api-maker.conf
> echo $((TRACKED / 4)) > /sys/module/nf_conntrack/parameters/hashsize
>
> cat > /etc/sysctl.d/90-api-maker.conf <<EOF
> net.netfilter.nf_conntrack_max = $TRACKED
> net.netfilter.nf_conntrack_tcp_timeout_time_wait = 30
> net.core.somaxconn = 16384
> net.ipv4.tcp_max_syn_backlog = 16384
> fs.nr_open = $((OPEN_FILES * 2))
> EOF
> sysctl -p /etc/sysctl.d/90-api-maker.conf
>
> # The services : LimitNOFILE=$OPEN_FILES in api_maker_be.service, twice that in the one of Caddy
> systemctl show api_maker_be -p LimitNOFILE
>
> sysctl net.netfilter.nf_conntrack_count net.netfilter.nf_conntrack_max   # how many are tracked now, and at most
> dmesg | grep 'nf_conntrack: table full'                                  # nothing : no connection was dropped
> ```
>
> API Maker reads the size of the table and its own open files when it starts, says in its log how many WebSockets it takes at the most, and warns when the table holds fewer connections than the server takes and than its memory has room for.
>
> One limit of the network is not set here, because API Maker keeps it itself while it runs : the memory Linux gives all its TCP connections together (`net.ipv4.tcp_mem`). The service runs as root and raises it as its connections come, two pages for each of them, never above a quarter of the memory of the server ([`tcpMemoryPercent`](https://docs.apimaker.dev/v4/docs/am-resources/api-maker-configurations.html)). Started as another user, it says in its log the line to run as root once it has that many connections :
>
> ```bash
> sysctl net.ipv4.tcp_mem          # three numbers of pages : nothing held back, pressure, the most
> ```

> **Example : MongoDB as a one node replica set**
>
> API Maker needs MongoDB as a replica set : transactions only work there. A replica set with authentication needs a key file.
>
> ```bash
> mkdir -p ~/config && cd ~/config
> openssl rand -base64 756 > keyfile
> chmod 600 keyfile && chown 999:999 keyfile
> ```
>
> **~/config/mongodb_api_maker.yml**
>
> ```yaml
> services:
>     am_mongodb:
>         image: mongo:8.0.10
>         restart: always
>         container_name: mongodb_api_maker
>         command: [ "--replSet", "rs0", "--bind_ip_all", "--port", "27017", "--keyFile", "/opt/keyfile" ]
>         ports:
>             - 38248:27017
>         environment:
>             - MONGO_INITDB_ROOT_USERNAME=api_maker_user   # 👈 your user
>             - MONGO_INITDB_ROOT_PASSWORD=CHANGE_ME         # 👈 your password
>             - MONGO_REPLICA_SET_NAME=rs0
>         healthcheck:
>             # Starts the replica set on the first run.
>             test: echo "try { rs.status() } catch (err) { rs.initiate({_id:'rs0',members:[{_id:0,host:'127.0.0.1:27017'}]}) }" | mongosh -u "api_maker_user" -p "CHANGE_ME" --port 27017 --quiet
>             interval: 10s
>             timeout: 30s
>             start_period: 5s
>             retries: 10
>         volumes:
>             - ~/docker-data/mongodb_api_maker/data:/data/db
>             - ~/docker-data/mongodb_api_maker/configdb:/data/configdb
>             - ./keyfile:/opt/keyfile
> ```
>
> ```bash
> docker compose --project-name api_maker -f ~/config/mongodb_api_maker.yml up -d
> ```
>
> **am__mongo_db_connection**
>
> ```text
> mongodb://api_maker_user:CHANGE_ME@127.0.0.1:38248/api_maker_db?authSource=admin&replicaSet=rs0&directConnection=true
> ```
>
> MongoDB Compass, Studio 3T or any other client connects with the same string.

> **Example : Redis**
>
> **~/config/redis_api_maker.yml**
>
> ```yaml
> services:
>     redis_api_maker:
>         image: 'redis:7.0.5-alpine'
>         restart: always
>         container_name: redis_api_maker
>         ports:
>             - '7479:6379'
>         command: redis-server --loglevel warning --requirepass CHANGE_ME   # 👈 your password
>         volumes:
>             - ~/docker-data/redis_api_maker:/data
>         logging:
>             driver: "json-file"
>             options:
>                 max-size: "5m"
>                 max-file: "3"
> ```
>
> ```bash
> docker compose --project-name api_maker -f ~/config/redis_api_maker.yml up -d
> ```
>
> **am__redisInternal and am__redisExternal**
>
> ```text
> {"nodes": [{port: 7479, host: "127.0.0.1", pass: "CHANGE_ME"}]}
> ```

> **Example : API Maker as a systemd service**
>
> systemd starts API Maker when the server boots, and again whenever it stops. One process uses every core of the server.
>
> **~/config/.env (the settings the installer writes)**
>
> ```bash
> BE_HOST_PORT="http://203.0.113.10:38246"     # 👈 where the admin panel reaches API Maker, its web sockets too
> am__serverName=primary
> am__logFilePath=/root/logs/api_maker_be.txt
> am__port=38246
> am__adminPanel__folder="/root/projects/api_maker/current/admin-panel"
> am__adminPanel__port=4626
> am__oracleClientPath=/opt/oracle/instantclient
> am__mongo_db_connection="mongodb://api_maker_user:CHANGE_ME@127.0.0.1:38248/api_maker_db?authSource=admin&replicaSet=rs0&directConnection=true"
> am__redisInternal='{"nodes": [{port:7479, host:"127.0.0.1", pass: "CHANGE_ME"}]}'
> am__passCommunication="…"   # 👈 20 characters or more
> am__passJWT="…"             # 👈 32 characters or more : API Maker warns at every start under 32
> am__passDBEncryptDecrypt="…" # 👈 32 characters
> ```
>
> **/etc/systemd/system/api_maker_be.service**
>
> ```ini
> [Unit]
> Description=API Maker
> Wants=network-online.target
> After=network-online.target docker.service
> # It never gives up : a database that is down for a while must not leave API Maker stopped.
> StartLimitIntervalSec=0
>
> [Service]
> Type=exec
> User=root
> WorkingDirectory=/root/projects/api_maker
> ExecStart=/root/projects/api_maker/current/api_maker_be --appRoot=/root/projects/api_maker --envPath=/root/config/.env
> StandardOutput=append:/root/logs/api_maker_be.txt
> StandardError=append:/root/logs/api_maker_be.txt
> Restart=always
> RestartSec=1s
> # SIGTERM : the requests in flight finish, the clients of the web sockets are taken out of Redis and closed, the logs
> # are written, the schedulers go to the other servers. The process gives that a minute at the most : it is killed
> # after 90 seconds.
> KillMode=mixed
> TimeoutStopSec=90s
> LimitNOFILE=1048576
>
> [Install]
> WantedBy=multi-user.target
> ```
>
> ```bash
> mkdir -p ~/logs
> systemctl daemon-reload
> systemctl enable --now api_maker_be
>
> systemctl status api_maker_be       # active (running)
> tail -f ~/logs/api_maker_be.txt     # what API Maker prints
> curl -s http://127.0.0.1:38246/ | head -5   # the page of the process
> ```
>
> The installer also rotates the log past 50 MB with `logrotate`, run by the timer `api_maker_logrotate`.

## Related

- [Install on a server](https://docs.apimaker.dev/v4/docs/getting-started/install-on-server.html) · [API Maker configuration](https://docs.apimaker.dev/v4/docs/am-resources/api-maker-configurations.html)