This page View Markdown Open in ChatGPT Open in Claude

Test cases¶

In API Maker, you can write test cases of your custom APIs and utility classes.
They run on the server, in the sandbox of your account, with the same g your custom APIs get. You can run them from the Test Cases page (menu API Info → Test Cases), or from the folder of the local client with npm run test.
Every run reports which lines of your custom APIs, utility classes and test cases ran (code coverage), on the page and in your IDE.

The Test Cases page¶

The page has three parts :

Left : every test case of the account, with the state its last run gave it (passed, failed, not run). Search by name, custom API or utility class, filter by state, use the arrow keys to move and Enter to run. The play button next to a test case runs that one.
Middle : the code of the selected test case. Ctrl+S saves, Ctrl+Enter runs it. After a run, the line of every failure is marked, and the gutter shows which lines ran (coverage). The Mocks button opens the mocks of the test case (see below).
Right : the run going on, or the last one : each file and each test as the server reports them, the failures with what was expected and what came, then the coverage of everything which ran. A run started from a terminal (npm run test) or from another tab shows here as well.

The switch Collect the code coverage at the bottom of the list decides whether the runs of this browser collect the coverage. It slows a run a little.

Add a new test case¶

Basic info¶

Give the name of the test case.
Select the type of the test case : Custom API or Utility class.
Select the custom API or the utility class it is about ('Test Item').

Mocks¶

The Mocks tab holds the mocks of the test case : calls of g.sys (g.sys.db.getAll, g.sys.system.hash, g.sys.cache.getKey ...) which answer with your data instead of reaching the database, redis or the web. When the tab is empty, it offers mocks to start from : rows of a table, one row by its id, a count, an external API, a secret, a cached value.
Each mock reads as a sentence : when the test calls a method, with the arguments (optional, one condition per argument, in order), answer with a value.
Write values the way a test writes them : JSON ({ "collection": "orders" }, [1, 2], "text", 42, true, null), objects written the JavaScript way ({ collection: 'orders' }) or plain text. The type comes from what you write.
An object matches an argument which has the same keys with the same values : its other keys do not matter, and only the first level is compared. A text, a number or a boolean must be equal. Without a condition, the mock answers every call of the method. An empty answer answers null.
The mocks are tried from the top : the first one whose method and arguments match answers. The tab warns about a mock which an earlier one hides, and about a condition on an argument the call never receives.
In code shows the same mock written for the code of a test, ready to copy.
Mocks can also be written in the code, next to the test which needs them (see below). They come first, so they win over the mocks of the tab.

Description¶

Here you can add more readable text to understand the test case.

Once you save the new test case you will get the default code of the test case like below.

import * as T from 'types';
import * as db from 'db-interfaces';
import * as assert from 'node:assert';

module.exports = [{
    name: 'Test case name 1',
    code: async (g: T.IAMGlobal) => {
        assert.strictEqual(10, 10);
    }
}];

As you can see in the above code, we have an array of test objects : one object per test, with its name and its code.
Every test gets the same g as a custom API : g.sys for the APIs of API Maker, g.logger for logs which show with the result, g.req and g.res.

Mocks in the code¶

Next to a test, mocks holds the mocks of that test alone :

import * as T from 'types';
import * as assert from 'node:assert';

module.exports = [{
    name: 'counts the orders of a customer',
    mocks: [
        // the call, when it applies (one entry per argument, optional), and what it answers
        { method: 'db.getAll', when: [{ collection: 'orders' }], return: [{ id: 1 }, { id: 2 }] },
        // the answer can be a function of the arguments, async if needed
        { method: 'g.sys.system.hash', return: (data: string) => 'hashed:' + data },
        // a condition can be a function of the argument
        { method: 'db.count', when: [(query: any) => query.collection === 'orders'], return: 2 },
    ],
    code: async (g: T.IAMGlobal) => {
        const rows = await g.sys.db.getAll({ instance: 'mongodb', database: 'shop', collection: 'orders' });
        assert.strictEqual(rows.length, 2);
        assert.strictEqual(g.sys.test.calls('db.getAll').length, 1); // the calls the mocks answered
    },
}];

method is the id of the API (SCHEMA_GET_ALL) or its name in code (g.sys.db.getAll, db.getAll). The methods of the Mocks tab are the ones which can be mocked.
g.sys.test.mock(...) adds a mock while the test runs, g.sys.test.clearMocks(method?) removes mocks, g.sys.test.calls(method?) gives the calls the mocks answered so far in this test.
A call which no mock answers reaches the real API, with the full permissions of the account.

Testing a custom API¶

g.sys.test.runCustomApi runs the code of a custom API of the account inside the test, with the request you describe. The g.sys calls the API makes are answered by the mocks of the test, and the coverage of the run sees the API. The hooks of the API are not run : it is the code of the API alone.

import * as T from 'types';
import * as assert from 'node:assert';

module.exports = [{
    name: 'orderCount answers the number of orders',
    mocks: [{ method: 'db.getAll', when: [{ collection: 'orders' }], return: [{ id: 1 }, { id: 2 }, { id: 3 }] }],
    code: async (g: T.IAMGlobal) => {
        const out = await g.sys.test.runCustomApi({ name: 'orderCount', query: { customer: 7 }, headers: { 'x-role': 'admin' } });
        assert.deepStrictEqual(out, { count: 3 });

        // the whole answer : data, logs, headers, warnings
        const full = await g.sys.test.runCustomApi({ name: 'orderCount', query: { customer: 7 } }, true);
        assert.deepStrictEqual(full.logs, ['3 orders']);

        // what the API throws is thrown here
        await assert.rejects(g.sys.test.runCustomApi({ name: 'orderCount', body: { fail: true } }), /failed on purpose/);
    },
}];

name is the name of the custom API, version (its version or the name of the version) runs another version than the one the API serves.
body, query, params and headers are the request the API sees.

Testing a utility class¶

A utility class is imported the way custom APIs import it, and its functions are called directly :

import * as T from 'types';
import * as assert from 'node:assert';
import * as GetClientSoftwareVersions from 'cloud/getVersion';
import * as ClientSoftware from 'cloud/clientSoftware';
import * as CloudTypes from 'cloud/CloudTypes';

module.exports = [
    {
        name: 'Get client software versions',
        code: async (g: T.IAMGlobal) => {
            const versions = await GetClientSoftwareVersions.getSoftwareVersions(g);
            assert.strictEqual(versions.length > 0, true);
        }
    },
    {
        name: 'Get specific software version',
        code: async (g: T.IAMGlobal) => {
            const version: CloudTypes.IVersion = await ClientSoftware.getSoftwareVersion(g, 'API Maker', 'v1.0.0');
            assert.strictEqual(version.version === 'v1.0.0', true);
        }
    }
];

'GetClientSoftwareVersions', 'ClientSoftware' and 'CloudTypes' are imported from the utility classes. The g.sys calls they make are answered by the mocks of the test too.

Execute / Run¶

On the page, the play button next to a test case runs that one, Run in the editor runs the selected one, Run all runs every test case of the account.
The results show test by test : a failed test shows its message, and a click on it shows what was expected and what came, side by side, with the line of the test in the editor.
A run can be cancelled : it stops after the file which is running.
The last run is kept 30 days, so the page shows it again after a reload. Clear results above the results of the run panel clears it : its results and its coverage. Clear coverage above the coverage clears only the coverage, the results stay. Neither is possible while a run is going on.
A failing test does not stop the file : every test of the file runs and reports.

Run from the terminal : `npm run test`¶

The git repository of the account holds the local client (localClient.js), and its package.json has two scripts : npm run start keeps your files in sync with the server, npm run test runs the test cases.

npm run test                      # every test case of the account
npm run test -- orders            # only the test cases whose name, custom API or utility class contains "orders"
npm run test -- --no-coverage     # without the code coverage
npm run test -- --bail            # stop after the first test case file which fails
npm run test -- --json run.json   # also write the whole result to a file
npm run test -- --help            # every option

The test cases run on the server, in the sandbox of the account, with their mocks : npm run test only sends the request and prints the steps as they happen. Keep npm run start running in another terminal so the files of your folder are what the server runs.
The exit code is 0 when everything passed, 1 when a test failed, 2 when the run did not complete, so it fits in a script or a CI job.
Ctrl+C cancels the run on the server.

Code coverage in your IDE¶

Every run collects which lines ran. npm run test writes them to coverage/lcov.info (and coverage/coverage-summary.json) in the folder of the local client, in the standard lcov format, with the absolute paths of the files of the repository on your machine (<folder of the local client>/src/Custom APIs/orderCount/orderCount.ts, .../src/Utility classes/..., .../src/Test cases/...), the way istanbul writes them.

VS Code : install the extension Coverage Gutters. Run Coverage Gutters: Watch (or Display Coverage) from the command palette : the lines which ran are marked green in the gutter, the ones which did not red, in the files of src. The marks refresh after every npm run test.
WebStorm : Run → Show Coverage Data…, then + and pick coverage/lcov.info. The coverage shows in the gutter of the editor and in the project tree.
The coverage folder is in the .gitignore of the repository : it is made on every run, for the IDE, never for git.
The coverage counts the custom APIs run with g.sys.test.runCustomApi, the utility classes the tests use, and the test cases themselves. A custom API called through the web (g.sys.system.callExternalApi) runs in a request of its own and is not part of it. A sandbox running on Bun has no profiler : the run says so, and reports no coverage.

Run through the API¶

POST /api/sites/am-test-cases/<user path>/run runs the test cases and answers with the whole run : the plan, the result of every file and test, the totals and the coverage. The body holds testCaseIds (every test case when absent), filter, coverage, bail and timeoutMS. POST .../run/<runId>/cancel stops a run, GET .../run/last gives the last run of the account, DELETE .../run/last clears it and DELETE .../run/last/coverage clears only its coverage.