This page View Markdown Open in ChatGPT Open in Claude

Test cases¶

In API Maker, you can write test cases of your custom APIs and utility classes.
They run on the server, in the sandbox of your account, with the same g your custom APIs get. You can run them from the Test Cases page (menu APIs & Logic → Test Cases), or from the folder of the local client with npm run test.
A test case is written in TypeScript or in any other language API Maker runs : Python, PHP, C#, Go, Rust, Java or Kotlin. The custom APIs it tests can be written in any language too, the same one or another.
Every run reports which lines of your custom APIs, utility classes and test cases ran (code coverage), in every language, on the page and in your IDE.

Quick start¶

Add a test case APIs & Logic → Test Cases → add : a name, the type (custom API or utility class) and the item it tests. Saving gives you its default code.
Write the tests Call the custom API with g.sys.test.runCustomApi (run_custom_api in Python…) and check what it answers. Give mocks to the calls of g.sys which must not reach the database or the web.
Run them Ctrl+Enter on the page, or npm run test in the folder of the local client : the results, then the lines which ran.
Run them on every push The same command in a CI job, its exit code 0 when everything passed.

Run from the terminal : `npm run test`¶

The git repository of the account holds the local client (localClient.js), and its package.json has two scripts, committed with the repository : npm run start keeps your files in sync with the server, npm run test runs the test cases. Neither needs npm install.

npm run test                      # every test case of the account, as they are in this folder
npm run test -- orders            # only the test cases whose name, custom API or utility class contains "orders"
npm run test -- --no-sync         # run what the server has, without sending the changes of this folder
npm run test -- --push            # this folder wins where it differs from the server (a CI job)
npm run test -- --no-coverage     # without the code coverage
npm run test -- --bail            # stop after the first test case file which fails
npm run test -- --json run.json   # also write the whole result to a file
npm run test -- --help            # every option

The code of your folder runs. npm run test first merges the folder with the server, as npm run start does when it starts : what you changed in the folder (a test case, the custom API it tests, a utility class) is sent to the server, what changed on the server is written to the folder, and the run waits until the server saved what was sent. Then the test cases run on the server, in the sandbox of the account, with their mocks, and every step is printed as it happens.
When npm run start runs in the same folder, it keeps the folder in sync already : npm run test starts the run at once. --no-sync runs what the server has, without merging.
The first merge of a folder which never synced (a fresh clone) takes the version of the server for every file which differs, and keeps the local one in .local-client/backups. With --push the folder wins instead : every item which differs from the server is sent to it (what only the server has is written to the folder, nothing is deleted on the server). An item the server can not save (code which does not compile) is said, and the run goes on with what the server has for it.
The exit code is 0 when everything passed, 1 when a test failed, 2 when the run did not complete, so it fits in a script or a CI job.
Ctrl+C cancels the run on the server.

In a CI job¶

local-client.config.js holds the sync token and is not in git. A CI job gives the settings as variables of its environment instead : AM_WEB_SOCKET_URL, AM_SYNC_TOKEN (a secret of the job) and AM_ADMIN_USER_PATH. A variable wins over the file when both are there.

.github/workflows/test-cases.yml

name: test cases
on: [push]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      - run: npm run test -- --push --json test-run.json
        env:
          AM_WEB_SOCKET_URL: wss://api.example.com
          AM_SYNC_TOKEN: ${{ secrets.AM_SYNC_TOKEN }}
          AM_ADMIN_USER_PATH: admin

A CI job clones the repository afresh : --push sends the code of the commit to the server, so the test cases of the commit run against the code of the commit. Point the job at a server which may take the code of the branch : a server for development or for tests, never production.
coverage/lcov.info and the file of --json are there after the run, for a coverage service or an artifact of the job.

Code coverage in your IDE¶

Every run collects which lines ran. npm run test writes them to coverage/lcov.info (and coverage/coverage-summary.json) in the folder of the local client, in the standard lcov format, with the absolute paths of the files of the repository on your machine (<folder of the local client>/src/custom_apis/orderCount/orderCount.ts, .../src/utility_classes/..., .../src/test_cases/... ; a file of another language by its own extension, orderCount.py), the way istanbul writes them.

VS Code : install the extension Coverage Gutters. Run Coverage Gutters: Watch (or Display Coverage) from the command palette : the lines which ran are marked green in the gutter, the ones which did not red, in the files of src. The marks refresh after every npm run test.
WebStorm : Run → Show Coverage Data…, then + and pick coverage/lcov.info. The coverage shows in the gutter of the editor and in the project tree.
The coverage folder is in the .gitignore of the repository : it is made on every run, for the IDE, never for git.
The coverage counts the custom APIs run with g.sys.test.runCustomApi, the utility classes the tests use, and the test cases themselves. A custom API called through the web (g.sys.system.callExternalApi) runs in a request of its own and is not part of it.
Every language counts its own lines : TypeScript through the profiler of V8 (Node.js), Python with sys.monitoring, Go with go tool cover, PHP, C# and Rust with a counter the runner writes before each statement, on the line of the statement, and Java and Kotlin with counters written into the compiled classes, at each line the compiler noted, all in a build of the code for the test run only. A line is a line of the code as it was written; the code which serves the API is never touched.
The utility classes are counted the same way, in their own language, and are in the report with the files of their classes in the repository (src/utility_classes/utils---Tax/Tax.py). A class the tests did not use is not in the report.
A statement which is the only one of an if or a loop without braces is counted with the if or the loop, not on its own (PHP, C#). A member of C# written as an expression (public static double Add(double amount) => ...;) counts its line.
A sandbox running on Bun has no profiler : the run says so, and reports no coverage of TypeScript.
Java and Kotlin count their lines on JDK 24 and later : on an older JDK (21) the tests run as well, and the coverage has no lines of theirs.
The file of a custom API in the report is its file in git : src/custom_apis/shippingCost/shippingCost.ts, or .py, .php, .cs, .go, .rs, .java, .kt for code of another language.

The Test Cases page¶

The page has three parts :

Left : every test case of the account, with the state its last run gave it (passed, failed, not run). Search by name, custom API or utility class, filter by state, use the arrow keys to move and Enter to run. The play button next to a test case runs that one.
Middle : the code of the selected test case. Ctrl+S saves, Ctrl+Enter runs it. After a run, the line of every failure is marked, and the gutter shows which lines ran (coverage). The Mocks button opens the mocks of the test case (see below). The picker in the title bar chooses the language of the test case and the runtime which runs it : a test case nobody wrote in yet becomes the default test case of the language you pick.
Right : the run going on, or the last one : each file and each test as the server reports them, the failures with what was expected and what came, then the coverage of everything which ran. A run started from a terminal (npm run test) or from another tab shows here as well.

The switch Collect the code coverage at the bottom of the list decides whether the runs of this browser collect the coverage. It slows a run a little.

Add a new test case¶

Basic info¶

Give the name of the test case.
Select the type of the test case : Custom API or Utility class.
Select the custom API or the utility class it is about ('Test Item').

Mocks¶

The Mocks tab holds the mocks of the test case : calls of g.sys (g.sys.db.getAll, g.sys.system.hash, g.sys.cache.getKey ...) which answer with your data instead of reaching the database, redis or the web. When the tab is empty, it offers mocks to start from : rows of a table, one row by its id, a count, an external API, a secret, a cached value, a call which fails.
Each mock reads as a sentence : when the test calls a method, with the arguments (optional, one condition per argument, in order), answer with a value or fail with an error, for a number of calls (optional).
Write values the way a test writes them : JSON ({ "collection": "orders" }, [1, 2], "text", 42, true, null), objects written the JavaScript way ({ collection: 'orders' }) or plain text. The type comes from what you write.
An object matches an argument which has the same keys with the same values, inside objects too : { "collection": "orders", "find": { "status": "paid" } } matches a query of the orders whose find has status: 'paid', whatever else the query and its find have. A list matches a list of as many items. A key whose value is null matches null or no such key. A text, a number or a boolean must be equal. Without a condition, the mock answers every call of the method. An empty answer answers null.
Fail with an error : the call fails as the API would, and the code sees its errors. Write the message (The database is down.), an error ({ "message": "The database is down.", "code": 503 }) or a list of errors ; an error without a code has 500. A call which asks for the whole answer (getFullResponse) gets { success: false, statusCode, errors } instead. Use it to test what your code does when the database, the cache or another server fails.
For a number of calls : the mock answers the first calls it matches, then the mocks below it (or the real API) answer. A first call which fails and a second one which works, to test a retry.
The mocks are tried from the top : the first one whose method and arguments match answers. The tab warns about a mock which an earlier one hides, and about a condition on an argument the call never receives.
In code shows the same mock written for the code of a test, ready to copy.
Mocks can also be written in the code, next to the test which needs them (see below). They come first, so they win over the mocks of the tab.

Description¶

Here you can add more readable text to understand the test case.

Once you save the new test case you get its default code, in the language picked in the title bar :

TypeScriptPythonPHPC#GoRustJavaKotlin

import * as T from 'types';
import * as db from 'db-interfaces';
import * as assert from 'node:assert';

module.exports = [{
    name: 'Test case name 1',
    code: async (g: T.IAMGlobal) => {
        assert.strictEqual(10, 10);
    }
}];

# A test case of Python : every function whose name starts with test_ is a test. What it raises is its failure.
#
#     out = await g.sys.test.run_custom_api("orders", body={"amount": 100})
#     await g.sys.test.mock("db.getAll", when={"collection": "customers"}, returns=[{"id": 1}])

from am import G


async def test_case_name_1(g: G):
    assert 4 + 6 == 10

<?php
// A test case of PHP : every function whose name starts with test_ is a test. What it throws is its failure.
//
//     $out = $g->sys->test->runCustomApi('orders', body: ['amount' => 100]);
//     $g->sys->test->mock('db.getAll', when: ['collection' => 'customers'], returns: [['id' => 1]]);

function test_case_name_1(AmG $g): void
{
    if (4 + 6 !== 10) {
        throw new Exception('4 + 6 is not 10');
    }
}

// A test case of C# : every static method whose name starts with Test and which takes g is a test. What it throws
// is its failure.
//
//     var output = await g.Sys.Test.RunCustomApi(new RunCustomApi("orders") { Body = new { amount = 100 } });
//     await g.Sys.Test.Mock(new Mock("db.getAll") { When = new { collection = "customers" }, Return = new[] { new { id = 1 } } });

file static class Tests
{
    public static void TestCaseName1(G g)
    {
        var sum = 4 + 6;
        if (sum != 10) throw new Exception($"4 + 6 is {sum}, not 10");
    }
}

// A test case of Go : its tests are listed in Tests. What a test returns as an error, or panics with, is its failure.
//
//  out, err := g.Sys.Test.RunCustomApi(am.RunCustomApi{Name: "orders", Body: am.M{"amount": 100}})
//  err := g.Sys.Test.Mock(am.Mock{Method: "db.getAll", When: am.M{"collection": "customers"}, Return: []any{am.M{"id": 1}}})
package code

import (
    "fmt"

    "apimaker/am"
)

func testCaseName1(g *am.G) error {
    if sum := 4 + 6; sum != 10 {
        return fmt.Errorf("4 + 6 is %d, not 10", sum)
    }
    return nil
}

// Tests are the tests of this test case, in the order they run.
var Tests = []am.Test{
    {Name: "Test case name 1", Run: testCaseName1},
}

// A test case of Rust : its tests are named in am::tests!. What a test returns as errors, or panics with (assert_eq!),
// is its failure.
//
//     let out = g.sys.test.run_custom_api(RunCustomApi::new("orders").body(json!({ "amount": 100 })))?;
//     g.sys.test.mock(Mock::new("db.getAll").when(json!({ "collection": "customers" })).returns(json!([{ "id": 1 }])))?;

use am::{Errors, G};

fn test_case_name_1(g: &mut G) -> Result<(), Errors> {
    assert_eq!(4 + 6, 10);
    Ok(())
}

am::tests!(test_case_name_1);

import am.G;

// A test case of Java : every static method whose name starts with test and which takes g is a test. What it throws
// is its failure.
//
//     Object output = g.sys.test.runCustomApi(new RunCustomApi("orders").body(JsonObject.of("amount", 100)));
//     g.sys.test.mock(new Mock("db.getAll").when(JsonObject.of("collection", "customers")).returns(List.of(JsonObject.of("id", 1))));

class Tests {
    static void testCaseName1(G g) {
        int sum = 4 + 6;
        if (sum != 10) throw new IllegalStateException("4 + 6 is " + sum + ", not 10");
    }
}

import am.G

// A test case of Kotlin : every function whose name starts with test and which takes g is a test. What it throws is
// its failure.
//
//     val output = g.sys.test.runCustomApi(RunCustomApi("orders").body(mapOf("amount" to 100)))
//     g.sys.test.mock(Mock("db.getAll").whenever(mapOf("collection" to "customers")).returns(listOf(mapOf("id" to 1))))

fun testCaseName1(g: G) {
    val sum = 4 + 6
    check(sum == 10) { "4 + 6 is $sum, not 10" }
}

In TypeScript a test case exports a list of tests, one object per test with its name and its code ; the other languages name their tests the way their own test tools do (see the table below).
Every test gets the same g as a custom API : g.sys for the APIs of API Maker, g.logger for logs which show with the result, g.req and g.res.

Mocks in the code¶

Next to a test, mocks holds the mocks of that test alone (in TypeScript) ; the other languages add them with g.sys.test.mocks(…) at the start of the test. A mock answers with a value, fails the call with an error, or answers with a function of the arguments of the call, as many times as it says, in every language :

TypeScriptPythonPHPC#GoRustJavaKotlin

import * as T from 'types';
import * as assert from 'node:assert';

const ORDERS = { instance: 'mysql_8', database: 'shop', collection: 'orders' };

module.exports = [{
    name: 'mocks answer values, errors and functions',
    mocks: [
        // a value, when the query has these keys (inner objects too)
        { method: 'db.query', when: [{ collection: 'orders', find: { status: 'paid' } }], return: [{ id: 1, total: 20 }, { id: 2, total: 30 }] },
        // an error, for the first call only : the next call gets the next mock
        { method: 'db.count', error: { message: 'The database is busy.', code: 503 }, times: 1 },
        { method: 'db.count', return: 2 },
        // a function of the arguments of the call (it may be async)
        { method: 'db.getById', return: (query: any) => ({ id: query.id, total: query.id * 10 }) },
    ],
    code: async (g: T.IAMGlobal) => {
        const rows = await g.sys.db.query({ ...ORDERS, find: { status: 'paid', customer: 7 } });
        assert.strictEqual(rows.length, 2);
        await assert.rejects(g.sys.db.count(ORDERS), (errors: any) => errors[0].code === 503);
        assert.strictEqual(await g.sys.db.count(ORDERS), 2);
        const order: any = await g.sys.db.getById({ ...ORDERS, id: 4 });
        assert.strictEqual(order.total, 40);
        assert.strictEqual(g.sys.test.calls('db.count').length, 2); // the calls the mocks answered
    },
}];

from am import Errors, G, Mock


async def test_mocks_answer_values_errors_and_functions(g: G):
    await g.sys.test.mocks([
        # a value, when the query has these keys (inner objects too)
        Mock("db.query", when={"collection": "orders", "find": {"status": "paid"}}, returns=[{"id": 1, "total": 20}, {"id": 2, "total": 30}]),
        # an error, for the first call only : the next call gets the next mock
        Mock("db.count", error={"message": "The database is busy.", "code": 503}, times=1),
        Mock("db.count", returns=2),
        # a function of the arguments of the call (it may be async)
        Mock("db.getById", returns=lambda query: {"id": query["id"], "total": query["id"] * 10}),
    ])
    rows = await g.sys.db.query("mysql_8", "shop", "orders", find={"status": "paid", "customer": 7})
    assert len(rows) == 2
    try:
        await g.sys.db.count("mysql_8", "shop", "orders")
    except Errors as busy:
        assert busy.code == 503
    else:
        raise AssertionError("the first count did not fail")
    assert await g.sys.db.count("mysql_8", "shop", "orders") == 2
    order = await g.sys.db.get_by_id("mysql_8", "shop", "orders", 4)
    assert order["total"] == 40
    assert len(await g.sys.test.calls("db.count")) == 2  # the calls the mocks answered

<?php
function test_mocks_answer_values_errors_and_functions(AmG $g): void
{
    $g->sys->test->mocks([
        // a value, when the query has these keys (inner objects too)
        new Mock('db.query', when: ['collection' => 'orders', 'find' => ['status' => 'paid']], returns: [['id' => 1, 'total' => 20], ['id' => 2, 'total' => 30]]),
        // an error, for the first call only : the next call gets the next mock
        new Mock('db.count', error: ['message' => 'The database is busy.', 'code' => 503], times: 1),
        new Mock('db.count', returns: 2),
        // a function of the arguments of the call
        new Mock('db.getById', returns: fn($query) => ['id' => $query->id, 'total' => $query->id * 10]),
    ]);
    $rows = $g->sys->db->query('mysql_8', 'shop', 'orders', find: ['status' => 'paid', 'customer' => 7]);
    if (count($rows) !== 2) {
        throw new Exception('2 rows were expected');
    }
    try {
        $g->sys->db->count('mysql_8', 'shop', 'orders');
        throw new Exception('the first count did not fail');
    } catch (AmErrors $busy) {
        if ($busy->getCode() !== 503) {
            throw $busy;
        }
    }
    if ($g->sys->db->count('mysql_8', 'shop', 'orders') !== 2) {
        throw new Exception('2 was expected');
    }
    $order = $g->sys->db->getById('mysql_8', 'shop', 'orders', 4);
    if ($order->total !== 40) {
        throw new Exception('a total of 40 was expected');
    }
    if (count($g->sys->test->calls('db.count')) !== 2) {   // the calls the mocks answered
        throw new Exception('two calls were expected');
    }
}

public static class OrderTests
{
    public static async Task TestMocksAnswerValuesErrorsAndFunctions(G g)
    {
        await g.Sys.Test.Mocks([
            // a value, when the query has these keys (inner objects too)
            new Mock("db.query") { When = new { collection = "orders", find = new { status = "paid" } }, Return = new[] { new { id = 1, total = 20 }, new { id = 2, total = 30 } } },
            // an error, for the first call only : the next call gets the next mock
            new Mock("db.count") { Error = new { message = "The database is busy.", code = 503 }, Times = 1 },
            new Mock("db.count") { Return = 2 },
            // a function of the arguments of the call (Mock.AnswerAsync for an async one)
            new Mock("db.getById") { Return = Mock.Answer(args => new { id = args[0]!["id"]!.GetValue<int>(), total = args[0]!["id"]!.GetValue<int>() * 10 }) },
        ]);
        var rows = await g.Sys.Db.Query(new Query("mysql_8", "shop", "orders") { Find = new { status = "paid", customer = 7 } });
        if (rows.Count != 2) throw new Exception("2 rows were expected");
        try
        {
            await g.Sys.Db.Count(new Count("mysql_8", "shop", "orders"));
            throw new Exception("the first count did not fail");
        }
        catch (AmErrors busy) when (busy.Code == 503)
        {
        }
        if (await g.Sys.Db.Count(new Count("mysql_8", "shop", "orders")) != 2) throw new Exception("2 was expected");
        var order = await g.Sys.Db.GetById(new GetById("mysql_8", "shop", "orders", 4));
        if (order.GetInt64("total") != 40) throw new Exception("a total of 40 was expected");
        if ((await g.Sys.Test.Calls("db.count")).Count != 2) throw new Exception("two calls were expected");   // the calls the mocks answered
    }
}

package code

import (
    "errors"

    "apimaker/am"
)

func mocksAnswerValuesErrorsAndFunctions(g *am.G) error {
    err := g.Sys.Test.Mocks([]am.Mock{
        // a value, when the query has these keys (inner objects too)
        {Method: "db.query", When: am.M{"collection": "orders", "find": am.M{"status": "paid"}}, Return: []any{am.M{"id": 1, "total": 20}, am.M{"id": 2, "total": 30}}},
        // an error, for the first call only : the next call gets the next mock
        {Method: "db.count", Error: am.M{"message": "The database is busy.", "code": 503}, Times: 1},
        {Method: "db.count", Return: 2},
        // a function of the arguments of the call
        {Method: "db.getById", Return: func(query am.M) (any, error) {
            return am.M{"id": query.Int("id"), "total": query.Int("id") * 10}, nil
        }},
    })
    if err != nil {
        return err
    }
    orders := am.Query{Instance: "mysql_8", Database: "shop", Collection: "orders", Find: am.M{"status": "paid", "customer": 7}}
    rows, err := g.Sys.Db.Query(orders)
    if err != nil {
        return err
    }
    if len(rows) != 2 {
        return errors.New("2 rows were expected")
    }
    counted := am.Count{Instance: "mysql_8", Database: "shop", Collection: "orders"}
    var busy *am.Errors
    if _, err := g.Sys.Db.Count(counted); !errors.As(err, &busy) || busy.Code() != 503 {
        return errors.New("the first count did not fail")
    }
    if count, err := g.Sys.Db.Count(counted); err != nil || count != 2 {
        return errors.New("2 was expected")
    }
    order, err := g.Sys.Db.GetById(am.GetById{Instance: "mysql_8", Database: "shop", Collection: "orders", Id: 4})
    if err != nil {
        return err
    }
    if order.Int("total") != 40 {
        return errors.New("a total of 40 was expected")
    }
    calls, err := g.Sys.Test.Calls("db.count") // the calls the mocks answered
    if err != nil {
        return err
    }
    if len(calls) != 2 {
        return errors.New("two calls were expected")
    }
    return nil
}

var Tests = []am.Test{
    {Name: "mocks answer values, errors and functions", Run: mocksAnswerValuesErrorsAndFunctions},
}

use am::{Count, Errors, G, GetById, Mock, Query, json};

fn mocks_answer_values_errors_and_functions(g: &mut G) -> Result<(), Errors> {
    g.sys.test.mocks(vec![
        // a value, when the query has these keys (inner objects too)
        Mock::new("db.query").when(json!({ "collection": "orders", "find": { "status": "paid" } })).returns(json!([{ "id": 1, "total": 20 }, { "id": 2, "total": 30 }])),
        // an error, for the first call only : the next call gets the next mock
        Mock::new("db.count").error(json!({ "message": "The database is busy.", "code": 503 })).times(1),
        Mock::new("db.count").returns(2),
        // a function of the arguments of the call
        Mock::new("db.getById").answers(|args| Ok(json!({ "id": args[0]["id"], "total": args[0]["id"].as_i64().unwrap_or(0) * 10 }))),
    ])?;
    let rows = g.sys.db.query(Query::new("mysql_8", "shop", "orders").find(json!({ "status": "paid", "customer": 7 })))?;
    assert_eq!(rows.len(), 2);
    match g.sys.db.count(Count::new("mysql_8", "shop", "orders")) {
        Err(busy) if busy.code() == 503 => {}
        other => return Err(am::fail(500, &format!("the first count did not fail : {other:?}"))),
    }
    assert_eq!(g.sys.db.count(Count::new("mysql_8", "shop", "orders"))?, 2);
    let order = g.sys.db.get_by_id(GetById::new("mysql_8", "shop", "orders", 4))?.unwrap_or_default();
    assert_eq!(order["total"], json!(40));
    assert_eq!(g.sys.test.calls(Some("db.count"))?.len(), 2); // the calls the mocks answered
    Ok(())
}

am::tests!(mocks_answer_values_errors_and_functions);

import am.AmErrors;
import am.Count;
import am.G;
import am.GetById;
import am.JsonObject;
import am.Mock;
import am.Query;
import java.util.List;
import java.util.Map;

class OrderTests {
    static void testMocksAnswerValuesErrorsAndFunctions(G g) {
        g.sys.test.mocks(List.of(
            // a value, when the query has these keys (inner objects too)
            new Mock("db.query").when(Map.of("collection", "orders", "find", Map.of("status", "paid"))).returns(List.of(Map.of("id", 1, "total", 20), Map.of("id", 2, "total", 30))),
            // an error, for the first call only : the next call gets the next mock
            new Mock("db.count").error(Map.of("message", "The database is busy.", "code", 503)).times(1),
            new Mock("db.count").returns(2),
            // a function of the arguments of the call
            new Mock("db.getById").answers(args -> {
                long id = ((JsonObject) args.get(0)).getLong("id");
                return Map.of("id", id, "total", id * 10);
            })));
        List<JsonObject> rows = g.sys.db.query(new Query("mysql_8", "shop", "orders").find(Map.of("status", "paid", "customer", 7)));
        if (rows.size() != 2) throw new IllegalStateException("2 rows were expected");
        try {
            g.sys.db.count(new Count("mysql_8", "shop", "orders"));
            throw new IllegalStateException("the first count did not fail");
        } catch (AmErrors busy) {
            if (busy.code != 503) throw busy;
        }
        if (g.sys.db.count(new Count("mysql_8", "shop", "orders")) != 2) throw new IllegalStateException("2 was expected");
        JsonObject order = g.sys.db.getById(new GetById("mysql_8", "shop", "orders", 4));
        if (order.getLong("total") != 40) throw new IllegalStateException("a total of 40 was expected");
        if (g.sys.test.calls("db.count").size() != 2) throw new IllegalStateException("two calls were expected");   // the calls the mocks answered
    }
}

import am.AmErrors
import am.Count
import am.G
import am.GetById
import am.JsonObject
import am.Mock
import am.Query

fun testMocksAnswerValuesErrorsAndFunctions(g: G) {
    g.sys.test.mocks(
        listOf(
            // a value, when the query has these keys (inner objects too)
            Mock("db.query").whenever(mapOf("collection" to "orders", "find" to mapOf("status" to "paid"))).returns(listOf(mapOf("id" to 1, "total" to 20), mapOf("id" to 2, "total" to 30))),
            // an error, for the first call only : the next call gets the next mock
            Mock("db.count").error(mapOf("message" to "The database is busy.", "code" to 503)).times(1),
            Mock("db.count").returns(2),
            // a function of the arguments of the call
            Mock("db.getById").answers { args ->
                val id = (args[0] as JsonObject).getLong("id")
                mapOf("id" to id, "total" to id * 10)
            },
        ),
    )
    val rows = g.sys.db.query(Query("mysql_8", "shop", "orders").find(mapOf("status" to "paid", "customer" to 7)))
    check(rows.size == 2) { "2 rows were expected" }
    try {
        g.sys.db.count(Count("mysql_8", "shop", "orders"))
        error("the first count did not fail")
    } catch (busy: AmErrors) {
        check(busy.code == 503) { "refused with ${busy.code}" }
    }
    check(g.sys.db.count(Count("mysql_8", "shop", "orders")) == 2L) { "2 was expected" }
    val order = g.sys.db.getById(GetById("mysql_8", "shop", "orders", 4))
    check(order.getLong("total") == 40L) { "a total of 40 was expected" }
    check(g.sys.test.calls("db.count").size == 2) { "two calls were expected" }   // the calls the mocks answered
}

A mock is made of :

Key	What it is
`method`	The call : the id of the API (`SCHEMA_GET_ALL`) or its name in code (`g.sys.db.getAll`, `db.getAll`). The methods of the Mocks tab are the ones which can be mocked : every call of `g.sys.db`, `g.sys.db.gen`, `g.sys.system` and `g.sys.cache`.
`when`	The condition of the first argument, or a list of conditions, one per argument, in order. A value (an object matches an argument which has what it has, inside objects too), or a function of the argument which says whether the mock answers. No `when` : every call of the method.
`return`	The answer : a value, or a function of the arguments of the call. What the function returns is the answer, what it throws (raises, returns as an error) is the failure of the call.
`error`	The call fails with these errors instead : a message, an error `{ message, code }`, a list of errors, or `true`. An error without a `code` has 500.
`times`	The mock answers this many calls, then the next mock (or the real API) answers. No `times` : every call.

A function is given the way each language gives one :

Language	An answer	A condition
TypeScript	`return: (query) => …`, `async` too	`when: [(query) => …]`
Python	`returns=lambda query: …`, or a function, `async def` too	`when=lambda query: …`
PHP	`returns: fn($query) => …`	`when: fn($query) => …`
C#	`Return = Mock.Answer(args => …)`, `Mock.AnswerAsync(async args => …)`	`When = Mock.Match(query => …)`
Go	`Return: func(query am.M) (any, error) { … }`, or an `am.MockAnswer` of the arguments	`When: func(query am.M) bool { … }`
Rust	`.answers(\\|args\\| Ok(…))`	`.when_matches(\\|query\\| …)`
Java	`.answers(args -> …)`	`.whenMatches(query -> …)`
Kotlin	`.answers { args -> … }`	`.whenMatches { query -> … }`

The function runs in the test, in its language, also when the call comes from a custom API the test runs in another language : the API asks the test.
g.sys.test.mock(...) adds a mock while the test runs, g.sys.test.clearMocks(method?) removes mocks, g.sys.test.calls(method?) gives the calls the mocks answered so far in this test.
A call which asks for the whole answer (getFullResponse, g.sys.db.full…) gets what the mock answers as its data, or { success: false, statusCode, errors } from a mock which fails it.
A call which no mock answers reaches the real API, with the full permissions of the account.

Testing a custom API¶

g.sys.test.runCustomApi runs the code of a custom API of the account inside the test, with the request you describe. The g.sys calls the API makes are answered by the mocks of the test, and the coverage of the run sees the API. The hooks of the API are not run : it is the code of the API alone.

TypeScriptPythonPHPC#GoRustJavaKotlin

import * as T from 'types';
import * as assert from 'node:assert';

module.exports = [{
    name: 'orderCount answers the number of orders',
    mocks: [{ method: 'db.getAll', when: [{ collection: 'orders' }], return: [{ id: 1 }, { id: 2 }, { id: 3 }] }],
    code: async (g: T.IAMGlobal) => {
        const out = await g.sys.test.runCustomApi({ name: 'orderCount', query: { customer: 7 }, headers: { 'x-role': 'admin' } });
        assert.deepStrictEqual(out, { count: 3 });

        // the whole answer : data, logs, headers, warnings
        const full = await g.sys.test.runCustomApi({ name: 'orderCount', query: { customer: 7 } }, true);
        assert.deepStrictEqual(full.logs, ['3 orders']);

        // what the API throws is thrown here
        await assert.rejects(g.sys.test.runCustomApi({ name: 'orderCount', body: { fail: true } }), /failed on purpose/);
    },
}];

from am import Errors, G


async def test_order_count_answers_the_number_of_orders(g: G):
    await g.sys.test.mock('db.getAll', when={'collection': 'orders'}, returns=[{'id': 1}, {'id': 2}, {'id': 3}])
    out = await g.sys.test.run_custom_api('orderCount', query={'customer': 7}, headers={'x-role': 'admin'})
    assert out == {'count': 3}, out

    # the whole answer : data, logs, headers, warnings
    full = await g.sys.test.run_custom_api_full('orderCount', query={'customer': 7})
    assert full.logs == ['3 orders'], full.logs

    # what the API raises is raised here
    try:
        await g.sys.test.run_custom_api('orderCount', body={'fail': True})
    except Errors as refused:
        assert 'failed on purpose' in refused.message
    else:
        raise AssertionError('orderCount did not fail')

<?php
function test_order_count_answers_the_number_of_orders(AmG $g): void
{
    $g->sys->test->mock('db.getAll', when: ['collection' => 'orders'], returns: [['id' => 1], ['id' => 2], ['id' => 3]]);
    $out = $g->sys->test->runCustomApi('orderCount', query: ['customer' => 7], headers: ['x-role' => 'admin']);
    if ($out->count !== 3) {
        throw new Exception('3 orders were expected');
    }

    // the whole answer : data, logs, headers, warnings
    $full = $g->sys->test->runCustomApiFull('orderCount', query: ['customer' => 7]);
    if ($full->logs !== ['3 orders']) {
        throw new Exception('the log of the count was expected');
    }

    // what the API throws is thrown here
    try {
        $g->sys->test->runCustomApi('orderCount', body: ['fail' => true]);
    } catch (AmErrors $refused) {
        if (!str_contains($refused->getMessage(), 'failed on purpose')) {
            throw $refused;
        }
        return;
    }
    throw new Exception('orderCount did not fail');
}

public static class OrderCountTests
{
    public static async Task TestOrderCountAnswersTheNumberOfOrders(G g)
    {
        await g.Sys.Test.Mock(new Mock("db.getAll") { When = new { collection = "orders" }, Return = new[] { new { id = 1 }, new { id = 2 }, new { id = 3 } } });
        var output = await g.Sys.Test.RunCustomApi(new RunCustomApi("orderCount") { Query = new { customer = 7 }, Headers = new JsonObject { ["x-role"] = "admin" } });
        if (output.GetInt64("count") != 3) throw new Exception("3 orders were expected");

        // the whole answer : data, logs, headers, warnings
        var full = await g.Sys.Test.RunCustomApiFull(new RunCustomApi("orderCount") { Query = new { customer = 7 } });
        if (!full.Logs.SequenceEqual(["3 orders"])) throw new Exception("the log of the count was expected");

        // what the API throws is thrown here
        try
        {
            await g.Sys.Test.RunCustomApi(new RunCustomApi("orderCount") { Body = new { fail = true } });
        }
        catch (AmErrors refused) when (refused.Message.Contains("failed on purpose"))
        {
            return;
        }
        throw new Exception("orderCount did not fail");
    }
}

package code

import (
    "errors"
    "slices"
    "strings"

    "apimaker/am"
)

func orderCountAnswersTheNumberOfOrders(g *am.G) error {
    err := g.Sys.Test.Mock(am.Mock{Method: "db.getAll", When: am.M{"collection": "orders"}, Return: []any{am.M{"id": 1}, am.M{"id": 2}, am.M{"id": 3}}})
    if err != nil {
        return err
    }
    out, err := g.Sys.Test.RunCustomApi(am.RunCustomApi{Name: "orderCount", Query: am.M{"customer": 7}, Headers: am.M{"x-role": "admin"}})
    if err != nil {
        return err
    }
    if am.AsM(out).Int("count") != 3 {
        return errors.New("3 orders were expected")
    }

    // the whole answer : data, logs, headers, warnings
    full, err := g.Sys.Test.RunCustomApiFull(am.RunCustomApi{Name: "orderCount", Query: am.M{"customer": 7}})
    if err != nil {
        return err
    }
    if !slices.Equal(full.Logs, []string{"3 orders"}) {
        return errors.New("the log of the count was expected")
    }

    // what the API returns as an error is returned here
    _, err = g.Sys.Test.RunCustomApi(am.RunCustomApi{Name: "orderCount", Body: am.M{"fail": true}})
    if err == nil || !strings.Contains(err.Error(), "failed on purpose") {
        return errors.New("orderCount did not fail")
    }
    return nil
}

var Tests = []am.Test{
    {Name: "orderCount answers the number of orders", Run: orderCountAnswersTheNumberOfOrders},
}

use am::{Errors, G, Mock, RunCustomApi, json};

fn order_count_answers_the_number_of_orders(g: &mut G) -> Result<(), Errors> {
    g.sys.test.mock(Mock::new("db.getAll").when(json!({ "collection": "orders" })).returns(json!([{ "id": 1 }, { "id": 2 }, { "id": 3 }])))?;
    let out = g.sys.test.run_custom_api(RunCustomApi::new("orderCount").query(json!({ "customer": 7 })).headers(json!({ "x-role": "admin" })))?;
    assert_eq!(out, json!({ "count": 3 }));

    // the whole answer : data, logs, headers, warnings
    let full = g.sys.test.run_custom_api_full(RunCustomApi::new("orderCount").query(json!({ "customer": 7 })))?;
    assert_eq!(full.logs, ["3 orders"]);

    // what the API returns as errors is returned here
    match g.sys.test.run_custom_api(RunCustomApi::new("orderCount").body(json!({ "fail": true }))) {
        Err(refused) if refused.message().contains("failed on purpose") => Ok(()),
        other => Err(am::fail(500, &format!("orderCount did not fail : {other:?}"))),
    }
}

am::tests!(order_count_answers_the_number_of_orders);

import am.AmErrors;
import am.G;
import am.JsonObject;
import am.Mock;
import am.RunCustomApi;
import am.RunCustomApiResponse;
import java.util.List;
import java.util.Map;

class OrderCountTests {
    static void testOrderCountAnswersTheNumberOfOrders(G g) {
        g.sys.test.mock(new Mock("db.getAll").when(Map.of("collection", "orders")).returns(List.of(Map.of("id", 1), Map.of("id", 2), Map.of("id", 3))));
        JsonObject output = JsonObject.from(g.sys.test.runCustomApi(new RunCustomApi("orderCount").query(Map.of("customer", 7)).headers(Map.of("x-role", "admin"))));
        if (output.getLong("count") != 3) throw new IllegalStateException("3 orders were expected");

        // the whole answer : data, logs, headers, warnings
        RunCustomApiResponse full = g.sys.test.runCustomApiFull(new RunCustomApi("orderCount").query(Map.of("customer", 7)));
        if (!full.getLogs().equals(List.of("3 orders"))) throw new IllegalStateException("the log of the count was expected");

        // what the API throws is thrown here
        try {
            g.sys.test.runCustomApi(new RunCustomApi("orderCount").body(Map.of("fail", true)));
        } catch (AmErrors refused) {
            if (refused.getMessage().contains("failed on purpose")) return;
            throw refused;
        }
        throw new IllegalStateException("orderCount did not fail");
    }
}

import am.AmErrors
import am.G
import am.JsonObject
import am.Mock
import am.RunCustomApi

fun testOrderCountAnswersTheNumberOfOrders(g: G) {
    g.sys.test.mock(Mock("db.getAll").whenever(mapOf("collection" to "orders")).returns(listOf(mapOf("id" to 1), mapOf("id" to 2), mapOf("id" to 3))))
    val output = JsonObject.from(g.sys.test.runCustomApi(RunCustomApi("orderCount").query(mapOf("customer" to 7)).headers(mapOf("x-role" to "admin"))))
    check(output.getLong("count") == 3L) { "3 orders were expected" }

    // the whole answer : data, logs, headers, warnings
    val full = g.sys.test.runCustomApiFull(RunCustomApi("orderCount").query(mapOf("customer" to 7)))
    check(full.logs == listOf("3 orders")) { "the log of the count was expected" }

    // what the API throws is thrown here
    try {
        g.sys.test.runCustomApi(RunCustomApi("orderCount").body(mapOf("fail" to true)))
    } catch (refused: AmErrors) {
        check(refused.message?.contains("failed on purpose") == true) { "refused with ${refused.message}" }
        return
    }
    error("orderCount did not fail")
}

name is the name of the custom API, version (its version or the name of the version) runs another version than the one the API serves.
body, query, params and headers are the request the API sees.

A custom API in another language¶

A custom API written in Python, PHP, C#, Go, Rust, Java or Kotlin is tested the same way : g.sys.test.runCustomApi runs it in the sandbox of its language (its group of separate sandboxes when it has one), and each of its calls of g.sys asks the test first whether one of its mocks answers it. Mocks which are functions run in the test, in the language of the test; what they return reaches the API in its language.

The custom API 'shippingCost', in Python
async def main(g):
    rates = await g.sys.db.get_all('mysql_8', 'shop', 'rates')
    rate = rates[0]['per_kg']
    if g.req.body.weight > 30:
        raise g.Errors('Too heavy to ship.', code=422)
    return {'cost': g.req.body.weight * rate}

TypeScriptPythonPHPC#GoRustJavaKotlin

Its test case
import * as T from 'types';
import * as assert from 'node:assert';

module.exports = [{
    name: 'the cost of a parcel',
    mocks: [{ method: 'db.getAll', return: [{ per_kg: 2.5 }] }],
    code: async (g: T.IAMGlobal) => {
        assert.deepStrictEqual(await g.sys.test.runCustomApi({ name: 'shippingCost', body: { weight: 4 } }), { cost: 10 });
        assert.strictEqual(g.sys.test.calls('db.getAll').length, 1);
        await assert.rejects(g.sys.test.runCustomApi({ name: 'shippingCost', body: { weight: 40 } }), (errors: any) => errors[0].code === 422);
    },
}];

Its test case
async def test_the_cost_of_a_parcel(g):
    await g.sys.test.mock("db.getAll", when={"collection": "rates"}, returns=[{"per_kg": 2.5}])
    out = await g.sys.test.run_custom_api("shippingCost", body={"weight": 4})
    assert out == {"cost": 10}, out
    assert len(await g.sys.test.calls("db.getAll")) == 1


async def test_too_heavy_is_refused(g):
    try:
        await g.sys.test.run_custom_api("shippingCost", body={"weight": 40})
    except g.Errors as refused:
        assert refused.code == 422
    else:
        raise AssertionError("a parcel of 40 kg was accepted")

Its test case
<?php
function test_the_cost_of_a_parcel(AmG $g)
{
    $g->sys->test->mock('db.getAll', when: ['collection' => 'rates'], returns: [['per_kg' => 2.5]]);
    $out = $g->sys->test->runCustomApi('shippingCost', body: ['weight' => 4]);
    if ($out->cost != 10) {
        throw new Exception("the cost is {$out->cost}");
    }
}

function test_too_heavy_is_refused(AmG $g)
{
    try {
        $g->sys->test->runCustomApi('shippingCost', body: ['weight' => 40]);
    } catch (AmErrors $refused) {
        if ($refused->getCode() !== 422) {
            throw new Exception("refused with {$refused->getCode()}");
        }
        return;
    }
    throw new Exception('a parcel of 40 kg was accepted');
}

Its test case
public static class ShippingTests
{
    public static async Task TestTheCostOfAParcel(G g)
    {
        await g.Sys.Test.Mock(new Mock("db.getAll") { When = new { collection = "rates" }, Return = new[] { new { per_kg = 2.5 } } });
        var output = await g.Sys.Test.RunCustomApi(new RunCustomApi("shippingCost") { Body = new { weight = 4 } });
        if (output.GetDouble("cost") != 10) throw new Exception("the cost is " + output?["cost"]);
    }

    public static async Task TestTooHeavyIsRefused(G g)
    {
        try
        {
            await g.Sys.Test.RunCustomApi(new RunCustomApi("shippingCost") { Body = new { weight = 40 } });
        }
        catch (AmErrors refused) when (refused.Code == 422)
        {
            return;
        }
        throw new Exception("a parcel of 40 kg was accepted");
    }
}

Its test case
package code

import (
    "errors"
    "fmt"

    "apimaker/am"
)

func theCostOfAParcel(g *am.G) error {
    if err := g.Sys.Test.Mock(am.Mock{Method: "db.getAll", When: am.M{"collection": "rates"}, Return: []any{am.M{"per_kg": 2.5}}}); err != nil {
        return err
    }
    out, err := g.Sys.Test.RunCustomApi(am.RunCustomApi{Name: "shippingCost", Body: am.M{"weight": 4}})
    if err != nil {
        return err
    }
    if cost := am.AsM(out).Float("cost"); cost != 10 {
        return fmt.Errorf("the cost is %v", cost)
    }
    return nil
}

func tooHeavyIsRefused(g *am.G) error {
    _, err := g.Sys.Test.RunCustomApi(am.RunCustomApi{Name: "shippingCost", Body: am.M{"weight": 40}})
    var refused *am.Errors
    if errors.As(err, &refused) && refused.Code() == 422 {
        return nil
    }
    return fmt.Errorf("a parcel of 40 kg was not refused : %v", err)
}

var Tests = []am.Test{
    {Name: "the cost of a parcel", Run: theCostOfAParcel},
    {Name: "too heavy is refused", Run: tooHeavyIsRefused},
}

Its test case
use am::{Errors, G, Mock, RunCustomApi, json};

fn the_cost_of_a_parcel(g: &mut G) -> Result<(), Errors> {
    g.sys.test.mock(Mock::new("db.getAll").when(json!({ "collection": "rates" })).returns(json!([{ "per_kg": 2.5 }])))?;
    let out = g.sys.test.run_custom_api(RunCustomApi::new("shippingCost").body(json!({ "weight": 4 })))?;
    assert_eq!(out["cost"].as_f64(), Some(10.0));
    Ok(())
}

fn too_heavy_is_refused(g: &mut G) -> Result<(), Errors> {
    match g.sys.test.run_custom_api(RunCustomApi::new("shippingCost").body(json!({ "weight": 40 }))) {
        Err(refused) if refused.code() == 422 => Ok(()),
        other => Err(am::fail(500, &format!("a parcel of 40 kg was not refused : {other:?}"))),
    }
}

am::tests!(the_cost_of_a_parcel, too_heavy_is_refused);

Its test case
import am.AmErrors;
import am.G;
import am.JsonObject;
import am.Mock;
import am.RunCustomApi;
import java.util.List;
import java.util.Map;

class ShippingTests {
    static void testTheCostOfAParcel(G g) {
        g.sys.test.mock(new Mock("db.getAll").when(Map.of("collection", "rates")).returns(List.of(Map.of("per_kg", 2.5))));
        JsonObject output = JsonObject.from(g.sys.test.runCustomApi(new RunCustomApi("shippingCost").body(Map.of("weight", 4))));
        if (output.getDouble("cost") != 10) throw new IllegalStateException("the cost is " + output.get("cost"));
    }

    static void testTooHeavyIsRefused(G g) {
        try {
            g.sys.test.runCustomApi(new RunCustomApi("shippingCost").body(Map.of("weight", 40)));
        } catch (AmErrors refused) {
            if (refused.code == 422) return;
            throw refused;
        }
        throw new IllegalStateException("a parcel of 40 kg was accepted");
    }
}

Its test case
import am.AmErrors
import am.G
import am.JsonObject
import am.Mock
import am.RunCustomApi

fun testTheCostOfAParcel(g: G) {
    g.sys.test.mock(Mock("db.getAll").whenever(mapOf("collection" to "rates")).returns(listOf(mapOf("per_kg" to 2.5))))
    val output = JsonObject.from(g.sys.test.runCustomApi(RunCustomApi("shippingCost").body(mapOf("weight" to 4))))
    check(output.getDouble("cost") == 10.0) { "the cost is " + output["cost"] }
}

fun testTooHeavyIsRefused(g: G) {
    try {
        g.sys.test.runCustomApi(RunCustomApi("shippingCost").body(mapOf("weight" to 40)))
    } catch (refused: AmErrors) {
        check(refused.code == 422) { "refused with ${refused.code}" }
        return
    }
    error("a parcel of 40 kg was accepted")
}

What the API raises comes back as its errors : a list in TypeScript, Errors in Python, AmErrors in PHP, C#, Java and Kotlin, *am.Errors in Go, Errors in Rust, with the code the API gave.
The coverage of the run has the lines of the Python code, as it was written.

Testing a utility class¶

A utility class is imported the way custom APIs import it, and its functions are called directly :

TypeScriptPythonPHPC#GoRustJavaKotlin

import * as T from 'types';
import * as assert from 'node:assert';
import * as Tax from 'utils/Tax';

module.exports = [{
    name: 'the tax of 100',
    code: async (g: T.IAMGlobal) => {
        assert.strictEqual(Tax.add(100), 120);
    },
}];

from am import G
from Tax import add


async def test_the_tax_of_100(g: G):
    assert add(100) == 120

<?php
function test_the_tax_of_100(AmG $g): void
{
    if (Tax::add(100) != 120) {
        throw new Exception('the tax of 100 is not 20');
    }
}

file static class TaxTests
{
    public static void TestTheTaxOf100(G g)
    {
        if (Tax.Add(100) != 120) throw new Exception("the tax of 100 is not 20");
    }
}

package code

import (
    "errors"

    "apimaker/am"
    tax "apimaker/src/utility_classes/utils---Tax"
)

func theTaxOf100(g *am.G) error {
    if tax.Add(100) != 120 {
        return errors.New("the tax of 100 is not 20")
    }
    return nil
}

var Tests = []am.Test{
    {Name: "the tax of 100", Run: theTaxOf100},
}

use am::{Errors, G};

use crate::Tax;

fn the_tax_of_100(g: &mut G) -> Result<(), Errors> {
    assert_eq!(Tax::add(100.0), 120.0);
    Ok(())
}

am::tests!(the_tax_of_100);

import am.G;

class TaxTests {
    static void testTheTaxOf100(G g) {
        if (Tax.add(100) != 120) throw new IllegalStateException("the tax of 100 is not 20");
    }
}

import am.G

fun testTheTaxOf100(g: G) {
    check(Tax.add(100.0) == 120.0) { "the tax of 100 is not 20" }
}

Tax is the utility class of the page of the utility classes. The g.sys calls a class makes are answered by the mocks of the test too.

Test cases in Python, PHP, C#, Go, Rust, Java and Kotlin¶

Pick the language in the title bar of the editor of the test case. Each language names its tests the way its own test tools do, and what a test raises is its failure :

Language	A test is	It fails when it
TypeScript	an object of the list the file exports : `{ name, code, mocks }`	throws (`node:assert`)
Python	a function whose name starts with `test_`, plain or `async`	raises (`assert`)
PHP	a function whose name starts with `test_`	throws
C#	a static method whose name starts with `Test` and which takes `G g`	throws
Go	an entry of the list `var Tests = []am.Test{ … }` : `{Name: "…", Run: fn}`, `fn` a `func(g *am.G) error`	returns an error, or panics
Rust	a `fn(g: &mut G) -> Result<(), Errors>` named in `am::tests!(…)`	returns `Err`, or panics (`assert_eq!`)
Java	a static method whose name starts with `test` and which takes `G g`, in any class	throws (not `assert` : a JVM checks it only with `-ea`)
Kotlin	a function at the top of the file whose name starts with `test` and which takes `g: G`	throws (`check`, `require`)

Every test gets the same g as the code of its language, and g.sys.test to run a custom API and to mock : the test case of shippingCost above is written in every language.

g.sys.test has the same calls in every language, named the way the language names things and with its types : runCustomApi (run_custom_api, RunCustomApi), mock, mocks (several at once), clearMocks, calls. runCustomApi takes name, version, body, query, params and headers, the way the language gives a call its keys (named arguments in Python and PHP, new RunCustomApi("name") { Body = … } in C#, am.RunCustomApi{Name: …} in Go, RunCustomApi::new("name").body(…) in Rust, new RunCustomApi("name").body(…) in Java and Kotlin) ; runCustomApiFull (run_custom_api_full, RunCustomApiFull) gives the whole answer : data, statusCode, headers, logs.
Mocks are the same in every language : method, when (the argument of the call, or a list of its arguments ; an object matches an argument which has the same keys with the same values, inside objects too), return (returns where return is a word of the language), error and times. A condition or an answer can be a function of the test, in every language : the runner of the test keeps it, and runs it when the mock answers a call.
The mocks of the Mocks tab work as they are, in every language. Every test of the file starts with them ; the mocks a test adds in its code come first, and are gone when the next test starts.
A test case tests code of any language : a test case in Python can run a custom API written in Go. The API runs in the sandbox of its own language, and the mocks of the test answer its calls of g.sys there too.
Utility classes are imported the way the other code of the language imports them : a test case in Python tests a utility class written in Python.
The coverage of a run has the lines of the test case, of the custom APIs it runs and of the utility classes they use, in every language. A file without tests is said in the plan of the run, with how the tests of its language are written.

Execute / Run¶

On the page, the play button next to a test case runs that one, Run in the editor runs the selected one, Run all runs every test case of the account.
The results show test by test : a failed test shows its message, and a click on it shows what was expected and what came, side by side, with the line of the test in the editor.
A run can be cancelled : it stops after the file which is running.
The last run is kept 30 days, so the page shows it again after a reload. Clear results above the results of the run panel clears it : its results and its coverage. Clear coverage above the coverage clears only the coverage, the results stay. Neither is possible while a run is going on.
A failing test does not stop the file : every test of the file runs and reports.

Run through the API¶

POST /api/sites/am-test-cases/<user path>/run runs the test cases and answers with the whole run : the plan, the result of every file and test, the totals and the coverage. The body holds testCaseIds (every test case when absent), filter, coverage, bail and timeoutMS. POST .../run/<runId>/cancel stops a run, GET .../run/last gives the last run of the account, DELETE .../run/last clears it and DELETE .../run/last/coverage clears only its coverage.

Test cases¶

Quick start¶

Run from the terminal : npm run test¶

In a CI job¶

Code coverage in your IDE¶

The Test Cases page¶

Add a new test case¶

Basic info¶

Mocks¶

Description¶

Mocks in the code¶

Testing a custom API¶

A custom API in another language¶

Testing a utility class¶

Test cases in Python, PHP, C#, Go, Rust, Java and Kotlin¶

Execute / Run¶

Run through the API¶

Run from the terminal : `npm run test`¶