Skip to content

26 · Testing your routing offline

The router is deterministic: the same turn, catalogue, vocabulary and context give the same plan, byte for byte. It runs in the licensed runtime on your machine, so your tests read every turn with no network already. What still reaches Stalwart's service is the rest: compiling a rulebook, reading your account, a solve. A cassette records those answers once, and your tests replay them with no network. This chapter is the recording, the replay, and what a replay does with a call it does not hold.

Language support

The selector reads turns written in English and in Brazilian Portuguese (pt-BR). The language is chosen, never guessed from the turn: the console's Settings page (Language the selector reads), language on a plan request, or the settings.language of your vocabulary file; with none of them, English. A Portuguese turn may be typed with its accents or without them. European Portuguese is not a grammar the selector has, and a turn in any other language is unread.

Record once, replay every run

record(path) and replay(path), from stalwart_agents.client.testing, are transports a Client takes. Recording talks to Stalwart's service and keeps every answer in the file; replaying answers from the file:

TURNS = ["show me order 4411", "charge order 4411 40.00", "zork the frobnicator"]

# Once, deliberately: talks to Stalwart's service and keeps every answer.
with Client(transport=record("service.cassette.json")) as client:
    recorded = client.compile_rulebook(rulebook)
    planned = [client.plan(turn, agent="desk", catalogue=catalogue).raw for turn in TURNS]

# In every test run after: the service's answers from the file, with no network.
with Client(transport=replay("service.cassette.json")) as client:
    compiled = client.compile_rulebook(rulebook)
    print(f"the rulebook compiled, the same record as recorded: {compiled == recorded}")
    # Plans are never in the cassette: the runtime reads each turn on this machine, in both runs.
    for turn, then in zip(TURNS, planned):
        plan = client.plan(turn, agent="desk", catalogue=catalogue)
        print(f"{turn!r}: {plan.outcome}, the same plan as before: {plan.raw == then}")

the rulebook compiled, the same record as recorded: True
'show me order 4411': run, the same plan as before: True
'charge order 4411 40.00': run, the same plan as before: True
'zork the frobnicator': unread, the same plan as before: True
CassetteMiss: the cassette holds no answer for GET /v1/account (key 89a246b29c7ddeeb)

A cassette holds the service's answers, not its logic. It is a file of requests and their answers, keyed by each request's method, path and body, so a replay is exact for the same request and a miss for any other. Commit it beside your tests.

Plans and tool checks are never in a cassette. The runtime answers them in your process, with no request, so they are as exact offline as online: in the output above, the runtime read each turn in both runs and gave the same plan.

Your client's upkeep with Stalwart's service -- renewing your machine's lease, its usage report -- moves with the clock, so a cassette neither records nor replays it: on a machine with a licence, it is sent as it would be without one. Everything else a replay answers from the file.

When a replay does not hold

with Client(transport=replay("service.cassette.json")) as client:
    try:
        client.account()                      # a call nobody recorded
    except CassetteMiss as miss:
        print("CassetteMiss:", miss)

The last line of the output:

  • CassetteMiss: the cassette has no answer for this request. A call nobody recorded, or one whose body moved: the test asked something new.

Re-recording, deliberately

A miss is the test doing its job: your code asks the service something it did not ask before. Re-record on purpose -- a CI job that runs the recording against Stalwart's service, or a command a developer runs -- and review the cassette's diff like code: each changed answer is a call that now answers differently.

The limits

Chapter 20's limits hold here, and these besides:

  • A cassette proves your code does what it did when it was recorded, with the service's answers of that day. It says nothing of calls it does not hold.
  • The runtime still answers under your licence: a test machine reads turns and checks calls with an activated licence, with or without a cassette.