Skip to content

24 · Checking a tool call

Not every call comes from a plan. Your model may choose a tool itself -- on a turn your setting handed it, or in a loop of its own -- and that call is checked before it runs exactly as a planned one is. This chapter is the check from your code's side: how to ask, what comes back, and what to do with each answer. Chapter 32 walks through every step of the check in order.

Language support

The selector reads turns written in English and in Brazilian Portuguese (pt-BR). The language is chosen, never guessed from the turn: the console's Settings page (Language the selector reads), language on a plan request, or the settings.language of your vocabulary file; with none of them, English. A Portuguese turn may be typed with its accents or without them. European Portuguese is not a grammar the selector has, and a turn in any other language is unread.

Asking

client.check_tool_call(tool, arguments, agent=...) checks one call; session.check_tool_call(...) checks it with the session's history (chapter 22). The check runs where the router runs: in the licensed runtime on your machine. Nothing of the call reaches Stalwart.

turns = ["charge order 4411 40.00"]           # the person's words, sent as evidence and never kept
facts = {"card_on_file": True}                # what the rulebook's $card_on_file is, as your code knows it
with Client() as client:
    history = History()                       # carried by you between checks, signed by the runtime

    def check(tool, arguments, **given):
        global history
        answer = client.check_tool_call(tool, arguments, agent="desk", catalogue=catalogue, rulebook=rulebook,
                                        turns=turns, history=history,
                                        session_id="chat-17", **given)
        history = answer.history
        print(f"{tool} {arguments}: {answer.decision} | {answer.reason} | {answer.reasons}"
              + (f" | at {answer.pointer}" if answer.pointer else ""))
        return answer

    check("get_order", {"order_id": "4411"}, origins={"order_id": "typed"})
    asked = check("charge_card", {"order_id": "4411", "amount": 40.00},
                  origins={"order_id": "typed", "amount": "typed"}, context=facts)
    # The person said yes: the same call, with the confirmation's token.
    check("charge_card", {"order_id": "4411", "amount": 40.00},
          origins={"order_id": "typed", "amount": "typed"}, context=facts, confirmation=asked.confirmation)
    check("charge_card", {"order_id": "4411", "amount": 400.00}, origins={"order_id": "typed", "amount": "model"},
          context=facts)
    check("delete_order", {"order_id": "4411"}, origins={"order_id": "typed"})
    check("get_order", {"order_id": "4411", "note": "urgent"}, origins={"order_id": "typed", "note": "model"})
    print(len(history.entries), "checks in the history")
get_order {'order_id': '4411'}: allow | allow | ['allow']
charge_card {'order_id': '4411', 'amount': 40.0}: confirm | write | ['write']
charge_card {'order_id': '4411', 'amount': 40.0}: allow | confirmed | ['confirmed']
charge_card {'order_id': '4411', 'amount': 400.0}: confirm | argument_origin: amount | ['argument_origin: amount']
delete_order {'order_id': '4411'}: refuse | unknown_tool | ['unknown_tool']
get_order {'order_id': '4411', 'note': 'urgent'}: refuse | argument_schema | ['argument_schema'] | at /note
6 checks in the history

What each call sends besides the tool and its arguments:

  • origins: where each argument came from -- typed by the person, model written by your model, a reply, your context, or result:<step> of an earlier call -- which decides which values a write accepts without asking;
  • turns: the person's own words, as evidence that a value claimed as typed was typed. Sent with the check and kept nowhere;
  • history: the checks made so far in the conversation (below);
  • context: the facts your rulebook reads, such as $card_on_file, as your code knows them: true or false by name. A fact not sent is false, so a rule that allows a charge only if $card_on_file refuses it (chapter 45 has both ways to give them);
  • confirmation: the token a person's approval returned, for the call they approved.

The answers

A ToolCheck has a decision -- allow, confirm or refuse -- the reason that decided it, every reason at once in reasons, and, for an argument, where in it (pointer). Like a plan, it has no truth value: confirm is neither yes nor no.

Answer What your code does
allow run the call once, with exactly these arguments, then report what became of it: session.ran(check, "ran"), or failed, outcome_unknown or compensated
confirm show the person the call, its arguments and where each came from, and the reasons; on a yes, check the same call again with confirmation=check.confirmation. An approval covers these arguments and no others
refuse run nothing. Give your model the reason as the tool's result, so it can tell the person or try something the check allows

In the output: the read is allowed; the charge the person typed asks, because every write is confirmed; the same charge with the approval's token is allowed (confirmed); an amount the model wrote asks again, naming the argument; delete_order is not in the catalogue; and note is no argument of get_order's, refused at /note.

The history

Every check adds an entry to the conversation's history: the step, the tool, the agent, digests of the arguments and their origins, the marks they carried, and the answer -- names and digests, never a value. Each answer carries a token signing that entry with every one before it, and the next check verifies the chain. You carry the history: in a Session it moves on by itself, and statelessly it is the History on each answer, which to_dict() stores and History.from_dict restores. A history edited, reordered or taken from another conversation fails the next check with history_invalid.

Receipts: checking once, enforcing twice

An enforcement point in front of your tools -- a gateway route, an API proxy -- sees each call and should not decide it a second time. An agent that checked its own call hands the enforcement point its history, and the receipt route of your own stalwart gateway (chapter 37) says whether that history is the receipt for exactly this call. The history's last entry is the receipt: allow with the reason receipt where it is the check allowing this tool with these arguments, for the agent the entry names; otherwise refuse, with receipt_missing (the history holds no entry), receipt_not_allowed (the last check did not allow its call), receipt_mismatch (a different tool, or different arguments, was checked), receipt_spent (an outcome is already reported for that call) or history_invalid. A receipt admits one call; the same call sent again is admitted only where sending it again is safe -- a read, a tool declared idempotent, or a write whose idempotency key rides in an argument. The answer's format is qls-tool-receipt/1, and a check's is qls-tool-check/1.

The limits

Chapter 20's limits hold here, and these besides:

  • The check decides by your catalogue, your rulebook and the history it is sent. A call your code runs without asking, or runs after a refuse, is not one it decided.
  • What became of a call -- whether it ran -- is your code's report. The next check reads it as reported.
  • It checks the call, not the reason for it: an allowed call is one your rules allow, not one the person necessarily wanted.