Tool Calling & Function Calling
From a Text Box to an Actor
A bare language model can only do one thing: read text and produce more text. It cannot look up today's weather, run a calculation it is unsure about, query a database, or send an email. Tool calling (also called function calling) is the mechanism that bridges this gap. It turns a text-only LLM into an agent that can act on the world — fetch fresh data, run code, and call external APIs — and then fold the results back into its reasoning.
The single most important idea on this page: the model never runs the tool. It only decides which tool to call and with what arguments, and emits that decision as structured data. Your application — the host runtime — actually executes the tool and hands the result back. The LLM is the brain; your code is the hands.
This separation is what makes tool calling both powerful and safe to govern: every action passes through code you control, where you can validate arguments, ask for permission, log, rate-limit, or refuse.
How the Mechanism Works
Tool calling is a four-part contract between your application and the model. Walk through it once and the rest of the page will click into place.
You provide tool schemas
For each tool you describe a name, a plain-English description (this is how the model decides when to use it), and a JSON-Schema of its parameters (types, which are required). These schemas go into the model's context alongside the user message.
The model emits a tool call
Instead of replying in prose, the model can choose to output a structured tool call: the function name plus a JSON object of arguments it filled in from the request. It is still just generating tokens — but in a shape your code can parse, not execute on its own.
Your runtime executes it
Your application parses the call, runs the real function (an API request, a DB query, a shell command...), and captures the result. This is the only step where anything actually happens in the world, and it is entirely your code.
The result goes back to the model
You append the tool's result to the conversation and call the model again. Now it can either emit another tool call (the loop continues) or write the final natural-language answer. This repetition is the agent loop.
Why "the model only emits the call" matters technically: the model has no execution sandbox, no network socket, no file system. Generating the text {"name":"get_weather","arguments":{"city":"Paris"}} does nothing by itself. A weather request only happens because your host runtime reads that JSON and chooses to make the HTTP call.
Interactive: The Function-Calling Loop
Pick a user request and step through the loop. The left panel shows the JSON tool schemas the model is given. The timeline on the right reveals one turn at a time so you can see the cycle: the model reasons, emits a structured JSON tool call, the runtime executes it and returns an observation, and finally the model writes a natural-language answer. Try the Parallel calls case (two calls in one turn) and the Multi-step case (a second call that depends on the first result).
Tools the model can call
{
"name": "get_weather",
"description": "Get the current weather for a city.",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "City name, e.g. \"Paris\""
}
},
"required": [
"city"
]
}
}{
"name": "calculator",
"description": "Evaluate a basic arithmetic expression and return the number.",
"parameters": {
"type": "object",
"properties": {
"expr": {
"type": "string",
"description": "Expression, e.g. \"23 * 19 + 7\""
}
},
"required": [
"expr"
]
}
}{
"name": "search",
"description": "Search a small offline knowledge base and return a short snippet.",
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "Free-text search query"
}
},
"required": [
"query"
]
}
}These JSON schemas are passed to the model. The model reads them but never runs the code behind them.
Click Step ▶ to advance the loop one turn at a time. Watch the colored phase tags cycle reason → act → observe → answer. The purple blocks are structured JSON the model emits; the others are natural-language or runtime turns.
Everything here is computed deterministically in the browser — the tools are mocked, there are no network calls. The "model" uses simple rule-based intent matching in place of a real LLM so the steps are reproducible, but the shape of the loop is exactly what production function calling looks like.
The Reason → Act → Observe Loop (ReAct)
The pattern you just stepped through is often called ReAct (Reasoning + Acting). The model interleaves a brief reasoning step ("I need live data, so I should call get_weather") with an action (emit the tool call), then observes the returned result before deciding the next move. The loop repeats until the model decides it has enough information to answer.
The Reason → Act → Observe segment loops as many times as needed before the model commits to a final Answer.
Termination: the loop ends when the model produces a turn with no tool call — a plain answer. In practice you also cap the number of iterations (a "max steps" budget) so a confused agent cannot loop forever.
Parallel Calls & Guaranteeing Valid JSON
Parallel tool calls
When several actions are independent of one another — for example, fetching the weather for Paris and London — the model can emit multiple tool calls in a single turn. Your runtime executes them together (often concurrently) and returns all the results at once. This cuts latency versus doing them one round-trip at a time. Crucially, calls can only be parallelized when one does not need another's output; a call that depends on a previous result must wait — that is a sequential, multi-step chain instead.
Structured outputs & constrained decoding
A tool call is only useful if the arguments are valid JSON matching the schema — a malformed call cannot be executed. Naively, a model might emit a trailing comma or the wrong type. Constrained decoding (the engine behind structured outputs) fixes this: at each generation step the decoder is restricted to only tokens that keep the output a valid continuation of the required JSON Schema. Tokens that would break the grammar are masked out before sampling.
The effect: the model is guaranteed to produce arguments that parse and conform to the schema (correct keys, correct types, required fields present). It does not guarantee the values are semantically right — the model can still pass the wrong city — only that the output is well-formed and executable.
Who Does What
The most common confusion about tool calling is the division of labor. This table makes it explicit.
| Step | The LLM (model) | The host runtime (your code) |
|---|---|---|
| Define tools | Reads the schemas, learns what is available | Writes the schemas, owns the real functions |
| Decide to call | Chooses the tool & fills in arguments (emits JSON) | Passes the call back; may validate / gate it |
| Execute | Does nothing — has no ability to run code | Runs the function, captures the result |
| Use result | Reads the observation, reasons, answers or calls again | Appends the result, re-invokes the model |
Error Handling, Security & Permissions
Because real tools touch real systems, robustness and safety are first-class concerns — not afterthoughts.
Error handling
- Bad arguments: validate the JSON against the schema before executing. If a required field is missing or a value is out of range, return a structured error as the tool result so the model can read it and retry.
- Tool failures: APIs time out and queries fail. Surface the error text back to the model — a good agent reads "rate limited, try again" and adapts, rather than hallucinating a fake answer.
- Loop budget: cap total iterations so a stuck agent halts instead of calling tools forever.
Security & permissions
The model can request dangerous actions. If you expose a tool like run_shell(cmd) or delete_file(path), the model can — through a mistake, an ambiguous instruction, or a malicious prompt injection hidden in tool output or a web page — emit a call to do real harm. Because the model emits but never executes, the host runtime is your one and only safety gate.
- Least privilege: only expose tools the task actually needs, scoped as narrowly as possible (read-only where you can).
- Human-in-the-loop confirmation: require explicit approval before executing irreversible or high-impact calls (payments, deletes, emails).
- Validate & sandbox: never blindly pass model output to a shell, SQL query, or file path. Whitelist, parameterize, and run risky tools in a sandbox.
- Beware prompt injection: treat content returned by tools (search results, fetched pages) as untrusted data, not as instructions to obey.
Key Takeaways
- Tool / function calling is what upgrades a text-only LLM into an agent that can act — fetching data, running code, and calling APIs.
- You supply tool schemas (name, description, JSON-Schema parameters); the model uses them to decide when and how to call.
- The model only emits a structured tool call (name + JSON arguments). It never executes the tool — your host runtime does, and returns the result.
- Feeding the result back and re-invoking the model creates the reason → act → observe agent loop, which ends when the model answers with no further call.
- Independent actions can be issued as parallel tool calls; dependent ones must be sequential (multi-step).
- Constrained decoding / structured outputs guarantee the arguments are valid, schema-conforming JSON — though not necessarily semantically correct.
- Since the runtime is the only thing that executes, it is also the only safety gate: validate arguments, apply least privilege, require confirmation for dangerous actions, and treat tool output as untrusted.