AI: One Course · Lesson 12 of 20
Tool Calling and Structured Output
How a model calls functions: schemas, the request/response loop, validation, parallel calls and error handling.
- Intermediate
- 26 min read
- 3 objectives
Before this lessonLesson 11: Evaluating and Improving RAG
What you will learn
- Explain the tool-calling loop
- Write a tool schema
- Validate and handle tool errors safely
Your Progress
0 of 20 lessons 0%
- Lessons0 / 20
- Completed0
- Est. time left~ 9 hours
Create a free account to keep your progress on every device.
A plain LLM can only produce text. It cannot check the weather, query your database, send an email or add two large numbers reliably. Tool calling (also called function calling) is the trick that fixes this, and it is the foundation of every agent you will meet. The idea is delightfully simple: the model does not run anything. It asks you to run something, and you tell it what happened.
The loop, in plain words
1. You tell the model what tools exist (name, description, parameters) -> "you may call get_weather(city)"
2. User: "Do I need an umbrella in Paris?"
3. Model replies NOT with text but with a tool request: get_weather({"city": "Paris"})
4. YOUR code runs get_weather("Paris") for real -> {"temp": 14, "rain_chance": 80}
5. You send that result back to the model as a 'tool' message
6. Model writes the final answer: "Yes, bring one. 80% chance of rain, around 14 degrees."
The model never touches the outside world. Your code is always in the middle.That last sentence is your security model. Because your code executes every tool call, your code decides whether to allow it.
Describing a tool: the schema
Each tool is described to the model with a name, a plain-English description (the model reads this to decide when to use it, so write it well), and a JSON Schema for its parameters:
{
"name": "get_weather",
"description": "Get the current weather for a city. Use when the user asks about weather or what to wear.",
"input_schema": {
"type": "object",
"properties": {
"city": {"type": "string", "description": "City name, e.g. Paris"},
"units": {"type": "string", "enum": ["celsius", "fahrenheit"]}
},
"required": ["city"]
}
}- Good descriptions matter more than anything. "Search the company knowledge base for policy questions. Do NOT use for general knowledge." beats "search".
- Use enums so the model cannot invent values.
- Keep tools small and single-purpose. Ten focused tools beat one
do_anything(command)tool. - Return helpful errors. "City not found. Try a larger nearby city" lets the model recover.
A full tool-calling loop you can run
We replace the real model with a tiny scripted stand-in so you can watch the message flow with no API key. The shape of the loop is identical to a real one:
import json
# ---------- your tools: ordinary Python functions ----------
def get_weather(city, units="celsius"):
data = {"paris": (14, 80), "cairo": (33, 0), "oslo": (2, 40)}
if city.lower() not in data:
return {"error": f"unknown city '{city}'. Try Paris, Cairo or Oslo."}
temp, rain = data[city.lower()]
return {"city": city, "temp": temp if units == "celsius" else round(temp * 9 / 5 + 32), "rain_chance": rain}
def add(a, b):
return {"result": a + b}
TOOLS = {"get_weather": get_weather, "add": add}
# ---------- a stand-in for the model ----------
def fake_model(messages):
"""Decides what to do next. A real LLM does this from its training."""
last = messages[-1]
if last["role"] == "user":
text = last["content"].lower()
if "umbrella" in text or "weather" in text:
city = next((c for c in ["Paris", "Cairo", "Oslo", "Rome"] if c.lower() in text), "Paris")
return {"type": "tool_call", "name": "get_weather", "args": {"city": city}}
return {"type": "text", "content": "I can help with weather questions."}
if last["role"] == "tool":
result = json.loads(last["content"])
if "error" in result:
return {"type": "text", "content": f"Sorry: {result['error']}"}
need = "Yes, bring an umbrella." if result["rain_chance"] > 50 else "No umbrella needed."
return {"type": "text", "content": f"{need} It is {result['temp']} degrees, and the chance of rain is {result['rain_chance']}%."}
# ---------- the loop: this is the part YOU write ----------
def run(user_message):
messages = [{"role": "user", "content": user_message}]
for step in range(5): # hard cap: never loop forever
reply = fake_model(messages)
if reply["type"] == "text":
return reply["content"]
print(f" model asks -> {reply['name']}({reply['args']})")
result = TOOLS[reply["name"]](**reply["args"]) # YOUR code executes it
print(f" tool says -> {result}")
messages.append({"role": "tool", "content": json.dumps(result)})
return "gave up after too many steps"
for q in ["Do I need an umbrella in Paris?", "What is the weather in Rome?"]:
print("USER:", q)
print("BOT :", run(q), "\n")USER: Do I need an umbrella in Paris?
model asks -> get_weather({'city': 'Paris'})
tool says -> {'city': 'Paris', 'temp': 14, 'rain_chance': 80}
BOT : Yes, bring an umbrella. It is 14 degrees, and the chance of rain is 80%.
USER: What is the weather in Rome?
model asks -> get_weather({'city': 'Rome'})
tool says -> {'error': "unknown city 'Rome'. Try Paris, Cairo or Oslo."}
BOT : Sorry: unknown city 'Rome'. Try Paris, Cairo or Oslo. The second question shows error handling in action: the tool returned a helpful error, the model relayed it gracefully instead of crashing or inventing weather for Rome.
What the real API messages look like
Providers differ in field names, but the structure is the same everywhere. Here is the flow using generic names:
// 1) You send the conversation + tool definitions. The model answers with a tool request:
{"role": "assistant", "tool_calls": [
{"id": "call_1", "name": "get_weather", "arguments": {"city": "Paris"}}
]}
// 2) You run it and append the result, linking it by id:
{"role": "tool", "tool_call_id": "call_1", "content": "{\"temp\": 14, \"rain_chance\": 80}"}
// 3) You call the model again. This time it answers in text.Validate everything the model sends
The model produces arguments as text, and text can be wrong: a missing field, a wrong type, an out-of-range number, or something deliberately malicious. Treat tool arguments like untrusted user input. Validate against the schema before running:
def validate(args, schema):
"""A tiny JSON-Schema-style validator (real code: use jsonschema or pydantic)."""
problems = []
for name in schema.get("required", []):
if name not in args:
problems.append(f"missing required field '{name}'")
for name, value in args.items():
spec = schema["properties"].get(name)
if spec is None:
problems.append(f"unexpected field '{name}'")
continue
types = {"string": str, "integer": int, "number": (int, float)}
if not isinstance(value, types[spec["type"]]):
problems.append(f"'{name}' should be {spec['type']}")
if "enum" in spec and value not in spec["enum"]:
problems.append(f"'{name}' must be one of {spec['enum']}")
if "maximum" in spec and isinstance(value, (int, float)) and value > spec["maximum"]:
problems.append(f"'{name}' must be <= {spec['maximum']}")
return problems
schema = {
"properties": {"amount": {"type": "number", "maximum": 100}, "currency": {"type": "string", "enum": ["USD", "EUR"]}},
"required": ["amount", "currency"],
}
for args in [{"amount": 25, "currency": "USD"},
{"amount": 5000, "currency": "USD"},
{"amount": "ten", "currency": "GBP"},
{"currency": "EUR", "note": "hi"}]:
print(args, "->", validate(args, schema) or "OK"){'amount': 25, 'currency': 'USD'} -> OK
{'amount': 5000, 'currency': 'USD'} -> ["'amount' must be <= 100"]
{'amount': 'ten', 'currency': 'GBP'} -> ["'amount' should be number", "'currency' must be one of ['USD', 'EUR']"]
{'currency': 'EUR', 'note': 'hi'} -> ["missing required field 'amount'", "unexpected field 'note'"]When validation fails, do not crash. Send the problems back to the model as the tool result ("amount must be <= 100") and let it try again. Models are good at repairing their own calls when you tell them what was wrong.
Parallel and multiple tool calls
A model can ask for several tools in one turn ("weather in Paris and Oslo"). Run independent calls concurrently to save time, append every result with the matching id, then call the model once more. Order and dependencies matter: if the second call needs the first call's output, the model will request them across separate turns.
Tools versus structured output
- Tool calling: the model wants something done (search, write a file, call an API). The result flows back to the model.
- Structured output: you just want the model's answer in a fixed shape (extract fields from an email into JSON). Nothing is executed. Most APIs let you supply a JSON Schema and guarantee a match. In practice, structured output is often implemented as a "tool" the model must call.
Safety rules for tools
- Least privilege. Give the read-only tool, not the read-write one, unless you truly need writes.
- Confirm dangerous actions. Deleting, sending money, sending email to customers: require a human click.
- Scope credentials. The database user your tool uses should only reach the tables it needs.
- Never build SQL or shell commands by pasting model text into a string. Use parameterised queries and allow-lists.
- Rate-limit and budget tool calls per request so a confused model cannot loop 10,000 times.
- Log every call with arguments and results for auditing and debugging.
Where tools are heading: MCP
Writing every integration by hand for every app does not scale. The Model Context Protocol (MCP) is an open standard that lets any AI app plug into any tool server (files, GitHub, databases, Slack, browsers) using one common protocol, like USB for AI tools. We build a toy MCP exchange in a later lesson.
# Write your solution here
