Learn / AI / AI (Artificial Intelligence) / Tool Calling and Structured Output

Tool Calling and Structured Output

How a model calls functions: schemas, the request/response loop, validation, parallel calls and error handling.

  • Intermediate
  • 26 min read
  • 3 objectives

Before this lessonLesson 11: Evaluating and Improving RAG

What you will learn

  • Explain the tool-calling loop
  • Write a tool schema
  • Validate and handle tool errors safely

Your Progress

0 of 20 lessons 0%

  • Lessons0 / 20
  • Completed0
  • Est. time left~ 9 hours

Create a free account to keep your progress on every device.

A plain LLM can only produce text. It cannot check the weather, query your database, send an email or add two large numbers reliably. Tool calling (also called function calling) is the trick that fixes this, and it is the foundation of every agent you will meet. The idea is delightfully simple: the model does not run anything. It asks you to run something, and you tell it what happened.

The loop, in plain words

sequenceDiagram participant You as Your app participant M as Model participant T as get_weather You->>M: Tools plus user question about Paris M-->>You: tool request get_weather city Paris You->>T: run get_weather T-->>You: temp 14, rain 80 percent You->>M: tool result M-->>You: Yes, bring an umbrella

The model never touches the outside world. Your code is always in the middle.

That last sentence is your security model. Because your code executes every tool call, your code decides whether to allow it.

Describing a tool: the schema

Each tool is described to the model with a name, a plain-English description (the model reads this to decide when to use it, so write it well), and a JSON Schema for its parameters:

{
  "name": "get_weather",
  "description": "Get the current weather for a city. Use when the user asks about weather or what to wear.",
  "input_schema": {
    "type": "object",
    "properties": {
      "city":  {"type": "string", "description": "City name, e.g. Paris"},
      "units": {"type": "string", "enum": ["celsius", "fahrenheit"]}
    },
    "required": ["city"]
  }
}
  • Good descriptions matter more than anything. "Search the company knowledge base for policy questions. Do NOT use for general knowledge." beats "search".
  • Use enums so the model cannot invent values.
  • Keep tools small and single-purpose. Ten focused tools beat one do_anything(command) tool.
  • Return helpful errors. "City not found. Try a larger nearby city" lets the model recover.

A full tool-calling loop you can run

We replace the real model with a tiny scripted stand-in so you can watch the message flow with no API key. The shape of the loop is identical to a real one:

import json

# ---------- your tools: ordinary Python functions ----------
def get_weather(city, units="celsius"):
    data = {"paris": (14, 80), "cairo": (33, 0), "oslo": (2, 40)}
    if city.lower() not in data:
        return {"error": f"unknown city '{city}'. Try Paris, Cairo or Oslo."}
    temp, rain = data[city.lower()]
    return {"city": city, "temp": temp if units == "celsius" else round(temp * 9 / 5 + 32), "rain_chance": rain}

def add(a, b):
    return {"result": a + b}

TOOLS = {"get_weather": get_weather, "add": add}

# ---------- a stand-in for the model ----------
def fake_model(messages):
    """Decides what to do next. A real LLM does this from its training."""
    last = messages[-1]
    if last["role"] == "user":
        text = last["content"].lower()
        if "umbrella" in text or "weather" in text:
            city = next((c for c in ["Paris", "Cairo", "Oslo", "Rome"] if c.lower() in text), "Paris")
            return {"type": "tool_call", "name": "get_weather", "args": {"city": city}}
        return {"type": "text", "content": "I can help with weather questions."}
    if last["role"] == "tool":
        result = json.loads(last["content"])
        if "error" in result:
            return {"type": "text", "content": f"Sorry: {result['error']}"}
        need = "Yes, bring an umbrella." if result["rain_chance"] > 50 else "No umbrella needed."
        return {"type": "text", "content": f"{need} It is {result['temp']} degrees, and the chance of rain is {result['rain_chance']}%."}

# ---------- the loop: this is the part YOU write ----------
def run(user_message):
    messages = [{"role": "user", "content": user_message}]
    for step in range(5):                                     # hard cap: never loop forever
        reply = fake_model(messages)
        if reply["type"] == "text":
            return reply["content"]
        print(f"  model asks -> {reply['name']}({reply['args']})")
        result = TOOLS[reply["name"]](**reply["args"])        # YOUR code executes it
        print(f"  tool says  -> {result}")
        messages.append({"role": "tool", "content": json.dumps(result)})
    return "gave up after too many steps"

for q in ["Do I need an umbrella in Paris?", "What is the weather in Rome?"]:
    print("USER:", q)
    print("BOT :", run(q), "\n")
Output
USER: Do I need an umbrella in Paris?
  model asks -> get_weather({'city': 'Paris'})
  tool says  -> {'city': 'Paris', 'temp': 14, 'rain_chance': 80}
BOT : Yes, bring an umbrella. It is 14 degrees, and the chance of rain is 80%. 

USER: What is the weather in Rome?
  model asks -> get_weather({'city': 'Rome'})
  tool says  -> {'error': "unknown city 'Rome'. Try Paris, Cairo or Oslo."}
BOT : Sorry: unknown city 'Rome'. Try Paris, Cairo or Oslo. 

The second question shows error handling in action: the tool returned a helpful error, the model relayed it gracefully instead of crashing or inventing weather for Rome.

What the real API messages look like

Providers differ in field names, but the structure is the same everywhere. Here is the flow using generic names:

// 1) You send the conversation + tool definitions. The model answers with a tool request:
{"role": "assistant", "tool_calls": [
  {"id": "call_1", "name": "get_weather", "arguments": {"city": "Paris"}}
]}

// 2) You run it and append the result, linking it by id:
{"role": "tool", "tool_call_id": "call_1", "content": "{\"temp\": 14, \"rain_chance\": 80}"}

// 3) You call the model again. This time it answers in text.

Validate everything the model sends

The model produces arguments as text, and text can be wrong: a missing field, a wrong type, an out-of-range number, or something deliberately malicious. Treat tool arguments like untrusted user input. Validate against the schema before running:

def validate(args, schema):
    """A tiny JSON-Schema-style validator (real code: use jsonschema or pydantic)."""
    problems = []
    for name in schema.get("required", []):
        if name not in args:
            problems.append(f"missing required field '{name}'")
    for name, value in args.items():
        spec = schema["properties"].get(name)
        if spec is None:
            problems.append(f"unexpected field '{name}'")
            continue
        types = {"string": str, "integer": int, "number": (int, float)}
        if not isinstance(value, types[spec["type"]]):
            problems.append(f"'{name}' should be {spec['type']}")
        if "enum" in spec and value not in spec["enum"]:
            problems.append(f"'{name}' must be one of {spec['enum']}")
        if "maximum" in spec and isinstance(value, (int, float)) and value > spec["maximum"]:
            problems.append(f"'{name}' must be <= {spec['maximum']}")
    return problems

schema = {
    "properties": {"amount": {"type": "number", "maximum": 100}, "currency": {"type": "string", "enum": ["USD", "EUR"]}},
    "required": ["amount", "currency"],
}

for args in [{"amount": 25, "currency": "USD"},
             {"amount": 5000, "currency": "USD"},
             {"amount": "ten", "currency": "GBP"},
             {"currency": "EUR", "note": "hi"}]:
    print(args, "->", validate(args, schema) or "OK")
Output
{'amount': 25, 'currency': 'USD'} -> OK
{'amount': 5000, 'currency': 'USD'} -> ["'amount' must be <= 100"]
{'amount': 'ten', 'currency': 'GBP'} -> ["'amount' should be number", "'currency' must be one of ['USD', 'EUR']"]
{'currency': 'EUR', 'note': 'hi'} -> ["missing required field 'amount'", "unexpected field 'note'"]

When validation fails, do not crash. Send the problems back to the model as the tool result ("amount must be <= 100") and let it try again. Models are good at repairing their own calls when you tell them what was wrong.

Parallel and multiple tool calls

A model can ask for several tools in one turn ("weather in Paris and Oslo"). Run independent calls concurrently to save time, append every result with the matching id, then call the model once more. Order and dependencies matter: if the second call needs the first call's output, the model will request them across separate turns.

Tools versus structured output

  • Tool calling: the model wants something done (search, write a file, call an API). The result flows back to the model.
  • Structured output: you just want the model's answer in a fixed shape (extract fields from an email into JSON). Nothing is executed. Most APIs let you supply a JSON Schema and guarantee a match. In practice, structured output is often implemented as a "tool" the model must call.

Safety rules for tools

  • Least privilege. Give the read-only tool, not the read-write one, unless you truly need writes.
  • Confirm dangerous actions. Deleting, sending money, sending email to customers: require a human click.
  • Scope credentials. The database user your tool uses should only reach the tables it needs.
  • Never build SQL or shell commands by pasting model text into a string. Use parameterised queries and allow-lists.
  • Rate-limit and budget tool calls per request so a confused model cannot loop 10,000 times.
  • Log every call with arguments and results for auditing and debugging.

Where tools are heading: MCP

Writing every integration by hand for every app does not scale. The Model Context Protocol (MCP) is an open standard that lets any AI app plug into any tool server (files, GitHub, databases, Slack, browsers) using one common protocol, like USB for AI tools. We build a toy MCP exchange in a later lesson.

# Write your solution here
Up next · Lesson 13Agentic AI: From Chatbots to AgentsWhat an agent really is, the ReAct loop with a runnable demo, workflows vs agents, patterns and failure modes.