The Agent Harness and Agent SDKs

The code around the model: loop, tools, context, memory, permissions, budgets. Build a mini harness and meet the SDKs.

• Advanced
• 32 min read
• 3 objectives

Before this lessonLesson 13: Agentic AI: From Chatbots to Agents

If you have used a coding assistant or a research agent, you have used a harness without knowing the word. The model is the brain; the harness is everything else: the loop, the tools, the memory, the guardrails, the budget, the logging. Two products using the identical model can feel completely different, and it is almost always the harness. This lesson opens one up and builds a small one.

What a harness is

flowchart TB subgraph H["Agent harness"] R[System prompt and rules] --> CB[Context builder] TR[Tool registry] --> MC[Model call] P[Permissions] --> TE[Tool executor] CB --> MC --> TE MEM[Memory / compaction] --> CB MC --> ST[Stream tokens] TE --> SB[Sandbox / hooks] TE --> OBS[Observations] OBS --> MEM end

The eight jobs of a harness

• 1. The loop. Call the model, run requested tools, feed results back, repeat until done or stopped.
• 2. Tools. Register them, describe them to the model, execute them, return errors gracefully.
• 3. Context management. Decide what the model sees each turn: instructions, history, retrieved files, tool outputs. Trim, summarise or drop the rest.
• 4. Memory. Short-term (the conversation), and long-term (notes, facts, preferences stored outside the model and re-inserted when relevant).
• 5. Permissions and safety. Which tools are allowed? Which need approval? What paths and commands are off limits?
• 6. Limits. Maximum steps, token and dollar budgets, timeouts, cancellation.
• 7. Streaming and UX. Show progress live (SSE/WebSocket), let users interrupt, display tool calls transparently.
• 8. Observability. Log every prompt, tool call and result; trace spans; enable replay for debugging and evals.

Build a mini harness

Here is a small but real harness in about seventy lines. It has a tool registry, a permission check, a step limit, a token budget, a transcript, and a stand-in model. Read it top to bottom; every production harness is this, plus more of each part.

import json

class Harness:
    def __init__(self, model, tools, max_steps=8, token_budget=400, needs_approval=()):
        self.model, self.tools = model, tools
        self.max_steps, self.token_budget = max_steps, token_budget
        self.needs_approval = set(needs_approval)
        self.transcript = []                       # everything that happened, for logging/replay

    def log(self, kind, data):
        self.transcript.append((kind, data))

    def estimate_tokens(self, messages):
        return sum(len(m["content"]) for m in messages) // 4       # crude 4-chars-per-token rule

    def run(self, goal, approve=lambda name, args: False):
        messages = [{"role": "user", "content": goal}]
        for step in range(1, self.max_steps + 1):
            if self.estimate_tokens(messages) > self.token_budget:
                self.log("stop", "token budget exceeded")
                return "stopped: budget"
            action = self.model(messages)                          # 1. ask the model
            if action["type"] == "final":
                self.log("final", action["content"])
                return action["content"]
            name, args = action["name"], action["args"]
            if name not in self.tools:                             # 2. unknown tool -> tell the model
                result = {"error": f"no such tool '{name}'"}
            elif name in self.needs_approval and not approve(name, args):
                result = {"error": "denied: a human must approve this action"}   # 3. permission gate
            else:
                try:
                    result = self.tools[name](**args)              # 4. execute
                except Exception as exc:                           # 5. errors go back to the model
                    result = {"error": str(exc)}
            self.log("tool", (step, name, args, result))
            messages.append({"role": "assistant", "content": f"call {name}"})
            messages.append({"role": "tool", "content": json.dumps(result)})
        return "stopped: step limit"

# ---------- tools ----------
NOTES = {"todo": "buy milk"}
def read_note(key): return {"value": NOTES.get(key, "(empty)")}
def write_note(key, value): NOTES[key] = value; return {"saved": key}
def delete_note(key): NOTES.pop(key, None); return {"deleted": key}

# ---------- a scripted model ----------
def model(messages):
    n = sum(1 for m in messages if m["role"] == "tool")
    plan = [("read_note", {"key": "todo"}), ("write_note", {"key": "todo", "value": "buy milk and eggs"}),
            ("delete_note", {"key": "todo"})]
    if n < len(plan):
        return {"type": "call", "name": plan[n][0], "args": plan[n][1]}
    return {"type": "final", "content": "Updated your todo note."}

h = Harness(model, {"read_note": read_note, "write_note": write_note, "delete_note": delete_note},
            needs_approval={"delete_note"})
print("RESULT:", h.run("Add eggs to my todo, then clean up."))
for kind, data in h.transcript:
    print(f"  {kind:<6} {data}")
print("notes now:", NOTES)

RESULT: Updated your todo note.
  tool   (1, 'read_note', {'key': 'todo'}, {'value': 'buy milk'})
  tool   (2, 'write_note', {'key': 'todo', 'value': 'buy milk and eggs'}, {'saved': 'todo'})
  tool   (3, 'delete_note', {'key': 'todo'}, {'error': 'denied: a human must approve this action'})
  final  Updated your todo note.
notes now: {'todo': 'buy milk and eggs'}

Look at the transcript: the harness let the two safe tools run, but blocked the destructive delete_note because no human approved it, and told the model so. The model never had the power to bypass the rule; the rule lives in code, not in a prompt. That is the essence of safe agent design.

Context engineering: the real craft

The model can only reason about what is in its context window, so deciding what goes in is the heart of agent quality. People call this context engineering. Techniques:

• Stable prefix first. Put the system prompt and tool definitions at the start, unchanged, so provider prompt caching gives you cheap, fast repeats.
• Observation trimming. A tool returned 40,000 tokens of HTML? Keep a summary or the relevant slice, not the whole thing.
• Compaction. When the transcript nears the limit, ask the model to summarise older turns and replace them with the summary.
• Just-in-time retrieval. Do not preload every file. Give the agent search and read tools and let it pull in what it needs.
• Scratchpad / notes. Let the agent write plans and findings to a file outside the context, and re-read them when needed.
• Sub-agents with clean context. Delegate a messy subtask to a fresh agent and return only a short result.

A tiny compaction demo: keep the system prompt, summarise the middle, keep the latest turns verbatim.

def compact(messages, keep_recent=2, summarise=lambda old: f"[summary of {len(old)} earlier messages]"):
    """Keep the first message and the latest few; replace the middle with a summary."""
    if len(messages) <= keep_recent + 1:
        return messages
    head, middle, tail = messages[:1], messages[1:-keep_recent], messages[-keep_recent:]
    return head + [{"role": "system", "content": summarise(middle)}] + tail

history = [{"role": "user", "content": "Goal: fix the login bug"}] + [
    {"role": "tool", "content": f"observation {i}: " + "x" * 200} for i in range(1, 7)]

tokens = lambda ms: sum(len(m["content"]) for m in ms) // 4
before = tokens(history)
history = compact(history)
print(f"messages: 7 -> {len(history)}")
print(f"tokens  : ~{before} -> ~{tokens(history)}")
for m in history:
    print(" ", m["role"], "|", m["content"][:50])

messages: 7 -> 4
tokens  : ~328 -> ~121
  user | Goal: fix the login bug
  system | [summary of 4 earlier messages]
  tool | observation 5: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
  tool | observation 6: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Memory: how an agent remembers

• Short-term: the conversation in context. Gone after the run unless saved.
• Long-term facts: notes saved to a file or database ("user prefers metric units") and loaded into the prompt in later sessions. Files like CLAUDE.md or AGENTS.md are exactly this: standing instructions the harness always loads.
• Episodic: records of past runs, retrievable by similarity ("last time this failed, the fix was...").
• Semantic / RAG: a knowledge base searched on demand.
• Procedural / skills: reusable instructions and scripts loaded only when a matching task appears.

Agent SDKs: harnesses you do not have to write

Writing a production harness is real work, so several vendors ship one as a library. You supply the goal, the tools and the rules; the SDK provides the loop, streaming, tracing, sessions and guardrails.

• Claude Agent SDK (Anthropic): the same harness that powers Claude Code, exposed as a library: built-in file/shell/search tools, permission modes, hooks, sub-agents, MCP servers, sessions.
• OpenAI Agents SDK: small primitives: Agents (model + instructions + tools), handoffs (delegate to another agent), guardrails (validate input/output), and built-in tracing.
• Google ADK, Microsoft Agent Framework / AutoGen, CrewAI, Pydantic AI, smolagents: other options with different opinions.
• LangGraph (next lesson): graph-based control flow for when you want explicit, inspectable state machines.

Here is the flavour of an SDK (illustrative; APIs evolve, so check the current docs). Notice how little code you write compared to the harness above:

# Illustrative Agents-SDK style code
from agents import Agent, Runner, function_tool

@function_tool
def get_order_status(order_id: str) -> str:
    """Look up the shipping status of an order."""
    return db.lookup(order_id)

support = Agent(
    name="Support",
    instructions="Help customers with orders. Be concise. Escalate refunds over $100 to a human.",
    tools=[get_order_status],
)

result = Runner.run_sync(support, "Where is order 4471?")
print(result.final_output)

# Illustrative Claude-Agent-SDK style code
from claude_agent_sdk import query, ClaudeAgentOptions

options = ClaudeAgentOptions(
    allowed_tools=["Read", "Grep", "Edit"],         # least privilege
    permission_mode="acceptEdits",                   # or ask for approval
    system_prompt="You are a careful code reviewer.",
)

async for message in query(prompt="Find and fix the failing test in tests/", options=options):
    print(message)

Build or buy?

• Use an SDK when your agent fits its model of the world: you get streaming, tracing, retries, sessions and best practices for free.
• Write your own thin loop when you need total control, or a very simple agent (like the one above), or unusual infrastructure. It is genuinely only a page of code.
• Either way, the hard parts are not the loop. They are tool design, context engineering, evaluation and safety.

A production readiness checklist

• Step, time and token/dollar limits, with a clear user-facing message when hit.
• Loop detection (same call repeatedly).
• Approval gates on destructive or external actions.
• Sandboxed execution for code and shell tools (containers, no network, read-only mounts).
• Structured logging and tracing of every model and tool call, with replay.
• An eval set of realistic tasks that runs on every change.
• Graceful degradation: a clear fallback when tools or the model fail.

# Write your solution here