Glossary, Cheat Sheet and Learning Roadmap

Every term from the course in one place, a one-page cheat sheet, and a roadmap for what to learn and build next.

• Beginner
• 20 min read
• 3 objectives

Before this lessonLesson 19: Interview Special II: Agents, System Design and Live Scenarios

You made it to the last lesson. Here is everything in one place: a glossary you can search with Ctrl+F, a one-page cheat sheet of the numbers and rules of thumb worth memorising, and a practical roadmap for what to build next. Bookmark this page.

Glossary, A to Z

• Agent: an LLM that chooses its own next steps in a loop, using tools, until a goal is met.
• Agentic RAG: retrieval driven by an agent that decides when and what to search and can search again.
• ANN (approximate nearest neighbour): fast vector search that trades a little recall for big speedups (HNSW, IVF).
• API key: a secret credential identifying and billing you; keep it server-side.
• Attention: the transformer mechanism that lets each token weigh every other token in the context.
• Backoff: waiting longer between retries (usually exponentially, with jitter).
• Base model: a pre-trained model that only continues text and does not yet follow instructions.
• Bi-encoder: encodes query and document separately; fast, used for retrieval.
• BM25: a keyword ranking function using rarity (IDF), saturating term frequency and length normalisation.
• Chain of thought: prompting or training a model to reason step by step before answering.
• Checkpointing: saving agent state after each step so runs can pause, resume and replay.
• Chunking: splitting documents into pieces to embed and retrieve.
• Context engineering: deciding what the model sees on each turn.
• Context window: the maximum tokens a model can consider in one request.
• Cosine similarity: the cosine of the angle between two vectors; the standard text-similarity measure.
• Cross-encoder: reads query and document together to score relevance; slower and more accurate; used for reranking.
• Embedding: a vector representing the meaning of text (or other data).
• Eval: a repeatable test that scores AI outputs against expectations.
• Few-shot: putting examples of the task in the prompt.
• Fine-tuning: further training a model on curated examples to change its behaviour.
• Function calling / tool calling: the model requests that your code run a described function and gets the result back.
• Grounding: tying answers to supplied source documents.
• GraphRAG: RAG over a knowledge graph of entities and relations.
• Guardrails: checks and constraints on inputs, outputs and actions.
• Hallucination: a fluent but false or unsupported output.
• Harness: the code around a model (loop, tools, context, memory, permissions, limits) that makes it an agent.
• HNSW: a layered proximity-graph index for ANN search.
• HyDE: retrieving with a hypothetical answer generated by the LLM instead of the raw question.
• Hybrid search: combining keyword (BM25) and vector search.
• Inference: running a trained model to produce output.
• Inverted index: a map from each term to the documents containing it.
• JSON Schema: a standard way to describe the shape of JSON, used for tool parameters and structured output.
• KV cache: stored attention keys and values that make generation and prompt caching faster.
• Latency: delay; measured as time to first token and total time.
• LLM: large language model, a neural network trained to predict the next token.
• LLM-as-judge: using a model to grade another model's output.
• LoRA: a parameter-efficient fine-tuning method.
• MCP: Model Context Protocol, an open standard for connecting AI apps to tools and data.
• Memory: information stored outside the model and re-inserted into context.
• Metadata filtering: restricting search to chunks matching attributes such as tenant or date.
• MRR: mean reciprocal rank, a retrieval metric.
• Multimodal: handling more than one type of data (text, images, audio).
• nDCG: a retrieval metric that rewards relevant results ranked higher.
• Orchestrator-worker: a lead agent delegating subtasks to worker agents.
• Parameters: the learned numbers inside a model.
• Pre-training: the first, large-scale next-token training stage.
• Prompt: the input text (and messages) sent to a model.
• Prompt caching: discounted, faster processing of a repeated prompt prefix.
• Prompt injection: malicious instructions hidden in text the model reads.
• RAG: retrieval-augmented generation; fetch relevant documents, then generate from them.
• ReAct: reason, act, observe; the classic agent loop.
• Recall@k: the fraction of relevant items found in the top k results.
• Reranking: a second, more precise scoring pass over retrieval candidates.
• RLHF: reinforcement learning from human feedback; aligns model behaviour with preferences.
• RRF: reciprocal rank fusion; merges ranked lists using ranks only.
• Sampling: choosing the next token from the model's probabilities.
• Semantic cache: reusing answers for near-identical questions found by embedding similarity.
• SSE: server-sent events; one-way streaming from server to client over HTTP.
• Streaming: sending output as it is generated instead of all at once.
• Structured output: model output constrained to a schema.
• System prompt: standing instructions setting role and rules.
• Temperature: a knob controlling randomness in sampling.
• Token: the sub-word unit models read and write.
• Tool: a function the model can ask your code to run.
• Top-k / top-p: sampling filters that limit which tokens are considered.
• Transformer: the neural architecture behind modern LLMs.
• Vector: a list of numbers; a point in a high-dimensional space.
• Vector database: a store that indexes vectors for fast similarity search with metadata.
• WebSocket: a persistent two-way connection between client and server.
• Zero-shot: asking a model to do a task with no examples.

The one-page cheat sheet

Numbers to remember

• 1 token is about 4 English characters, or 0.75 words. A page of text is roughly 500 to 700 tokens.
• Output tokens usually cost 3 to 5 times input tokens.
• Chunk size: start at 200 to 800 tokens with 10 to 20 percent overlap.
• Retrieve 30 to 100 candidates, rerank, keep 3 to 8 for the prompt.
• RRF constant k is about 60. BM25 defaults: k1 between 1.2 and 2.0, b about 0.75.
• Typical embedding sizes: 384, 768, 1024, 1536, 3072.
• Per-step reliability compounds: 0.95 over 10 steps is 0.60.
• Golden eval set: 50 to 200 real questions, including unanswerable ones.

Rules of thumb

• Simplest thing that works: prompt, then workflow, then agent.
• Debug retrieval before the prompt; log the retrieved chunks with every answer.
• Hybrid search plus reranking beats either alone on messy data.
• RAG for knowledge, fine-tuning for behaviour.
• Temperature near 0 for facts, code and extraction.
• Validate every tool argument; enforce permissions in code, not in prompts.
• Stream tokens; use SSE for one-way, WebSockets or WebRTC for two-way and voice.
• Never give one agent private data, untrusted content and an outbound channel together.
• Measure first: evals, tracing, cost per request.

Which tool for which job?

• Exact terms, IDs, error codes: BM25 / keyword.
• Paraphrase and meaning: embeddings.
• Both: hybrid with RRF, then a cross-encoder rerank.
• One-way token streaming: SSE. Interactive, two-way, binary: WebSocket / WebRTC.
• Knowledge that changes: RAG. Style and format: fine-tune or prompt.
• Fixed steps: workflow. Unknown steps with verifiable results: agent.
• Branching, approvals, resumable runs: LangGraph-style graph.
• Messy multi-source documents: LlamaIndex-style ingestion, plus good parsing.
• Plugging tools into many apps: MCP.

A roadmap: what to build next

You learn this field by building, in order of increasing difficulty. Each project below reuses a chunk of this course:

• Project 1: A prompt playground (1 weekend). A small web page that calls a model API with a system prompt, streams the reply over SSE, and lets you change temperature. Teaches APIs, streaming and sampling.
• Project 2: Chat with a PDF (1 to 2 weeks). Parse, chunk, embed, store, retrieve, answer with citations. Add a refusal path. Teaches embeddings, chunking and RAG.
• Project 3: Make Project 2 good (2 weeks). Build a 50-question golden set, measure recall@5, add BM25 and RRF, add a reranker, and watch the numbers move. This is the most valuable project on the list, because it teaches evaluation.
• Project 4: A tool-using assistant (1 to 2 weeks). Give it 3 tools (search, calculator, one real API), add step limits, validation and a human approval gate. Teaches tool calling and harness design.
• Project 5: A LangGraph workflow with human approval (2 weeks). Something like a support-triage flow with a pause before refunds. Teaches state, checkpoints and interrupts.
• Project 6: Production hardening (ongoing). Add tracing, an eval suite in CI, caching, budgets, prompt-injection tests and cost dashboards.

How to keep learning

• Read primary sources: provider docs and changelogs, model cards, and a few key papers (Attention Is All You Need, RAG, ReAct, BM25 background).
• Rebuild from scratch whatever you use through a framework, at least once. That is how the ideas stick.
• Keep a personal eval set of your own hard questions and re-run it whenever a new model ships.
• Ship small things and get feedback. Real users find failures no benchmark will.
• Be sceptical: benchmark claims and demos are marketing until you test them on your data.

Where to go next on this site

• LangChain course for hands-on chains, retrieval and agents in code.
• FastAPI course to build the API and SSE layer behind an AI product.
• Python, SQL and System Design to strengthen the fundamentals every AI engineer leans on.
• Build a RAG Chatbot in Python for a long-form, production-oriented walkthrough.

# Write your solution here