LLM Apps and LangChain Basics

How LLM apps work, and calling a chat model through LangChain.

A large language model (LLM) predicts text. Given a conversation, it produces a helpful continuation. On its own that is a chat box; wrapped in code that gives it data, tools and memory, it becomes an application. LangChain is a Python (and JavaScript) library that standardizes those building blocks so you can swap models and compose steps easily.

Core vocabulary

• Messages: a conversation is a list of messages with roles: system (instructions), human (the user) and ai (the model).
• Tokens: models read and write chunks of text called tokens (roughly three quarters of a word). Pricing and limits are measured in tokens.
• Context window: the maximum tokens the model can consider at once (prompt plus answer).
• Temperature: randomness. 0 gives focused, repeatable answers; higher values are more creative.

python3 -m venv .venv && source .venv/bin/activate
pip install langchain langchain-openai
export OPENAI_API_KEY="sk-..."        # never hard-code or commit keys

from langchain_openai import ChatOpenAI
from langchain_core.messages import SystemMessage, HumanMessage

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

reply = llm.invoke([
    SystemMessage(content="You are a concise assistant."),
    HumanMessage(content="Explain a database index in one sentence."),
])
print(reply.content)

A database index is a data structure that lets the database find rows quickly without scanning the whole table.

LangChain provides a common interface: invoke for one call, stream for token-by-token output and batch for many inputs. To use another provider, change only the import and class, for example ChatAnthropic from langchain_anthropic.

for chunk in llm.stream("Write a haiku about Python"):
    print(chunk.content, end="", flush=True)

Streaming makes chat apps feel fast: users start reading immediately instead of waiting for the full answer.

Costs and safety basics

• Log token usage (reply.usage_metadata) from day one.
• Set a maximum output length and timeouts.
• Treat model output as untrusted text: never run it as code or SQL without validation.

reply = llm.invoke([
    SystemMessage(content="Translate to Spanish. Reply only with the translation."),
    HumanMessage(content="Where is the train station?"),
])
print(reply.content)
print(reply.usage_metadata)