LangChain · Lesson 10 of 15
Streaming and Callbacks
Stream tokens to a client and hook callbacks for logging and tokens used.
- Intermediate
- 14 min read
- 3 objectives
Before this lessonLesson 9: LCEL in Depth
What you will learn
- Stream a chain
- Count tokens
- Log each step
Your Progress
0 of 15 lessons 0%
- Lessons0 / 15
- Completed0
- Est. time left~ 4 hours
Create a free account to keep your progress on every device.
Waiting for a full completion feels slow. Stream tokens, and use callbacks to record what happened without wrapping every call.
Streaming tokens
for chunk in chain.stream({"question": "Explain embeddings"}):
print(chunk, end="", flush=True)In an API, write each chunk to an SSE response (see the AI course on streaming) and end with a done event.
Callbacks
from langchain_core.callbacks import BaseCallbackHandler
class TokenCounter(BaseCallbackHandler):
def __init__(self):
self.tokens = 0
def on_llm_end(self, response, **kwargs):
usage = response.llm_output.get("token_usage") or {}
self.tokens += usage.get("total_tokens", 0)
counter = TokenCounter()
chain.invoke({"question": "Hi"}, config={"callbacks": [counter]})
print(counter.tokens)