Low-Level Design: Classes, APIs and Concurrency

Turn a feature into entities, interfaces, a state machine and thread-safe code.

• Advanced
• 32 min read
• 3 objectives

Before this lessonLesson 5: Queues, APIs and Reliability

High-level design (HLD) answers "what boxes exist and how do they talk?" — load balancers, services, databases, queues, caches. Low-level design (LLD) answers "what does the code inside one box look like?" — classes, interfaces, method signatures, state machines, locks, error paths. Companies ask both, often as separate rounds: HLD for the architecture, LLD for "design a rate limiter / parking lot / elevator / Splitwise, in code."

They meet at the seams. HLD says "an idempotent POST /orders behind a queue"; LLD is the OrderService.place() method that takes an idempotency key, holds the right lock, writes the row and the outbox event in one transaction, and returns the same response on a retry.

The six-step LLD process

1. Clarify and bound. Same as HLD: what is in v1, what is out. Single process or distributed? One machine or many? Who calls this — a UI, another service, a library consumer?
2. Find the nouns (entities). Read the requirements and underline every noun. Those are your candidate classes. Then delete the ones that are really just attributes.
3. Find the verbs (behaviours), and give each an owner. Each verb becomes a method on the class that owns the data it touches. If a verb needs data from three classes, it probably belongs in a service/coordinator class.
4. Draw the class diagram. Relationships and cardinality. Prefer composition ("an Order has line items") over inheritance ("an Order is a ...").
5. Define interfaces at the variation points. Anywhere the requirements say "for now" or "for example", put an interface: payment methods, notification channels, limiting algorithms, pricing rules.
6. Then the hard parts. State machine for legal transitions, concurrency (what two threads can touch at once), error handling, and how you would unit-test it.

SOLID, in one line each, with the LLD payoff

• S — Single responsibility. One class, one reason to change. Payoff: Order holds order data; it does not also format emails or talk to Stripe.
• O — Open/closed. Open to extension, closed to modification. Payoff: a new payment method is a new class, not a new elif in a 300-line function.
• L — Liskov substitution. A subtype must work anywhere its parent works. Payoff: if ReadOnlyAccount.withdraw() throws, it should not be a subclass of Account.
• I — Interface segregation. Many small interfaces beat one fat one. Payoff: a read-only consumer does not implement six write methods it will never call.
• D — Dependency inversion. Depend on abstractions, and inject them. Payoff: your test passes a FakeClock and an InMemoryStore and runs in a millisecond with no Redis.

Also say DRY (do not duplicate knowledge), YAGNI (do not build the plugin system nobody asked for) and composition over inheritance — deep class hierarchies are the most common LLD mistake.

The patterns that actually come up

• Strategy — swap an algorithm at runtime behind one interface. Rate limiting algorithms, pricing rules, ranking, eviction policies. The single most useful LLD pattern.
• Factory — one place that decides which concrete class to build from config or input. Channel by type, parser by file format.
• State — each state is an object that knows its own legal transitions. Order lifecycle, ride lifecycle, elevator.
• Observer / pub-sub — publishers do not know their subscribers. Order placed → notify customer, restaurant, analytics.
• Decorator — wrap an object to add behaviour without touching it. Retry, caching, metrics, auth around a repository.
• Repository — hide persistence behind an interface so the domain never mentions SQL. The seam that makes unit tests possible.
• Builder — construct an object with many optional fields readably.
• Command — an action as an object, so it can be queued, logged, retried or undone.
• Singleton — mention it, then say why you would rather inject one instance: global mutable state makes tests order-dependent.

Worked LLD: a rate limiter

This is the classic LLD prompt, and it is a good one because the algorithm is explicitly pluggable, the storage is explicitly swappable (memory now, Redis later), and it has a real concurrency question.

Step 1 — Requirements

Functional: given a key (user, IP, API token) and a rule (N requests per window), allow or deny; report how many remain and when to retry. Multiple algorithms. Different rules per endpoint. Non-functional: the check adds under 1 ms, is thread-safe, and works across many app servers when backed by Redis. Out of scope for v1: the admin UI, per-tenant billing.

Step 2 — Entities and interfaces

classDiagram class RateLimiter { -RuleResolver rules -StrategyFactory factory +check(key, route) Decision } class Decision { +bool allowed +int remaining +float retry_after } class Rule { +string name +int limit +int window_seconds +string algorithm } class LimitStrategy { <<interface>> +allow(key, rule, now) Decision } class TokenBucket class SlidingWindowLog class FixedWindowCounter class Storage { <<interface>> +get(key) +set(key, value, ttl) +incr(key, ttl) } class MemoryStorage class RedisStorage class Clock { <<interface>> +now() float } RateLimiter --> LimitStrategy RateLimiter --> Rule RateLimiter --> Decision LimitStrategy <|.. TokenBucket LimitStrategy <|.. SlidingWindowLog LimitStrategy <|.. FixedWindowCounter LimitStrategy --> Storage LimitStrategy --> Clock Storage <|.. MemoryStorage Storage <|.. RedisStorage

Three interfaces mark the three things the requirements said would change: the algorithm, the storage, and (for tests) the clock.

Step 3 — Code

from abc import ABC, abstractmethod
from dataclasses import dataclass
import threading, time

# ---------- value objects: immutable, no behaviour beyond validation ----------

@dataclass(frozen=True)
class Rule:
    limit: int              # requests allowed
    window_seconds: float   # per this window
    algorithm: str = "token_bucket"

    def __post_init__(self):
        if self.limit <= 0 or self.window_seconds <= 0:
            raise ValueError("limit and window must be positive")

    @property
    def refill_per_second(self) -> float:
        return self.limit / self.window_seconds

@dataclass(frozen=True)
class Decision:
    allowed: bool
    remaining: int
    retry_after: float      # seconds the client should wait; 0 when allowed

# ---------- injected dependencies (dependency inversion) ----------

class Clock(ABC):
    @abstractmethod
    def now(self) -> float: ...

class SystemClock(Clock):
    def now(self) -> float:
        return time.monotonic()      # monotonic: never jumps backwards on NTP sync

class FakeClock(Clock):              # exists purely so tests control time
    def __init__(self, t=0.0): self.t = t
    def now(self) -> float: return self.t
    def advance(self, dt): self.t += dt

# ---------- strategy: the algorithm is the thing that varies ----------

class LimitStrategy(ABC):
    @abstractmethod
    def allow(self, key: str, rule: Rule) -> Decision: ...

class TokenBucket(LimitStrategy):
    """Bucket holds `limit` tokens, refills continuously, allows bursts."""

    def __init__(self, clock: Clock):
        self._clock = clock
        self._state: dict[str, tuple[float, float]] = {}   # key -> (tokens, last_seen)
        self._lock = threading.Lock()                      # guards _state

    def allow(self, key: str, rule: Rule) -> Decision:
        now = self._clock.now()
        # One critical section covering read-modify-write: without it, two threads
        # both read 1 token, both decrement, and the limit is silently doubled.
        with self._lock:
            tokens, last = self._state.get(key, (float(rule.limit), now))
            tokens = min(rule.limit, tokens + (now - last) * rule.refill_per_second)
            if tokens >= 1.0:
                self._state[key] = (tokens - 1.0, now)
                return Decision(True, int(tokens - 1.0), 0.0)
            self._state[key] = (tokens, now)
            wait = (1.0 - tokens) / rule.refill_per_second
            return Decision(False, 0, round(wait, 3))

class FixedWindowCounter(LimitStrategy):
    """Cheapest option. Allows up to 2x the limit across a window boundary."""

    def __init__(self, clock: Clock):
        self._clock = clock
        self._counts: dict[tuple[str, int], int] = {}
        self._lock = threading.Lock()

    def allow(self, key: str, rule: Rule) -> Decision:
        now = self._clock.now()
        window = int(now // rule.window_seconds)
        with self._lock:
            used = self._counts.get((key, window), 0)
            if used < rule.limit:
                self._counts[(key, window)] = used + 1
                return Decision(True, rule.limit - used - 1, 0.0)
        reset_at = (window + 1) * rule.window_seconds
        return Decision(False, 0, round(reset_at - now, 3))

# ---------- the façade the rest of the app calls ----------

class RateLimiter:
    def __init__(self, rules: dict[str, Rule], clock: Clock | None = None):
        clock = clock or SystemClock()
        self._rules = rules
        self._default = Rule(limit=100, window_seconds=60)
        # Factory: one place maps an algorithm name to a class.
        self._strategies: dict[str, LimitStrategy] = {
            "token_bucket": TokenBucket(clock),
            "fixed_window": FixedWindowCounter(clock),
        }

    def check(self, key: str, route: str) -> Decision:
        rule = self._rules.get(route, self._default)
        strategy = self._strategies[rule.algorithm]
        return strategy.allow(f"{route}:{key}", rule)   # namespaced: routes never collide


limiter = RateLimiter(
    rules={
        "POST /login": Rule(limit=5, window_seconds=60, algorithm="fixed_window"),
        "POST /urls":  Rule(limit=3, window_seconds=10, algorithm="token_bucket"),
    },
    clock=FakeClock(1000.0),
)

for i in range(5):
    d = limiter.check("user:42", "POST /urls")
    print(i, "allow" if d.allowed else "deny", "remaining=", d.remaining, "retry_after=", d.retry_after)

0 allow remaining= 2 retry_after= 0.0
1 allow remaining= 1 retry_after= 0.0
2 allow remaining= 0 retry_after= 0.0
3 deny remaining= 0 retry_after= 3.333
4 deny remaining= 0 retry_after= 3.333

Step 4 — What to say about this code

• Why an interface for the algorithm: the requirement "different algorithms per endpoint" is a variation point, so it is a Strategy. Adding a sliding-window log is a new class and one factory entry — nothing existing changes (open/closed).
• Why Rule and Decision are frozen dataclasses: immutable value objects cannot be mutated by a caller and are safe to share between threads.
• Why Clock is injected: otherwise testing a 60-second window takes 60 seconds. With FakeClock you assert refill behaviour deterministically.
• Where the lock is, and why: the check is a read-modify-write, so it must be atomic. Lock per key (a striped lock or a lock map) rather than one global lock, or the limiter becomes the bottleneck it was meant to prevent.
• The distributed version: swap the in-memory dict for Redis and make the whole read-modify-write atomic server-side with a Lua script (or INCR plus EXPIRE for fixed window). The class structure does not change — that is the point of the Storage interface. Lesson 9 covers the algorithms and the Redis details.

State machines: the other half of LLD

Most real LLD questions contain a lifecycle — an order, a ride, a ticket, a document. Draw the legal transitions and put them in one table, then make every transition go through one method. This is how you stop a cancelled order from being delivered.

stateDiagram-v2 [*] --> CREATED CREATED --> PAID: payment captured CREATED --> CANCELLED: user cancels / payment fails PAID --> ACCEPTED: restaurant accepts PAID --> REFUNDED: restaurant rejects ACCEPTED --> PREPARING PREPARING --> READY READY --> PICKED_UP: courier assigned and collects PICKED_UP --> DELIVERED DELIVERED --> [*] CANCELLED --> [*] REFUNDED --> [*]

Terminal states have no outgoing edges. Anything not drawn here is an error, not a special case.

from enum import Enum

class State(str, Enum):
    CREATED = "CREATED";   PAID = "PAID";         ACCEPTED = "ACCEPTED"
    PREPARING = "PREPARING"; READY = "READY";     PICKED_UP = "PICKED_UP"
    DELIVERED = "DELIVERED"; CANCELLED = "CANCELLED"; REFUNDED = "REFUNDED"

# The single source of truth. New rule? Edit this table, not ten if-statements.
ALLOWED = {
    State.CREATED:   {State.PAID, State.CANCELLED},
    State.PAID:      {State.ACCEPTED, State.REFUNDED},
    State.ACCEPTED:  {State.PREPARING, State.REFUNDED},
    State.PREPARING: {State.READY},
    State.READY:     {State.PICKED_UP},
    State.PICKED_UP: {State.DELIVERED},
    State.DELIVERED: set(), State.CANCELLED: set(), State.REFUNDED: set(),
}

class IllegalTransition(Exception):
    pass

class Order:
    def __init__(self, order_id: str):
        self.id = order_id
        self.state = State.CREATED
        self.history: list[tuple[State, State]] = []

    def transition_to(self, target: State) -> None:
        if target not in ALLOWED[self.state]:
            raise IllegalTransition(f"{self.id}: {self.state} -> {target} is not allowed")
        self.history.append((self.state, target))
        self.state = target

o = Order("ord_1")
for s in (State.PAID, State.ACCEPTED, State.PREPARING, State.READY):
    o.transition_to(s)
print(o.state, len(o.history), "transitions")
try:
    o.transition_to(State.CANCELLED)          # too late to cancel: food is cooked
except IllegalTransition as e:
    print("rejected:", e)

State.READY 4 transitions
rejected: ord_1: State.READY -> State.CANCELLED is not allowed

In a distributed system, add one more thing: persist the transition with a conditional write — UPDATE orders SET state='PAID' WHERE id=? AND state='CREATED'. If it updates zero rows, someone else already moved it, and you must not proceed. That single line is compare-and-swap, and it is how you make a state machine safe when two servers process the same webhook.

Concurrency in LLD

• Name the shared mutable state. Every concurrency bug lives there. If nothing is shared and mutable, you have no problem.
• Race condition: two threads interleave a read-modify-write and one update is lost. Fix with a lock, an atomic operation (INCR, compareAndSet), or a conditional UPDATE ... WHERE version = ?.
• Keep critical sections tiny. Never hold a lock across a network call — that converts a slow dependency into a full stall.
• Deadlock: two threads each hold what the other needs. Prevent it by always acquiring locks in a fixed global order, and using timeouts on lock acquisition.
• Lock granularity. One global lock is correct and slow; per-key or striped locks scale. Say which you chose and why.
• Prefer immutability and queues to shared mutable state. A single-threaded consumer reading a queue needs no locks at all.
• Idempotency is the distributed version of a lock. You cannot lock across services reliably, so make the operation safe to repeat instead.

Designing the API layer (the HLD/LLD seam)

Whatever the round, you will write endpoints. Checklist:

• Resources, not verbs. POST /orders, not POST /createOrder. Sub-resources for relationships: GET /orders/{id}/events.
• Version it — /api/v1/… — from day one.
• Status codes that mean something. 200 ok, 201 created, 202 accepted (async work started), 204 no content, 400 malformed, 401 unauthenticated, 403 unauthorised, 404 missing, 409 conflict, 422 semantically invalid, 429 rate limited, 500 our bug, 503 try later.
• Idempotency on unsafe writes. Idempotency-Key: <uuid> header; store key → response.
• Cursor pagination, not OFFSET: ?limit=20&cursor=eyJ0…. Offset gets slower as it grows and skips or repeats rows when data shifts.
• Structured errors: {"error": {"code": "ALIAS_TAKEN", "message": "...", "request_id": "..."}} — a machine-readable code plus a trace id.
• Long operations return 202 plus a job id and a polling endpoint, rather than holding a connection open.
• Never trust the client: validate at the boundary, and never let the client set price, role or user id.

How to run an LLD round

1. Restate the problem and list v1 features (2 min).
2. List entities and their key fields out loud (3 min).
3. Name the variation points and declare the interfaces (3 min).
4. Draw the class diagram (5 min).
5. Write the one or two methods that carry the real logic — in full, compiling-quality code (10 min).
6. Discuss concurrency, error handling and how you would test it (5 min).
7. Offer one extension: "if we needed this across servers, here is the change."

# Write your solution here