System Design · Lesson 1 of 18
System Design Fundamentals
Requirements, latency, throughput, CAP and a framework for every design interview.
- Intermediate
- 45 min read
- 6 objectives
What you will learn
- Run a 45-minute interview with the 7-step framework
- Design a small system end to end in ten minutes
- Quantify scalability, reliability, availability and efficiency
- Estimate QPS, storage and bandwidth without the classic traps
- Apply CAP and PACELC per feature
- Match architecture to read-heavy or write-heavy load
Your Progress
0 of 18 lessons 0%
- Lessons0 / 18
- Completed0
- Est. time left~ 16 hours
Progress is saved in this browser. No account needed.
Tip: pressing Next marks this lesson complete automatically.
System design is the work of deciding which pieces a software system is made of — clients, load balancers, stateless services, caches, databases, queues, object stores, CDNs — how they talk to each other, and what each one promises when something goes wrong. It is not about memorising the architecture of a famous company. It is about turning a vague product ask into a set of numbers, and turning those numbers into choices you can defend.
This lesson is the foundation for the whole course. It covers three things, each with a worked example:
- The interview itself — what is being graded, and a seven-step framework with a minute-by-minute budget for a 45-minute round, including what to actually say at each step.
- A complete dry run — a Pastebin-style service designed end to end in ten minutes, so you can see the framework produce a real answer rather than an outline.
- The vocabulary of trade-offs — scalability, reliability, availability, efficiency and manageability with real numbers; the latency table every designer should carry around; estimation with runnable code and the traps that sink candidates; CAP and PACELC applied to real databases; and how read-heavy and write-heavy workloads push you toward different architectures.
What a system design interview actually evaluates
A coding round has a correct output. A design round does not, which is exactly why candidates who prepare only by reading architectures of well-known products tend to do badly: they recite a design instead of reasoning towards one. The interviewer is scoring a handful of signals, and all of them are about how you get to a design.
| Signal | What a strong candidate does | What a weak candidate does |
|---|---|---|
| Scoping | Asks 4–6 sharp questions, writes down the answers, names what is out of scope | Starts drawing boxes in minute one |
| Quantitative reasoning | Produces QPS, storage and bandwidth, then uses them to make a decision | Computes numbers and never refers to them again |
| Technical soundness | Each component exists for a stated reason; data flows are complete end to end | Adds Kafka, Redis and a service mesh because "that is what big companies use" |
| Trade-offs | Names at least two options for each hard choice and says why one wins here | Presents one option as obviously correct |
| Depth | Can go two levels down on the part the interviewer cares about (schema, algorithm, failure handling) | Stays at box level for 45 minutes |
| Operational maturity | Talks about failure modes, monitoring, deploys and what breaks at 10x | Assumes every component is always up |
| Communication | Narrates, checks in, lets the interviewer steer the deep-dive | Goes silent for five minutes, or refuses to change direction |
Expectations rise with level. For a mid-level role, a coherent end-to-end design with sensible components is usually enough. For senior, the interviewer expects you to drive the conversation, surface trade-offs unprompted and handle one deep-dive well. For staff and above, they are looking for judgement: which problem is actually hard, what you would not build, how the design evolves over two years, and how it fails.
The seven-step framework, with a 45-minute budget
Every design in this course follows the same seven steps. The order matters: each step produces something the next step consumes. Requirements produce the numbers; numbers pick the storage; the API and data model constrain the architecture; the architecture tells you where to dig.
5 min"] --> I["2. API
4 min"] I --> E["3. Estimation
4 min"] E --> D["4. Data model
5 min"] D --> H["5. High-level design
9 min"] H --> X["6. Deep dive
13 min"] X --> B["7. Bottlenecks and trade-offs
5 min"] E -. numbers pick storage .-> D H -. interviewer picks .-> X
About 45 minutes in total. The numbers from step 3 feed every later step; after step 5, hand the steering wheel to the interviewer.
| Step | Minutes | You produce | Done when |
|---|---|---|---|
| 1. Requirements | 0–5 | Functional list, out-of-scope list, non-functional targets | The interviewer agrees with the written list |
| 2. API / interface | 5–9 | 3–6 endpoints with inputs, outputs, status codes | Every functional requirement maps to an endpoint |
| 3. Estimation | 9–13 | Read and write QPS (avg and peak), storage, bandwidth, cache size | You have said what the numbers imply |
| 4. Data model | 13–18 | Entities, keys, indexes, choice of store and why | Every endpoint's main query has an index |
| 5. High-level design | 18–27 | 5–8 boxes, arrows labelled with what flows | You can trace one read and one write end to end |
| 6. Deep dive | 27–40 | Detail on the 1–2 hardest parts | The interviewer moves on or time runs out |
| 7. Bottlenecks and trade-offs | 40–45 | Single points of failure, what breaks at 10x, a trade-off summary | You have closed the loop on the requirements |
Step 1 — Clarify the requirements (about 5 minutes)
The prompt is intentionally vague. "Design Twitter" could mean the timeline, search, trends, DMs, ads or the media pipeline — each a different system. Your first job is to shrink it to something designable in 40 minutes, and to get the interviewer to agree.
- Functional: what the user can do. Write them as verbs: post a tweet, follow a user, view home timeline.
- Out of scope: say it out loud. "I will skip DMs, ads and search unless you want them." This is not laziness; it is showing you know the real product is larger.
- Non-functional: scale (users, items), latency targets (p99, not average), availability target, consistency needs, durability, geography, and any compliance constraint.
- The shape of the traffic: read/write ratio, spikiness, and whether some users are much bigger than others (celebrities, viral items, large tenants).
What to say: "Before I draw anything, let me pin down scope. Users post short text, maybe with images; they follow others; they see a home timeline. I will assume 200M daily users, reads dominate writes by about 100 to 1, a timeline should load in under 300 ms at p99, and it is fine if a new post takes a few seconds to appear for followers. I will leave out search and DMs. Does that match what you had in mind?"
Common wrong answer: asking twenty questions. You want the five that change the design. "Should the button be blue?" does not; "is it acceptable for a follower to see a post five seconds late?" decides your whole consistency model.
Step 2 — Define the API (about 4 minutes)
The API is the contract, and writing it down is the fastest way to discover a misunderstanding. Keep it to the core endpoints and include what matters: pagination style, idempotency on writes, and error codes.
POST /v1/tweets { text, media_ids[] } -> 201 { tweet_id, created_at }
Idempotency-Key: <uuid> # the app retries on flaky networks; do not post twice
GET /v1/timeline?cursor=<opaque>&limit=50 -> 200 { tweets[], next_cursor }
POST /v1/users/{id}/follow -> 204 | 404 | 409 ALREADY_FOLLOWING
GET /v1/users/{id}/tweets?cursor= -> 200 { tweets[], next_cursor }What to say: "I will use cursor pagination rather than offsets because the timeline changes while you scroll; offsets would show duplicates. Writes take an idempotency key." The user id comes from the auth token, not a parameter — a detail interviewers notice.
Step 3 — Back-of-the-envelope estimation (about 4 minutes)
You are not trying to be precise. You are trying to find out which order of magnitude you are in, because that decides whether one Postgres box is enough or whether you need to shard, whether content fits in RAM, and whether bandwidth is a real constraint. Produce four numbers — peak read QPS, peak write QPS, storage over the retention period, and peak bandwidth — then say what each one implies. The estimation section later in this lesson shows the techniques in detail.
What to say: "200M daily users at about 5 timeline loads each is a billion reads a day, so roughly 12k per second, call it 35k at peak. That is too many for one database but trivially cacheable. Writes are about 100x lower, a few hundred per second, which a single primary handles. So the read path is my scaling problem, not the write path." The last sentence is the whole point of the step.
Step 4 — Data model (about 5 minutes)
List the entities, their keys and the access pattern for each. Then — and only then — pick the store. The choice should follow from the query shapes and the numbers, not from habit.
- Entities: User(id, handle, created_at), Tweet(id, author_id, text, created_at), Follow(follower_id, followee_id, created_at).
- Access patterns: "tweets by author, newest first" means an index on
(author_id, created_at DESC); "who does X follow" and "who follows X" means both directions of the follow table are indexed. - Store: relational for users and follows (small, relational, transactional), a wide-column or sharded store for tweets if the volume demands it, object storage for media — never media blobs in the database.
What to say: "Tweet ids should be time-sortable, like Snowflake ids, so I can paginate by id and shard without a central sequence."
Step 5 — High-level design (about 9 minutes)
Draw 5–8 boxes that satisfy every functional requirement end to end: client, load balancer, stateless services, a cache, the database, object storage and a CDN, and a queue if work can be deferred. Label arrows with what flows over them. Then trace one write and one read out loud through the diagram — this catches missing boxes immediately.
What to say at the end: "That is the end-to-end skeleton. The two parts I think are hardest are timeline generation for users who follow celebrities, and the tweet storage sharding. Which would you like me to go deeper on?" Handing over the steering wheel here is a strong senior signal: you have shown breadth, and now the interviewer chooses where to test depth.
Step 6 — Detailed design (about 13 minutes)
Go two levels deeper on one or two components. For each, present at least two approaches, the trade-off between them, and a decision tied back to a requirement or a number. Typical deep dives: the partitioning key and what happens with hot keys, the fan-out strategy, the cache layer and its invalidation, the ID generator, the consistency of one specific operation.
What to say: "There are two ways to build the timeline: fan-out on write, where each post is pushed into every follower's precomputed list, and fan-out on read, where we merge at query time. Write fan-out makes reads a single cache lookup but a user with 50M followers turns one post into 50M writes. So I will do a hybrid: push for normal users, pull for accounts above about 100k followers, and merge the two at read time."
Step 7 — Bottlenecks, failure modes and trade-offs (about 5 minutes)
Walk the diagram looking for trouble. Is there a single point of failure? What happens when the cache cluster dies — does the database survive the full read load? What is the replication story, and what is lost if a region goes? How is the system monitored, and what alert fires first? Then close with a short trade-off summary: "I chose X over Y because of Z; the cost is W; I would revisit it when metric M crosses N."
Worked example — Pastebin-lite in ten minutes
Here is the framework run at speed, the way you would do it if an interviewer said "let's warm up with something small". Pastebin: a user pastes text, gets a short URL, and anyone with the URL can read it. Watch how every step produces a concrete artefact and how the numbers drive the decisions.
Minute 0–2: requirements
- Functional: create a paste (text up to 1 MB) and get a short URL; read a paste by URL; optional expiry (10 min, 1 day, 1 month, never); optional custom alias.
- Out of scope: accounts, editing, syntax highlighting (a client concern), search, analytics beyond a view counter.
- Non-functional: 1M new pastes a day, reads about 10x writes, read p99 under 200 ms, 99.9% availability, a paste must never be lost once the URL is returned, URLs must not be guessable in sequence.
Minute 2–3: API
POST /v1/pastes { content, expires_in?: "1d", alias?: "my-notes" }
-> 201 { key: "aZ3kP9q", url, expires_at }
| 409 ALIAS_TAKEN | 413 TOO_LARGE | 429 RATE_LIMITED
GET /v1/pastes/{key} -> 200 { content, created_at, expires_at }
| 404 NOT_FOUND | 410 EXPIRED
DELETE (none in v1: pastes expire; abuse takedown is an admin tool)Minute 3–5: estimation
# Pastebin-lite: the whole estimate in under two minutes of talking.
SECONDS_PER_DAY = 86_400
new_pastes_per_day = 1_000_000
read_write_ratio = 10 # stated assumption: each paste read ~10 times
avg_paste_kb = 10 # most pastes are small logs / snippets
retention_years = 5
peak_factor = 3
writes = new_pastes_per_day / SECONDS_PER_DAY
reads = writes * read_write_ratio
print(f"writes: {writes:.0f}/s avg, {writes * peak_factor:.0f}/s peak")
print(f"reads : {reads:.0f}/s avg, {reads * peak_factor:.0f}/s peak")
storage_tb = new_pastes_per_day * 365 * retention_years * avg_paste_kb / 1e9
print(f"storage over {retention_years} years: {storage_tb:.1f} TB (x3 replicas = {storage_tb * 3:.0f} TB)")
egress_mb_s = reads * peak_factor * avg_paste_kb / 1000
print(f"peak read bandwidth: {egress_mb_s:.1f} MB/s = {egress_mb_s * 8:.0f} Mbit/s")
# Cache the hot 20% of one day's reads (the 80/20 rule is a starting guess, not a law)
cache_gb = new_pastes_per_day * read_write_ratio * 0.2 * avg_paste_kb / 1e6
print(f"cache for hot 20% of daily reads: {cache_gb:.0f} GB -> fits one Redis node")
keys = new_pastes_per_day * 365 * retention_years
print(f"keys needed: {keys / 1e9:.2f} billion -> base62^7 = {62**7 / 1e12:.1f} trillion is plenty")writes: 12/s avg, 35/s peak reads : 116/s avg, 347/s peak storage over 5 years: 18.2 TB (x3 replicas = 55 TB) peak read bandwidth: 3.5 MB/s = 28 Mbit/s cache for hot 20% of daily reads: 20 GB -> fits one Redis node keys needed: 1.82 billion -> base62^7 = 3.5 trillion is plenty
Now say what the numbers mean — this is the part candidates skip. 35 writes/s and 350 reads/s are small: a single well-indexed relational primary with one replica handles this with room to spare, so no sharding in v1. 18 TB over five years is not small for a database, but it is trivial for object storage. 20 GB of hot data fits in one cache node. 28 Mbit/s is not a constraint. So the design is "small metadata database, content in object storage, a cache in front" — and the interesting problem is key generation, not scale.
Minute 5–6: data model
-- Metadata only: ~200 bytes a row, so 1.8B rows is ~400 GB. Fits one Postgres box for years.
CREATE TABLE pastes (
key VARCHAR(16) PRIMARY KEY, -- 7-char base62, or a custom alias
object_path TEXT NOT NULL, -- where the content lives in object storage
size_bytes INT NOT NULL,
created_at TIMESTAMPTZ NOT NULL,
expires_at TIMESTAMPTZ NULL -- NULL = never
);
CREATE INDEX ON pastes (expires_at) WHERE expires_at IS NOT NULL; -- for the cleanup job
-- Content: object storage at pastes/{key}. Up to 1 MB each, written once, never updated.Why split content from metadata? Content is large, immutable and read by key — the exact workload object storage is built for, at a fraction of the cost per GB. Keeping 10 KB–1 MB blobs in the relational database bloats backups, replication and the buffer cache for no benefit.
Minute 6–8: high-level design
stateless, 3+ instances"] API --> KG["Key range allocator
hands out blocks of 1,000 keys"] API --> RC[("Redis
hot pastes, 20 GB")] API --> DB[("Postgres
metadata: primary + replica")] API --> OS[("Object storage
content")] JOB[Expiry sweeper] --> DB JOB --> OS
Write: get a key, upload content to object storage, then insert the metadata row, then return the URL. Read: Redis first; on a miss, metadata from the replica and content from object storage, then fill Redis.
Trace the write: the service takes the next key from its locally held block → uploads content to pastes/{key} → inserts the metadata row → returns 201. The order matters: if the upload succeeds but the insert fails, you have an orphan object (harmless, a sweeper removes it); if you inserted first and the upload failed, you would have a URL pointing at nothing. Trace the read: Redis GET paste:{key} → on a miss, read metadata from the replica, check expires_at (return 410 if past), fetch content, fill Redis with a TTL no longer than the remaining lifetime.
Minute 8–9: deep dive on key generation
- Hash the content (MD5, take 7 chars): identical pastes collide, and you need collision handling anyway. Rejected.
- Random 7-char base62 and retry on conflict: 3.5 trillion keys, 1.8 billion used, so a collision is about a 1-in-2,000 event; a unique constraint plus one retry is fine. Simple and good.
- Counter with a range allocator: each service instance leases a block of 1,000 ids from a tiny coordinator and encodes each id in base62. No collisions, no retries. But sequential ids are guessable, which violates a requirement — so scramble the id with a reversible permutation before encoding.
Decision: random keys with a unique constraint for v1 — fewest moving parts, satisfies the non-guessable requirement. The range allocator is what you would reach for if write volume grew 100x. (Lesson 7, Unique ID Generation, covers this in full.)
Minute 9–10: bottlenecks and trade-offs
- Single points of failure: the Postgres primary (mitigated by a synchronous standby and automated failover, which also gives the "never lost" guarantee), the load balancer (use a managed, redundant one).
- Viral paste: one key at thousands of reads per second. Redis absorbs it; put the read endpoint behind a CDN with
Cache-Control: public, max-age=60for pastes that never expire. - Abuse: rate limit writes per IP (see Rate Limiter), cap size at 1 MB.
- Expiry: lazy (check on read, return 410) plus a nightly sweeper that deletes rows and objects in batches using the partial index.
- Trade-off summary: "I chose one relational primary over a distributed store because 35 writes/s does not justify the operational cost; content in object storage because it is 18 TB of immutable blobs; random keys because they satisfy non-guessability with no coordination. I would revisit the database choice above roughly 5,000 writes/s."
That is a complete, defensible design in ten minutes. The full URL-shortener case study in lesson 8 takes the same problem much further.
The five properties every design is judged on
Every requirement you collect in step 1 eventually maps to one of five properties. Knowing them by name, with a number attached to each, is what lets you turn "it should be fast and not go down" into something you can design against.
1. Scalability — handling more work by adding resources
A system is scalable if you can serve more load — more requests, more data, more users in more places — by adding resources, without a redesign and without the per-request cost climbing. There are three directions of growth, and they are solved differently:
- Load scalability (more requests per second): add stateless servers behind a load balancer, add caches and read replicas.
- Data scalability (more bytes or rows): partition the data across machines — sharding, covered in lesson 4.
- Geographic scalability (users on more continents): CDNs, regional deployments, data residency.
Vertical scaling (a bigger box: more cores, RAM, faster disks) is simple, keeps everything on one machine so transactions and joins stay easy, and is often the right first move — a modern server with 64 cores and 512 GB of RAM carries more than most startups ever need. Its limits are a hard ceiling, a price curve that turns steep at the top end, a single point of failure, and usually a restart to resize. Horizontal scaling (more boxes) has no practical ceiling and survives the loss of any one machine, but it requires the work to be splittable: stateless services, partitioned data, and coordination only where unavoidable. Databases that were built to spread across nodes (Cassandra, DynamoDB, CockroachDB) scale out by design; a classic single-primary relational database scales up first and out with replicas and sharding later.
Worked example — why "just add servers" has a ceiling. Any part of a request that has to happen in one place — a write to a single primary, a global lock, a leader — does not get faster when you add machines. Amdahl's law quantifies it:
# Why "just add servers" stops working: the serial part caps the speed-up.
# Amdahl: speedup(N) = 1 / (s + (1 - s) / N), where s = fraction that cannot be spread out
# (a single primary DB write, a global lock, a leader election...).
def amdahl(serial_fraction, n):
return 1 / (serial_fraction + (1 - serial_fraction) / n)
print(f"{'servers':>7} | {'s=1%':>7} {'s=5%':>7} {'s=20%':>7}")
for n in (1, 4, 16, 64, 256):
row = " ".join(f"{amdahl(s, n):>6.1f}x" for s in (0.01, 0.05, 0.20))
print(f"{n:>7} | {row}")
print(f"ceiling as N -> infinity: {1/0.01:.0f}x, {1/0.05:.0f}x, {1/0.20:.0f}x")servers | s=1% s=5% s=20%
1 | 1.0x 1.0x 1.0x
4 | 3.9x 3.5x 2.5x
16 | 13.9x 9.1x 4.0x
64 | 39.3x 15.4x 4.7x
256 | 72.1x 18.6x 4.9x
ceiling as N -> infinity: 100x, 20x, 5x
With only 5% of the work serialised, 256 servers give you less than a 19x speed-up, and no number of servers ever gets you past 20x. That is why good scalable designs spend their effort removing the serial part — partitioning the primary, replacing global locks with per-key ones, avoiding cross-shard transactions — rather than adding more of the parallel part. In an interview, the question "what is the one thing every request has to touch?" finds this bottleneck fast.
2. Reliability — continuing to work correctly despite faults
A reliable system keeps delivering correct results when parts of it fail: a disk dies, a node reboots, a network link drops packets, a deploy has a bug. Faults are not rare events at scale — with 10,000 disks and an annual failure rate of 1–2%, you replace several disks every week. Reliability is achieved by redundancy of both compute and data (replicas, multiple instances, multiple availability zones) plus detection and recovery (health checks, retries with backoff, idempotent operations, checksums).
Worked example. A shopping cart lives on one server. The server dies. An unreliable design loses the cart. A reliable design stores the cart in a replicated store (three copies across three zones), so any surviving replica serves it and the user never notices. The cost is explicit: three times the storage and a replication protocol to keep copies in step.
3. Availability — the fraction of time the system is usable
Availability is the percentage of time (or of requests) that the system responds successfully. It is usually stated in "nines", and it is worth knowing what each target actually permits:
# How much downtime does each "nines" target actually allow?
MIN_PER_YEAR = 365.25 * 24 * 60
def fmt(minutes):
if minutes >= 60 * 24: return f"{minutes / 1440:.1f} days"
if minutes >= 60: return f"{minutes / 60:.1f} h"
if minutes >= 1: return f"{minutes:.1f} min"
return f"{minutes * 60:.0f} s"
print(f"{'target':<10}{'per year':>12}{'per month':>12}{'per week':>11}")
for label, a in (("99%", 0.99), ("99.9%", 0.999), ("99.95%", 0.9995),
("99.99%", 0.9999), ("99.999%", 0.99999)):
down = (1 - a) * MIN_PER_YEAR
print(f"{label:<10}{fmt(down):>12}{fmt(down / 12):>12}{fmt(down / 52.18):>11}")
# Components in SERIES multiply: every hop must be up for a request to succeed.
chain = {"LB": 0.9999, "app": 0.9995, "cache": 0.9999, "db": 0.9995}
serial = 1.0
for a in chain.values():
serial *= a
print(f"\nserial chain of {len(chain)}: {serial:.5f} -> {fmt((1 - serial) * MIN_PER_YEAR)}/year")
# Redundant copies in PARALLEL: down only if ALL copies are down at once.
def parallel(a, n):
return 1 - (1 - a) ** n
for n in (1, 2, 3):
p = parallel(0.99, n)
print(f"{n} x 99% copies in parallel: {p:.6f} -> {fmt((1 - p) * MIN_PER_YEAR)}/year")target per year per month per week 99% 3.7 days 7.3 h 1.7 h 99.9% 8.8 h 43.8 min 10.1 min 99.95% 4.4 h 21.9 min 5.0 min 99.99% 52.6 min 4.4 min 1.0 min 99.999% 5.3 min 26 s 6 s serial chain of 4: 0.99880 -> 10.5 h/year 1 x 99% copies in parallel: 0.990000 -> 3.7 days/year 2 x 99% copies in parallel: 0.999900 -> 52.6 min/year 3 x 99% copies in parallel: 0.999999 -> 32 s/year
Three lessons hide in that output. First, each extra nine is ten times harder: 99.9% allows a long evening of downtime a year, 99.99% allows under an hour, 99.999% allows about five minutes — less than a single slow manual rollback. Second, components in series multiply: four hops that are each 99.95–99.99% give you 99.88% overall, worse than any single part. Every dependency you add to the request path lowers your ceiling. Third, redundancy in parallel is how you climb: two independent 99% copies give 99.99%, because both must fail at once. The catch is the word independent — two replicas on the same rack, same power feed or the same buggy config push fail together, which is why you spread them across zones and roll out changes gradually.
Reliability versus availability. A reliable system is available, but an available system is not necessarily reliable. A service that always answers but occasionally returns another customer's data is 100% available and deeply unreliable. Conversely you can reach high availability with mediocre hardware by making repair fast. Availability is MTBF / (MTBF + MTTR) — mean time between failures over total time — and the lever you usually control is MTTR, the time to repair:
# Tail latency amplification: a page that fans out to N backends is as slow as the slowest.
# Each backend is fast 99% of the time and slow (>= 1 s) 1% of the time.
p_slow = 0.01
for n in (1, 10, 50, 100):
p_page_slow = 1 - (1 - p_slow) ** n
print(f"fan-out to {n:>3} backends: {p_page_slow:>5.1%} of page loads hit a slow call")
# Availability from MTBF / MTTR: the lever you usually control is repair time.
mtbf_h = 30 * 24 # the box fails about once a month
for mttr_min in (240, 30, 2): # manual rebuild, on-call restart, automatic failover
a = mtbf_h / (mtbf_h + mttr_min / 60)
print(f"MTTR {mttr_min:>3} min -> availability {a:.4%}")fan-out to 1 backends: 1.0% of page loads hit a slow call fan-out to 10 backends: 9.6% of page loads hit a slow call fan-out to 50 backends: 39.5% of page loads hit a slow call fan-out to 100 backends: 63.4% of page loads hit a slow call MTTR 240 min -> availability 99.4475% MTTR 30 min -> availability 99.9306% MTTR 2 min -> availability 99.9954%
The same machine, failing once a month, goes from two nines to four and a half nines purely by making recovery automatic. That is the argument for health checks, automatic failover and fast rollbacks over buying more expensive hardware. The first half of the output is about efficiency, covered next.
4. Efficiency — latency and throughput
Efficiency has two measures. Latency (response time) is how long one operation takes; throughput is how many operations complete per unit time. They are related but not the same: a cargo ship has terrible latency and enormous throughput. In distributed systems the cost of an operation is dominated by two things — how many messages cross the network (each one pays a round trip) and how large they are (each byte pays bandwidth). A design that makes one request fan out into 50 sequential calls is inefficient however fast each call is.
Always talk about latency as percentiles. The average hides the users who suffer: p50 is the typical request, p99 is the slowest 1 in 100, and at a million requests a minute, p99 is ten thousand unhappy users every minute. Tail latency also amplifies with fan-out — the first half of the output above shows that a page calling 100 backends, each slow only 1% of the time, is slow 63% of the time. Mitigations: fewer, larger calls; parallel rather than sequential fan-out; timeouts with hedged requests (send a second copy after the p95 deadline and take whichever answers first); and caching.
5. Manageability — how easy it is to operate, diagnose and change
Also called serviceability or operability. A system that is hard to understand is hard to repair, and slow repair is lower availability — MTTR again. Manageability is about:
- Observability: metrics (rates, errors, latency percentiles, saturation), structured logs, and distributed traces that follow one request across services.
- Safe change: small deploys, canaries, feature flags, automatic rollback on error-rate regressions, schema migrations that are backward compatible.
- Simplicity: fewer kinds of component. Every new database or queue technology is another thing to patch, back up, monitor and be paged for at 3 a.m.
- Automation: self-healing (replace unhealthy instances), autoscaling, and alerts on user-visible symptoms rather than on every internal cause.
Worked example. Two designs for the same service: design A uses Postgres, Redis and one stateless service; design B adds Kafka, Cassandra, Elasticsearch and a graph database for a marginal latency win. At 100 QPS, B is not faster in any way a user notices, but it has four times the on-call surface. Choosing A and saying why is a manageability argument, and a senior-level one.
Latency numbers every designer should know
You cannot reason about a design without a feel for how long things take. Exact figures vary by hardware and year; what matters is the order of magnitude and the ratios between rows. Memorise the shape: nanoseconds for memory, microseconds for SSD and in-process work, a fraction of a millisecond inside a datacenter, milliseconds for spinning disks and cross-zone hops, tens to hundreds of milliseconds across continents.
| Operation | Approximate time | What it tells you |
|---|---|---|
| L1 cache reference | 1 ns | The floor for any computation |
| Branch mispredict | 3 ns | |
| L2 cache reference | 4 ns | |
| Mutex lock / unlock (uncontended) | 20 ns | Contended locks are far worse — microseconds |
| Main memory reference | 100 ns | 100x L1: data layout matters |
| Compress 1 KB (fast codec) | 2 µs | Compression is usually cheaper than sending the bytes |
| Read 1 MB sequentially from memory | 5–10 µs | |
| Random 4 KB read from NVMe SSD | 20–100 µs | ~1,000x slower than RAM |
| Read 1 MB sequentially from SSD | ~200 µs | |
| Round trip within one datacenter / zone | 0.2–0.5 ms | A cache hit over the network costs about this |
| Round trip across zones in one region | 1–2 ms | Synchronous replication across zones is affordable |
| Indexed point query on a relational DB (incl. network) | 1–5 ms | |
| Read 1 MB sequentially from spinning disk | ~5 ms | |
| Disk seek | ~10 ms | Why random I/O on HDD is avoided |
| Round trip, US east ↔ west coast | 60–70 ms | |
| Round trip, US ↔ Europe | 80–100 ms | Synchronous cross-region writes add this to every commit |
| Round trip, US ↔ India / Australia | 180–250 ms | Why global products need regional deployments and CDNs |
| New TLS connection | 1–2 extra round trips | Why connection reuse and edge termination matter |
And a few throughput rules of thumb, again as orders of magnitude: a stateless app server handles roughly 1,000–10,000 simple requests/s; a single Redis node 100,000+ operations/s; a single relational primary a few thousand to ~10,000 simple writes/s and many more reads; a 1 Gbit/s link carries 125 MB/s; a day has 86,400 seconds — round it to 100,000 for mental maths, and 1 million per day becomes about 12 per second.
Worked example — reading the table. A page needs 20 user profiles. Twenty sequential cache lookups at 0.5 ms each is 10 ms; one batched MGET is about 0.5 ms. Twenty sequential cross-region calls at 80 ms is 1.6 seconds — which is why you never put a cross-region hop inside a loop. The table turns "should I batch this?" from an opinion into arithmetic.
Estimation techniques and the traps that sink candidates
Back-of-the-envelope estimation is a skill, and like any skill it has a method:
- State assumptions out loud and write them down — daily users, actions per user, object sizes, read/write ratio, retention. The interviewer can correct an assumption; they cannot correct a number that came from nowhere.
- Convert per-day to per-second by dividing by 86,400 (or 100,000 for a quick mental version).
- Apply a peak factor: 2–3x for a global product with smooth traffic, 5–10x for anything tied to meal times, match days, sales or market open.
- Multiply by size for storage and bandwidth, and by retention for total storage.
- Add the overheads: replication (usually 3x), indexes and metadata (20–50%), and headroom (keep disks under ~70% full).
- Round aggressively and keep one significant figure — 347 reads/s is "about 350", 18.2 TB is "about 20 TB".
- Say what it means: "this fits on one box", "this needs sharding", "this must be cached", "bandwidth is not a concern".
The code below shows four traps side by side with the corrected number, plus the powers of two you use when sizing storage:
# Four estimation traps, each shown as the wrong number next to the right one.
# Trap 1: forgetting peak. Averages hide the lunch/evening spike.
daily = 200_000_000
avg = daily / 86_400
print(f"1) avg {avg:,.0f} QPS vs peak (x3) {avg * 3:,.0f} QPS -> size for peak")
# Trap 2: bits vs bytes. Network links are quoted in bits.
mb_per_s = 250
print(f"2) {mb_per_s} MB/s is {mb_per_s * 8:,} Mbit/s -> a 1 Gbit NIC is already saturated x2")
# Trap 3: forgetting replication, indexes and headroom on storage.
raw_tb = 40
real_tb = raw_tb * 3 * 1.3 / 0.7 # 3 replicas, +30% index/overhead, fill disks to 70%
print(f"3) raw {raw_tb} TB vs provisioned {real_tb:,.0f} TB")
# Trap 4: one server per request type. Concurrency = throughput x latency (Little's law).
qps, latency_s = 5_000, 0.080 # 80 ms per request, mostly waiting on the DB
in_flight = qps * latency_s
workers_per_box = 64
print(f"4) {qps:,} QPS x {latency_s * 1000:.0f} ms = {in_flight:.0f} requests in flight "
f"-> {in_flight / workers_per_box:.1f} boxes of {workers_per_box} workers, before headroom")
# Handy powers of two for storage sizing
for name, p in (("KB", 10), ("MB", 20), ("GB", 30), ("TB", 40)):
print(f" 2^{p} = {2**p:>17,} (~1 {name})")1) avg 2,315 QPS vs peak (x3) 6,944 QPS -> size for peak 2) 250 MB/s is 2,000 Mbit/s -> a 1 Gbit NIC is already saturated x2 3) raw 40 TB vs provisioned 223 TB 4) 5,000 QPS x 80 ms = 400 requests in flight -> 6.2 boxes of 64 workers, before headroom 2^10 = 1,024 (~1 KB) 2^20 = 1,048,576 (~1 MB) 2^30 = 1,073,741,824 (~1 GB) 2^40 = 1,099,511,627,776 (~1 TB)
- Sizing for the average. The system falls over at 7 p.m., not at the daily mean. Always state a peak factor and design for it.
- Bits versus bytes. Storage is in bytes, networks in bits. Being off by 8x on bandwidth is a classic.
- Raw storage only. 40 TB of data is over 200 TB of provisioned disk once you add three replicas, index overhead and headroom.
- Ignoring concurrency. Little's law — requests in flight = throughput × latency — tells you how many workers, connections or threads you need. Slow downstream calls multiply the servers you need even at the same QPS.
- Estimating and then ignoring it. The worst trap of all: computing that writes are 35/s and then proposing a sharded Cassandra cluster.
CAP and PACELC — the consistency trade-off, precisely
When data is copied to more than one machine, those copies can disagree. The CAP theorem says that during a network partition — some nodes cannot reach others — a replicated system must choose between:
- Consistency (C): every read sees the most recent successful write, as if there were a single copy (formally, linearizability).
- Availability (A): every request to a non-failed node gets a non-error response, even if the node cannot reach its peers.
Partition tolerance is not really a choice. Networks do partition — a switch fails, a cable is cut, a zone loses connectivity — so any system running on more than one machine must tolerate it. The popular phrasing "pick two of three" is therefore misleading: the real statement is "when a partition happens, you pick C or A". A "CA" system is just a single-node system that has not met a partition yet.
Worked example. Two replicas of an account balance; the link between them breaks; one client writes to replica A while another reads from replica B. The same event, handled both ways:
# A two-replica store during a network partition. Same failure, two policies.
class Replica:
def __init__(self, name):
self.name, self.data, self.peer, self.linked = name, {}, None, True
def write(self, key, value, mode):
if not self.linked and mode == "CP":
return f"{self.name}: REJECT write (cannot reach peer, would diverge)"
self.data[key] = value
if self.linked:
self.peer.data[key] = value # synchronous replication
return f"{self.name}: OK, replicated"
return f"{self.name}: OK locally (peer will be reconciled later)"
def read(self, key, mode):
if not self.linked and mode == "CP":
return f"{self.name}: REJECT read (might be stale)"
return f"{self.name}: {key} = {self.data.get(key)}"
for mode in ("CP", "AP"):
a, b = Replica("A"), Replica("B")
a.peer, b.peer = b, a
print(f"--- {mode} ---")
print(a.write("balance", 100, mode))
a.linked = b.linked = False # the network splits
print(a.write("balance", 40, mode)) # client 1 talks to A
print(b.read("balance", mode)) # client 2 talks to B--- CP --- A: OK, replicated A: REJECT write (cannot reach peer, would diverge) B: REJECT read (might be stale) --- AP --- A: OK, replicated A: OK locally (peer will be reconciled later) B: balance = 100
The CP store refuses both the write and the read rather than risk two different balances — correct, but unavailable to those clients until the network heals. The AP store accepts the write on A and serves the old value of 100 from B — available, but a client just read a stale balance, and when the link heals the two replicas must reconcile (last-writer-wins, version vectors or application-level merging). Neither is wrong. Which one you want depends on the feature: a bank ledger wants CP; a shopping cart, a like counter or a social timeline wants AP.
PACELC — the trade-off that exists every day
Partitions are rare. CAP says nothing about the other 99.9% of the time. PACELC fills the gap: if Partition, choose Availability or Consistency; Else, choose Latency or Consistency. Even with a perfect network, a write that waits for every replica to acknowledge is consistent and slow (it pays a cross-zone or cross-region round trip on every commit); a write that returns after one replica is fast and briefly inconsistent. That everyday latency cost is what you are actually trading most of the time.
| System (default config) | On partition | Otherwise | In practice |
|---|---|---|---|
| Single-primary Postgres/MySQL with async replicas | C for the primary (replicas may be stale) | L — replica reads can lag | Read your own writes from the primary; tolerate lag elsewhere |
| Postgres with a synchronous standby | C | C — each commit waits for the standby | No data loss on failover, +1–2 ms per commit across zones |
| Cassandra / Dynamo-style stores | A | L | Tunable: QUORUM reads and writes buy consistency back at a latency cost |
| DynamoDB | A | L by default | Eventually consistent reads by default; strongly consistent reads optional, at twice the cost |
| MongoDB (majority write concern) | C | C | Writes acknowledged by a majority; minority side stops accepting writes |
| Spanner / CockroachDB | C | C | Consensus on every write; pays latency to be consistent globally |
Between "strong" and "eventual" there is a useful ladder of guarantees you can name: read-your-writes (you always see your own updates — fix: route a user's reads to the primary for a few seconds after they write), monotonic reads (you never see time go backwards — fix: pin a user to one replica), causal consistency (a reply is never visible before the message it answers), and eventual consistency (replicas converge if writes stop). Most user-facing features need one of the middle rungs, not full linearizability. The databases lesson goes deeper into replication and quorums.
What to say in the interview is never "I'll use an AP database". It is per feature: "Payments and inventory are CP — I would rather reject an order than oversell. The feed and like counts are AP with eventual consistency — a like showing up two seconds late costs nothing."
Read-heavy versus write-heavy workloads
The read/write ratio from step 3 is one of the most decisive numbers in the whole interview, because the two shapes push the architecture in opposite directions.
| Read-heavy (100:1 or more) | Write-heavy (writes dominate or match reads) | |
|---|---|---|
| Examples | Product catalogue, news site, URL redirect, social timeline reads, docs | Metrics and logs, IoT sensors, chat ingest, location pings, clickstream, ad impressions |
| Main tools | Caches (CDN, Redis, in-process), read replicas, denormalised read models, precomputation | Queues and logs (Kafka), batching, append-only / LSM-tree stores, partitioning by write key |
| Storage fit | B-tree indexes, relational stores with replicas, search indexes | Cassandra, time-series databases, columnar stores for analytics |
| Consistency | Usually tolerant of seconds of staleness | Often tolerant of delayed visibility; durability of the write matters most |
| Main risks | Cache stampede, hot keys, stale data after writes | Write amplification, hot partitions, backpressure, unbounded storage growth |
Worked example. A URL shortener reads 100x more than it writes: a cache with a 99% hit ratio turns 10,000 redirects/s into 100 database reads/s, and one primary is plenty. A fleet-tracking system receives 50,000 GPS pings/s and reads a position only when someone opens a map: caching does nothing for it. Instead you batch pings, write the current position to an in-memory store (overwriting, not appending), and stream the history through a log into cheap columnar storage. Same scale in QPS, completely different architecture — and the ratio told you which one before you drew a box. Many real products are both: in the food delivery capstone, the menu is read-heavy and courier locations are write-heavy, and they get different stores.
Common mistakes, and what to do instead
- Jumping to a solution. Drawing a Kafka box before knowing the write rate. Instead: requirements and numbers first; every component must be justified by one of them.
- Over-engineering. Microservices, sharding and multi-region for 100 QPS. Instead: start with the simplest design that meets the numbers and say what would make you add each piece.
- Buzzwords without mechanism. "We'll use consistent hashing" with no idea what it solves. Instead: name the problem first — "adding a cache node should not remap every key" — then the technique.
- One database for everything. Instead: classify each dataset by access pattern, size and consistency need, and allow different stores.
- Ignoring failure. Every box drawn as always-up. Instead: for each component, say what happens when it dies and how fast it recovers.
- Averages instead of percentiles. "Latency is 50 ms." Instead: "p99 under 200 ms at peak load".
- Silence. The interviewer cannot grade reasoning they cannot hear. Instead: think out loud, and check in after each step.
- Not adapting. Defending a choice after the interviewer adds a constraint that breaks it. Instead: "Given that new requirement, I would change X to Y, because…" — updating on new information is a strong signal, not a weakness.
- Running out of time before the deep dive. Instead: watch the clock; the deep dive is where most of the grade is.
Practice
# Write your solution here
# Write your solution here
// Write your solution here
Finished reading? Mark this lesson complete to track your progress.
