System Design Fundamentals

Requirements, latency, throughput and a framework for design questions.

System design is deciding how the pieces of a software system (servers, databases, caches, queues) fit together so it works correctly for many users, stays fast and survives failures. There is rarely one right answer; the skill is choosing sensible trade-offs and explaining why.

Key vocabulary

• Latency: how long one request takes (milliseconds).
• Throughput: how many requests you handle per second (QPS).
• Availability: the fraction of time the system works. "Three nines" (99.9%) allows about 8.8 hours of downtime a year; "four nines" about 53 minutes.
• Scalability: ability to handle growth by adding resources.
• Reliability: keeps working correctly despite faults.

A framework for any design

• 1. Clarify requirements. Functional (what it does) and non-functional (scale, latency, availability, consistency). Ask questions; do not assume.
• 2. Estimate scale. Users, requests per second, data size. Back-of-the-envelope is fine.
• 3. Sketch the API and data model. Main endpoints and tables.
• 4. Draw a high-level design. Clients, load balancer, app servers, database, cache.
• 5. Deep-dive on the hard parts. The bottleneck, the trickiest requirement.
• 6. Discuss trade-offs and failure modes. What breaks? What would you change with 100x traffic?

Back-of-the-envelope estimates

Handy numbers: a day has about 86,400 seconds (call it 100,000). One million requests per day is about 12 per second. A read from memory takes around 100 nanoseconds, an SSD read around 100 microseconds, and a cross-continent network round trip around 150 milliseconds.

# Example: a photo app with 10M daily users
users = 10_000_000
uploads_per_user = 2
avg_photo_mb = 3

uploads_per_day = users * uploads_per_user            # 20M
uploads_per_sec = uploads_per_day / 86_400            # ~230/s average
peak_per_sec = uploads_per_sec * 3                    # ~700/s at peak
storage_per_day_tb = uploads_per_day * avg_photo_mb / 1_000_000   # 60 TB/day
print(round(uploads_per_sec), round(peak_per_sec), storage_per_day_tb)

231 694 60.0

Sixty terabytes a day tells you immediately that the design needs object storage (like S3) and a CDN, not a database column holding image bytes.

Read-heavy vs write-heavy

Most consumer systems read far more than they write (often 100 to 1 or more). That points to caching and read replicas. Write-heavy systems (logging, metrics) suggest queues, batching and append-friendly storage.

daily = 50_000_000
avg = daily / 86_400        # ~579 QPS
peak = avg * 5              # ~2,894 QPS
print(round(avg), round(peak))
# One 1,000 QPS server is not enough at peak: need at least 3-4 plus redundancy.