Foundations: Back-of-Envelope — Numbers & System Limits

What you'll learn

• Numbers to Know

• Powers of 2 Reference

  | Power of 2 | Exact | Power of 10 (Approx) | Bytes | Mapped Value | |------------|-------|---------------------|-------|-------------| | 2^7 | 128 | ~10^2 | — | — | | 2^8 | 256 | ~10^2 | — | — | | 2^10 | 1,024 | ~10^3 | 1 KB | 1 Thousan…

• Latency Hierarchy

  | Operation | Latency | |-----------|---------| | L1 cache reference | ~0.5 ns | | Branch mispredict | ~5 ns | | L2 cache reference | ~7 ns | | Mutex lock/unlock | ~25 ns | | Main memory reference | ~100 ns | | Compress 1KB | ~10 μs | |…

• Availability Table

  | Availability | Downtime/Year | Downtime/Month | Downtime/Week | Downtime/Day | |-------------|--------------|----------------|---------------|--------------| | 99% (two 9s) | 3.65 days | 7.31 hours | 1.68 hours | 14.40 min | | 99.9% (t…

• Cloud SLA Numbers (Vendor-Published)

  Real numbers you can quote when someone asks "how durable is X?" — these are the vendor's published contracts, not folk wisdom: | Service | Durability (annual) | Availability (SLA) | Notes | |---|---|---|---| | AWS S3 Standard | 99.99999…

• Component Scale Triggers

  | Component | Comfortable Capacity | Scale When... | |-----------|---------------------|---------------| | Single Postgres | Few TB storage, ~50K TPS | Write > 10K TPS; storage > TB | | Single Redis | 100K+ ops/sec, up to 1TB RAM | Hit r…

• Component Throughput Estimates (Per Instance)

  | Component | Reads/sec | Writes/sec | Max Effective Capacity | |-----------|-----------|------------|----------------------| | RDBMS (Postgres) | ~10,000 | ~5,000 | ~3 TB storage | | Distributed Cache (Redis) | ~100,000 | ~100,000 | 16-…

• Quick Estimation Approach

  - Don't do calculations at the start - Do them when making decisions: "Should we shard?" → then calculate - Walk through math in context: "50K req/sec, each server handles ~5K, so ~10 servers plus headroom" - Modern hardware is more powe…

• Back-of-Envelope Conversion Shortcuts

  Time conversions: Request rate shortcuts: Storage units: Handy product: 1 million rows × 1 KB each = 1 GB Common field sizes: Typical user assumptions (if not given): - 500M total users, 1M daily active users (0.2% DAU ratio for niche; u…

• System Limits — What One Machine Can Actually Do

  Before drawing boxes, know the ceiling of a single commodity server. "Should we shard?" and "should we add a load balancer?" are answered by comparing the workload to these numbers — not by guessing. Interviewers explicitly test whether…

• Concurrent Connections Per Machine

  The famous C10K → C10M problem. In 1999 handling 10,000 concurrent connections was hard; today handling 10M on one box is achievable with the right stack. Ballpark limits for a modern Linux server (16-32 cores, 64-128 GB RAM): | Connecti…

• Bandwidth Per Server

  Bandwidth is the second wall people forget. A server's NIC caps its aggregate throughput regardless of how fast the CPU is. | NIC (Network Interface Card) | Line rate | Practical sustained (application-level) | |-------------------------…

• CPU, Memory, and Disk Per Server

  | Resource | Modern commodity box | Practical ceiling | |---|---|---| | CPU | 16-64 cores @ ~3 GHz | ~50K-200K req/sec for simple CRUD; ~10K-30K for CPU-heavy work (encoding, ML inference) | | RAM | 64-512 GB | Redis fits everything hot;…

• Framework / Runtime Overhead (Realistic Req/sec Ceilings)

  | Stack | Simple JSON endpoint (single core) | With DB round-trip | |---|---|---| | Node.js (Express) | ~30K rps | ~5K rps | | Go (net/http) | ~80K rps | ~15K rps | | Rust (Actix / Axum) | ~200K rps | ~20K rps | | Java (Spring Boot) | ~2…

• How to Actually Use These Numbers in an Interview

  Say the target is 1M concurrent chat users: 1. Connection budget. WebSocket active traffic ≈ 100K per server → 10 chat gateway servers minimum, more likely 20 for headroom + rolling deploys. 2. Bandwidth check. 1M users × 10 msg/min × 20…

• Sizing a cloud fleet — "how many EC2 instances do I need for X RPS?"

  Interviewers love this question because most candidates answer with a memorized number. The honest answer: cloud providers don't cap concurrent connections directly — an EC2 instance runs out of one of four physical resources first, and…