Foundations: The Architect's Compass

What you'll learn

• Why System Design?

  System design interviews test your ability to design large-scale distributed systems. The key is to think about scalability, reliability, and performance.

• Interview Delivery Framework

  The most common failure mode: not delivering a working system. A structured approach provides a clear fallback path when overwhelmed.

• Step 1: Requirements (~5 min)

  Functional Requirements: - "Users/Clients should be able to..." statements - Engage in back-and-forth with interviewer (like speaking to a PM) - Arrive at top 3 core features — a long list hurts more than helps Non-functional Requirement…

• Step 2: Core Entities (~2 min)

  - List core entities as a simple bullet list - Helps define terms and understand central data - Don't list the full data model yet — build columns/fields later - Questions: Who are the actors? What nouns satisfy the requirements?

• Step 3: API / System Interface (~5 min)

  - Define the contract between system and users - Often maps directly to functional requirements - Sketch 4-5 key endpoints in a couple minutes and move on - Protocol: REST (default), GraphQL (diverse clients), gRPC (internal services) -…

• Step 4: High-Level Design (~10-15 min)

  - Draw boxes and arrows: components (servers, databases, caches) and interactions - Go through API endpoints one-by-one, building sequentially - Focus on simple design which first meets the core requirements - Note areas for future compl…

• Step 5: Deep Dives (~10 min)

  Use remaining time to: - Ensure non-functional requirements are met - Address edge cases and bottlenecks - Improve based on interviewer probes Seniority expectations: - Junior: Interviewer points out improvement areas - Senior: Should id…

• Architecture Building Philosophy

  Two fundamental approaches exist for building a system's architecture in an interview or in practice:

• Spiral Building

  Design the core component first (typically the database or communication protocol that best fits your use case) and build the entire system around that. When it works: You have deep experience and immediately recognize the core pattern.…

• Incremental Building (Recommended for Interviews)

  Start with a day-zero architecture (single server, single database) and evolve iteratively: 1. Start with the simplest possible architecture 2. Analyze how each component would behave under load and at scale 3. Identify the bottleneck 4.…

• The Six Foundation Pillars

  Every architectural decision in system design affects one of these six areas: | Pillar | What It Means | Example Decision | |--------|---------------|-----------------| | Database | Data storage and access patterns | SQL vs NoSQL, indexi…

• CAP Theorem — Practical Decision Guide

  Since network partitions are unavoidable, the real choice is Consistency vs Availability: | Choose | Behavior During Partition | Good For | |--------|--------------------------|----------| | AP (Availability) | Every node serves requests…

• Numbers to Know

• Powers of 2 Reference

  | Power of 2 | Exact | Power of 10 (Approx) | Bytes | Mapped Value | |------------|-------|---------------------|-------|-------------| | 2^7 | 128 | ~10^2 | — | — | | 2^8 | 256 | ~10^2 | — | — | | 2^10 | 1,024 | ~10^3 | 1 KB | 1 Thousan…

• Latency Hierarchy

  | Operation | Latency | |-----------|---------| | L1 cache reference | ~0.5 ns | | Branch mispredict | ~5 ns | | L2 cache reference | ~7 ns | | Mutex lock/unlock | ~25 ns | | Main memory reference | ~100 ns | | Compress 1KB | ~10 μs | |…

• Availability Table

  | Availability | Downtime/Year | Downtime/Month | Downtime/Week | Downtime/Day | |-------------|--------------|----------------|---------------|--------------| | 99% (two 9s) | 3.65 days | 7.31 hours | 1.68 hours | 14.40 min | | 99.9% (t…

• Component Scale Triggers

  | Component | Comfortable Capacity | Scale When... | |-----------|---------------------|---------------| | Single Postgres | Few TB storage, ~50K TPS | Write > 10K TPS; storage > TB | | Single Redis | 100K+ ops/sec, up to 1TB RAM | Hit r…

• Component Throughput Estimates (Per Instance)

  | Component | Reads/sec | Writes/sec | Max Effective Capacity | |-----------|-----------|------------|----------------------| | RDBMS (Postgres) | ~10,000 | ~5,000 | ~3 TB storage | | Distributed Cache (Redis) | ~100,000 | ~100,000 | 16-…

• Quick Estimation Approach

  - Don't do calculations at the start - Do them when making decisions: "Should we shard?" → then calculate - Walk through math in context: "50K req/sec, each server handles ~5K, so ~10 servers plus headroom" - Modern hardware is more powe…

• Back-of-Envelope Conversion Shortcuts

  Time conversions: Request rate shortcuts: Storage units: Handy product: 1 million rows × 1 KB each = 1 GB Common field sizes: Typical user assumptions (if not given): - 500M total users, 1M daily active users (0.2% DAU ratio for niche; u…

• System Limits — What One Machine Can Actually Do

  Before drawing boxes, know the ceiling of a single commodity server. "Should we shard?" and "should we add a load balancer?" are answered by comparing the workload to these numbers — not by guessing. Interviewers explicitly test whether…

• Concurrent Connections Per Machine

  The famous C10K → C10M problem. In 1999 handling 10,000 concurrent connections was hard; today handling 10M on one box is achievable with the right stack. Ballpark limits for a modern Linux server (16-32 cores, 64-128 GB RAM): | Connecti…

• Bandwidth Per Server

  Bandwidth is the second wall people forget. A server's NIC caps its aggregate throughput regardless of how fast the CPU is. | NIC | Line rate | Practical sustained (application-level) | |---|---|---| | 1 Gbps NIC | ~125 MB/s | ~100 MB/s…

• CPU, Memory, and Disk Per Server

  | Resource | Modern commodity box | Practical ceiling | |---|---|---| | CPU | 16-64 cores @ ~3 GHz | ~50K-200K req/sec for simple CRUD; ~10K-30K for CPU-heavy work (encoding, ML inference) | | RAM | 64-512 GB | Redis fits everything hot;…

• Framework / Runtime Overhead (Realistic Req/sec Ceilings)

  | Stack | Simple JSON endpoint (single core) | With DB round-trip | |---|---|---| | Node.js (Express) | ~30K rps | ~5K rps | | Go (net/http) | ~80K rps | ~15K rps | | Rust (Actix / Axum) | ~200K rps | ~20K rps | | Java (Spring Boot) | ~2…

• How to Actually Use These Numbers in an Interview

  Say the target is 1M concurrent chat users: 1. Connection budget. WebSocket active traffic ≈ 100K per server → 10 chat gateway servers minimum, more likely 20 for headroom + rolling deploys. 2. Bandwidth check. 1M users × 10 msg/min × 20…

• Key Technologies Quick Reference

  | Technology | Primary Use | When to Reach For It | |-----------|-------------|---------------------| | PostgreSQL | Transactional data, default DB | Most systems (start here) | | CockroachDB | Distributed SQL with ACID | Global consiste…

• Key Architecture Patterns

• 1. Pushing Real-Time Updates

  Start simple → add complexity only when needed: - HTTP polling → SSE → WebSockets - Pub/Sub services for decoupling publishers/subscribers - Consistent hash ring for routing to stateful servers

• 2. Handling Long-Running Tasks (Two Async Patterns)

  - Web server validates request → pushes job to queue → returns job ID immediately - Separate workers execute tasks asynchronously - Don't use queues for short-running tasks — synchronous is simpler and provides better UX Pattern A — Defe…

• 3. Dealing with Contention

  - Start with database-level solutions (pessimistic/optimistic locking) - Move to distributed locks only when spanning multiple services - Queue-based serialization for ordering conflicting operations

• 4. Scaling Reads (Natural Progression)

  1. Optimize within DB (indexing, denormalization) 2. Read replicas for horizontal scale 3. External cache (Redis) for hot data 4. CDN for static/semi-static content

• 5. Scaling Writes

  - Horizontal sharding (choose partition key carefully) - Write queues to buffer spikes (Kafka) - Batching to reduce per-operation overhead - Load shedding for overload protection

• Distributed Key-Value Store Design

  A fundamental building block for many systems. Key techniques: | Problem | Solution | |---------|----------| | Partitioning large datasets | Consistent hashing | | High availability reads | Replication + multi-datacenter | | Conflict res…

• Unique ID Generation (Distributed)

  | Approach | Pros | Cons | |----------|------|------| | Auto-increment by K (multi-master) | Uses existing DB | Hard to scale, not time-sortable | | UUID (128-bit) | No coordination | Too long, not sortable | | Ticket server (centralized…

• 6. Handling Large Files

  - Presigned URLs: client uploads directly to blob storage (bypasses app servers) - CDN with signed URLs for downloads - Chunked/multipart upload for resumability

• 7. Multi-Step Processes

  - Range from simple orchestration to workflow engines (Temporal, Step Functions) - Event sourcing: each step emits events triggering subsequent steps - Key shift: "scattered state management" → "declarative workflow definitions"

• 8. Platform Layer (Separating Web from Logic)

  Instead of web servers talking directly to databases, introduce an intermediate platform/service layer: Three reasons to do this: 1. Independent scaling — web tier and platform tier scale separately. A new API endpoint means more platfor…

• Scaling an Existing Microservice (100x Growth)

  A common interview question: "Your service handles 100 RPM today. How would you scale it to 10,000 RPM?" This tests systematic thinking about identifying and eliminating bottlenecks.

• Methodology

• Concrete Example: Scaling a Sign-Up Service

• Scaling Decision Tree

  ---

• Quick Problem Fixes

  Small, high-yield recipes for the problems interviewers love to probe and production teams hit weekly. Each entry states the failure mode, the pattern, real usage, and the gotcha. ---

• Fix 1 — How to Prevent Double Payments (Idempotent Financial Operations)

  The failure mode. A client calls POST /payments, the server charges the customer, then the response is lost (client timeout, retry-happy proxy, mobile-network drop). The client retries. The customer is now charged twice. Common causes: -…

• Fix 2 — Cross-System Transactions (Saga, Outbox, Idempotency + Ordering, TCC)

  The failure mode. You need to update two systems atomically: - Charge the customer (Payments service, MySQL). - Reserve inventory (Inventory service, DynamoDB). - Enqueue a shipping event (Kafka). The classic monolith answer — a database…