Designing a Video Streaming Service
High Level Design·intermediate·~30 min read
- system-design
- video-streaming
- distributed-systems
- hld
What you'll learn
Problem Statement
Design a large-scale video streaming platform where users can upload, encode, store, and stream video content. The system must handle billions of users, support adaptive bitrate streaming, process hundreds of hours of new video every min…
Requirements
Functional Requirements
1. Upload videos of various formats and sizes 2. Stream/watch videos with adaptive quality based on network speed 3. Search videos by title, description, and tags 4. Like, dislike, and comment on videos 5. Subscribe to channels and recei…
Non-Functional Requirements
1. High availability — streaming must work 24/7 (downtime = lost viewers permanently) 2. Low latency — playback should start within 1-2 seconds 3. Eventual consistency — view counts, likes can be slightly delayed 4. Reliability — uploade…
Extended Requirements
1. Recommendations and trending algorithms 2. Geo-restricted content delivery 3. Live streaming capability 4. Content moderation and copyright detection ---
Design Considerations
The system has fundamentally different characteristics for reads vs. writes: | Dimension | Upload (Write) | Streaming (Read) | |-----------|---------------|-------------------| | Ratio | 1 | 200 | | Latency tolerance | Minutes (async enc…
Capacity Estimation
Users and Traffic
| Metric | Value | |--------|-------| | Total registered users | 1.5 Billion | | Daily active users (DAU) | 800 Million | | Average videos watched per user per day | 5 | | Total views per second | 46,000 | | Upload:View ratio | 1:200 | |…
Storage
| Metric | Calculation | Value | |--------|-------------|-------| | Minutes uploaded per minute | 500 hrs × 60 | 30,000 min/min | | Average storage per minute of video | ~50 MB (compressed) | | | Raw storage growth per minute | 30,000 ×…
Bandwidth
| Direction | Calculation | Value | |-----------|-------------|-------| | Upload ingress | 230 uploads/sec × avg 22 MB | ~5 GB/s | | Streaming egress | 46K views/sec × avg 22 MB chunk | ~1 TB/s | The 200:1 ratio manifests as a 200:1 band…
System APIs
Upload Video
Why 202 Accepted (not 200 OK)? The video is received but not processed. Encoding takes minutes to hours. The client receives an immediate acknowledgment and monitors progress via the status URL. The video becomes publicly available only…
Search Videos
Pagination via opaque tokens (not numeric offsets): Since videos are constantly being uploaded and deleted, a numeric offset like offset=40 might skip or duplicate results between pages. Opaque tokens encode cursor state that's stable re…
Stream Video
Parameters: - offset: Byte position for seeking or resuming playback - codec: h264, vp9, av1 — based on client decoder support - resolution: 240p, 360p, 480p, 720p, 1080p, 1440p, 2160p — based on bandwidth The client's adaptive bitrate a…
Database Design
Schema
Replication Strategy: Primary-Secondary
With a 200:1 read-to-write ratio, the read path needs significant scaling: Why this works for video metadata: - Writes are rare (230/sec) — primary handles them easily - Reads are massive (46K/sec) — spread across many secondaries - Stal…
High-Level Architecture (Detailed)
---
Design Evolution & Trade-offs
How this system grew from an MVP to what a production version looks like — and the choices you'd defend in an interview. Video streaming is a fundamentally asymmetric workload: uploads are rare and CPU-heavy, streaming is constant and ba…
Stage 1 — MVP: One box, one database, one bucket
The simplest thing that could possibly work for a video streaming service — enough to demo, enough to fail interestingly. - A single API server (monolith) that handles upload, encoding, metadata, and streaming. - A single relational data…
Stage 2 — Scale-out: Separate the read path from the write path
The single most important architectural move for a streaming service happens before you shard, cache, or CDN — split encoding from serving. - API layer becomes stateless; N replicas behind an L7 load balancer. - Encoding moves off the AP…
Stage 3 — Bottleneck: Origin bandwidth (egress fan-out)
At 46K concurrent views/sec × ~5 Mbps average, the steady-state egress requirement is roughly 1 Tbps sustained — and that assumes a globally distributed viewer base. Serving that from your origin storage in one region is not economically…
Stage 4 — Second bottleneck: Encoding throughput and cost
CDN solves the read side. Now the write side becomes the problem. At 500 hours of new video per minute, each hour needing ~15 CPU-minutes across 15 codec × resolution variants, you need roughly 7,500 encoder-minutes of CPU per minute of…
Stage 5 — Third bottleneck: Metadata read amplification
You've solved bytes (CDN) and encoding (segment-parallel spot fleet). Now every video-page load fires 3–8 metadata reads: video row, uploader row, top comments, subscription state, like state, view count. At 46K playback starts/sec × 5 r…
Stage 6 — Mature architecture
Bringing every stage together. Notice the two independent paths: the write path (upload → encode → publish) and the read path (viewer → CDN → manifest → chunks). They intersect only at object storage — which is exactly the correct seam. ---
Cost vs Complexity — When to Pick What
| Concern | MVP choice | Growth choice | Enterprise / Netflix-scale | |---------|-----------|---------------|----------------------------| | Compute (API) | Single VM | K8s deployment, HPA on RPS | Multi-region K8s, active-active | | Enc…
Migration Path (MVP → Enterprise)
Order matters. Each step assumes the previous one is stable. Skipping ahead breaks assumptions the later steps rely on. 1. Move encoding off the API path (introduce Kafka + a worker pool). This is the first thing you do, before scaling a…
Detailed Upload Flow
Step-by-Step
Resumable Upload Protocol (Detailed)
Large video files (1-50 GB) frequently encounter network interruptions. A non-resumable upload means re-sending gigabytes of data. Why this matters at scale: With 230 uploads/second and average file sizes of hundreds of MB, even a 1% fai…
Video Encoding Pipeline (Deep Dive)
Why Encode to Multiple Formats?
| Device | Codec Support | Why It Matters | |--------|--------------|----------------| | Old smart TVs, consoles | H.264 only | Broadest compatibility | | Modern browsers (Chrome, Firefox) | VP9 | 30-50% smaller than H.264 at same qualit…
Encoding Architecture
Adaptive Bitrate Streaming (ABR)
The client continuously monitors download speed and switches quality levels mid-stream: The manifest file (generated during encoding) tells the client what quality levels exist and where to find each chunk. The client makes all switching…
Thumbnail Storage: Why Bigtable?
The Thumbnail Problem
Each video generates 3-5 thumbnails (different timestamps). With billions of videos: - Billions of small files (each 50-200 KB) - Random access pattern (need any video's thumbnail instantly) - Read-heavy (thumbnails shown on every search…
Why Not Regular File Storage?
Traditional file systems struggle with billions of tiny files: - Inode exhaustion - Directory lookup performance degrades - Metadata overhead per file exceeds the file content
Why Wide-Column Store (Bigtable)?
Advantages: - Stores small binary blobs efficiently (no filesystem overhead per file) - Row key lookup is O(1) via hash — instant access to any video's thumbnails - Column families let you store multiple thumbnails per video in one row -…
Video Deduplication (Detailed)
The Problem
Users frequently upload: - The same viral video copied from elsewhere - Slightly modified versions (different resolution, added watermark, trimmed) - TV show clips that thousands of users all capture independently Without deduplication,…
Inline vs. Post-Process Deduplication
| Approach | When It Runs | Pros | Cons | |----------|-------------|------|------| | Inline (preferred) | During upload, before encoding | Saves encoding CPU immediately; no wasted storage ever written | Adds latency to upload pipeline |…
Block-Matching Algorithm
Why block-level, not whole-file? - Whole-file hash (like MD5) misses near-duplicates (same content, different container format) - Block matching detects partial overlaps (e.g., a clip from a longer video) - Perceptual hashing handles re-…
Data Partitioning and Sharding (Deep Dive)
Why VideoID Sharding (Not UserID)
| Sharding Key | Distribution | Access Pattern | Problem | |-------------|--------------|----------------|---------| | UserID | All videos from one user on one shard | Viewing user's channel is fast | Celebrity with 100M subscribers → on…
Consistent Hashing Implementation
Handling Hot Videos
Even with VideoID sharding, a single viral video creates a hot partition: Solution layers: 1. CDN absorbs most traffic — viral videos are definitely in CDN cache 2. Read replicas for metadata — multiple copies of the metadata shard 3. Ca…
Caching Strategy (Deep Dive)
Multi-Level Cache Architecture
CDN Strategy: The 80-20 Rule
Dynamic caching decisions: - New uploads from popular channels → pre-warm CDN immediately - Videos trending upward → aggressively cache at more edge locations - Videos declining in views → let CDN evict via LRU - Long-tail content (watch…
Metadata Cache (LRU)
| Parameter | Value | Rationale | |-----------|-------|-----------| | Cache size | ~500 GB | Hot metadata for top 20% of active videos | | Eviction policy | LRU | Popular videos naturally stay cached | | TTL | 5 minutes | Eventual consis…
Load Balancing (Detailed)
Consistent Hashing with Dynamic HTTP Redirections
Traditional load balancing (round-robin, least-connections) doesn't optimize for cache locality. If any server can handle any request, in-server caches have low hit rates. Improved approach: Why this hybrid works: - Normal case: consiste…
Fault Tolerance (Comprehensive)
Failure Scenarios and Recovery
| Component | Failure Mode | Impact | Recovery Mechanism | |-----------|-------------|--------|-------------------| | Upload Service node | Crashes mid-upload | Current chunk lost | Client resumes from last confirmed offset | | Processin…
Consistent Hashing's Role in Fault Tolerance
---
Search System
Architecture
Ranking Signals
| Signal | Weight | Rationale | |--------|--------|-----------| | Text relevance (TF-IDF/BM25) | High | Query terms matching title/description | | View count | Medium | Social proof — popular videos are likely relevant | | Recency | Medi…
Complete Data Flow: Watching a Video
---
Summary: Key Design Decisions and Their Rationale
| Decision | What It Solves | Trade-off | |----------|---------------|-----------| | HTTP 202 for uploads | Decouples upload from encoding; handles long processing | Client must poll for completion | | Processing queue | Absorbs upload s…