Designing a Collaborative Document Editor
High Level Design·advanced·~25 min read
- system-design
- real-time
- collaboration
- crdt
- websocket
- hld
What you'll learn
1. Problem Statement and Requirements
What Is a Collaborative Document Editor?
A collaborative document editor allows multiple users to edit the same document simultaneously and see each other's changes in real-time. Users see live cursors, presence indicators, and instant text updates -- as if everyone were writin…
Functional Requirements
- Create, read, update, and delete documents - Multiple users editing the same document concurrently - Real-time propagation of changes to all connected editors - Cursor and selection tracking (see where others are typing) - Presence ind…
Non-Functional Requirements
- Low latency: Propagate edits to all users within 100ms under normal conditions - Consistency: All users converge to the same document state (eventual consistency) - Conflict-free: Concurrent edits must be merged without data loss -- no…
Back-of-the-Envelope Estimation
For a system with 50 million documents: - 5% active at any time = 2.5 million active documents - Average 3 editors per active document = 7.5 million WebSocket connections - Average typing speed: 5 characters/second per user = 37.5 millio…
2. The Core Challenge: Concurrent Edits
The fundamental problem in collaborative editing is handling simultaneous changes to the same document position.
The Naive Approach Fails
Consider two users editing the text "HELLO": This is unacceptable. Both edits must be preserved. The challenge is determining what the correct merged result should be and computing it efficiently.
Two Approaches to Conflict Resolution
The field has developed two major approaches: 1. Operational Transformation (OT) -- Transform operations against each other 2. CRDTs (Conflict-free Replicated Data Types) -- Design data structures where merge order doesn't matter
3. Operational Transformation (OT)
OT treats every edit as an operation (insert or delete at a position) and transforms concurrent operations so they can be applied in any order and still produce the same result.
How OT Works
When two operations are generated concurrently, one must be transformed against the other before applying:
OT Transformation Rules
Server-Ordered OT
In practice, a central server assigns a canonical order to all operations: 1. Client sends operation to server 2. Server assigns a sequence number (canonical ordering) 3. Server transforms the operation against any concurrent ops it hasn…
OT Characteristics
Advantages: - Simpler operations (insert/delete with position) - Well-proven at massive scale - Lower per-character memory overhead than CRDTs - Efficient for centralized architectures Disadvantages: - Requires a central server for order…
4. CRDTs (Conflict-free Replicated Data Types)
CRDTs take a different approach: design the data structure so that operations are commutative -- applying them in any order always produces the same result.
How Text CRDTs Work
Instead of positions (which shift), each character gets a globally unique, immutable ID. The document is an ordered sequence of these IDs.
Popular CRDT Approaches
| Approach | Mechanism | Trade-off | |----------|-----------|-----------| | RGA (Replicated Growable Array) | Linked list with unique IDs per element | Good balance of speed and simplicity | | YATA (Yet Another Transformation Approach) |…
CRDT Characteristics
Advantages: - No central coordinator needed for conflict resolution - Excellent offline support (merge when reconnected) - Provably correct convergence - Works in peer-to-peer architectures Disadvantages: - Higher memory overhead (unique…
5. Choosing OT vs CRDT
| Factor | OT | CRDT | |--------|-----|------| | Architecture | Centralized (needs server ordering) | Decentralized (peer-to-peer possible) | | Offline support | Limited (must reconcile with server) | Excellent (merge locally) | | Memory…
6. High-Level Architecture
Component Responsibilities
| Component | Role | |-----------|------| | Client | Local editing, optimistic updates, send ops over WebSocket | | Load Balancer | Route by documentid to correct collaboration server | | Collaboration Server | OT engine, canonical order…
7. WebSocket Management and Routing
Why WebSockets?
Collaborative editing requires bidirectional, low-latency communication. HTTP polling is too slow; Server-Sent Events are unidirectional. WebSockets provide full-duplex, persistent connections with minimal overhead.
Consistent Hashing for Document Routing
All editors of the same document must connect to the same collaboration server (so the OT engine has the full operation context):
Handling Server Failures
When a collaboration server fails: 1. Health check detects failure 2. Consistent hashing reassigns documents to next server on ring 3. New server loads latest snapshot + recent operations from storage 4. Clients reconnect (WebSocket clos…
Cross-Server Communication
For system-wide events (user goes offline, document access changes):
8. Document Storage and Persistence
Dual Storage Strategy
Operation Log Schema
Snapshot Strategy
- Create snapshot every N operations (e.g., every 1000 ops) or every T seconds - Snapshot = serialized full document state + the sequence number it represents - On client connect: serve latest snapshot + all operations after that snapsho…
Storage Technology Choices
| Store | Use Case | Why | |-------|----------|-----| | Append-only log (Kafka/Cassandra) | Operation log | High write throughput, ordered, durable | | Object store (S3) | Snapshots | Cheap, durable, infrequent reads | | In-memory (Redis…
Design Evolution & Trade-offs
How this system grew from an MVP to what a production version looks like — and the choices you'd defend in an interview. Interviewers rarely want the final diagram out of the gate. They want to see you reason your way there: start simple…
Stage 1 — MVP: One box, one database
The simplest thing that could possibly work for a collaborative editor. - A single Node.js (or Go) API server hosting both HTTP endpoints and WebSocket upgrades. - A single Postgres database storing documents, users, and an operation log…
Stage 2 — Scale-out: Multiple pods behind a load balancer
- API layer split into a stateless HTTP tier (auth, doc CRUD, share links) and a stateful WebSocket tier (OT engine, presence). - HTTP tier is trivially horizontally scaled behind an L7 load balancer. - WebSocket tier is fronted by an L7…
Stage 3 — Bottleneck: Per-document OT contention & hot documents
This is the characteristic scaling wall of a collaborative editor. Unlike a typeahead or a URL shortener where you can shard by user or key almost arbitrarily, a collaborative editor has a hard constraint: every editor of a given documen…
Stage 4 — Second bottleneck: Op-log write throughput
Once OT contention is unblocked, the next wall is durability. Every keystroke on every document must be written somewhere durable before we can safely ack it to the originating client (otherwise a pod crash after ack loses data — an unac…
Stage 5 — Third bottleneck: Presence fan-out
Presence (cursors, selections, "who's online") is deceptively expensive. It's low-value data — nobody's document is lost if we drop a cursor update — but the message rate can dwarf actual edits. A doc with 500 editors, each moving their…
Cost vs Complexity — When to Pick What
A summary table showing which choice is appropriate at each maturity level: | Concern | MVP choice | Growth choice | Enterprise choice | |---------|-----------|---------------|-------------------| | Compute | Single VM (HTTP + WS on one…
Migration Path (MVP → Enterprise)
A stepped list showing how you'd move a live system without a maintenance window. Steps are ordered by dependency — later steps assume the earlier ones landed. 1. Introduce OT / CRDT before anything else. This is a correctness change, no…
9. Presence and Cursor Tracking
Cursor Broadcasting
Each client periodically sends its cursor position: Throttling: Cursor updates are throttled to ~10 updates/second per user. More frequent updates waste bandwidth without perceptible improvement.
Presence State Machine
Presence Data Store
Presence is ephemeral -- stored in Redis with TTL:
10. Scaling to Millions of Documents
Horizontal Scaling Strategy
Hot Document Problem
A viral document with 10,000 concurrent editors overwhelms a single server. Solutions: Approach 1: Section-based sharding Approach 2: Hierarchical broadcasting
11. Version History and Time Travel
Reconstructing Past Versions
Since we store the complete operation log, any historical version is reconstructable:
Version Metadata
Branching (Optional Advanced Feature)
Fork a document at any version to create an independent copy. Implemented as: create new document with a snapshot at the fork point. Future operations go to the new document's log.
12. Offline Editing and Reconnection
Client-Side Operation Queue
When connectivity is lost: 1. Client continues editing locally (optimistic UI) 2. All operations are queued in local storage (IndexedDB) 3. Document state diverges from server state
Reconnection Protocol
Conflict Resolution During Reconnect
With OT: The server transforms the client's buffered operations against all operations that occurred during the offline period. This may require multiple transformation passes if many operations occurred. Edge case: Very long offline per…
13. Security and Access Control
Permission Model
Operation Validation
Every operation arriving at the server is validated: - Is the user authenticated and authorized for this document? - Is the operation well-formed (valid position, valid type)? - Does the operation's base version match (no missed operatio…
14. Complete Data Flow
Operation Lifecycle (End to End)
15. Summary: Key Design Decisions
| Decision | Choice | Rationale | |----------|--------|-----------| | Conflict resolution | OT (server-ordered) | Centralized service, proven at scale, lower memory | | Transport | WebSocket | Bidirectional, low-latency, persistent conne…