Design Google Docs Like a Senior Engineer

Designing a Real-Time Collaborative Document Editor

Overview of the Design Process

• The design process is divided into four phases: gathering requirements, high-level design, database design, and handling concurrency.

Functional Requirements

• Users must be able to create, edit, and delete documents; editing is the most complex aspect. Multiple users should see changes in real-time.

• Documents need to persist through page refreshes to ensure no data loss during accidental refreshes.

• Content types include rich text (bold, italics), which influences database choices for storage. Clarifying questions about user access and offline support are also important.

Non-Functional Requirements

• Edits should appear within 100 milliseconds; thus, WebSocket connections are preferred over HTTP polling due to latency concerns.

• Availability should be at least 99.9%, with durability ensuring no data loss during server crashes by maintaining local state on the client side. Consistency requires all users to see the same document state eventually.

User Scale Considerations

• Estimating user scale includes assumptions about active users (1% of 10 million = 100,000) and concurrent editors per document (maximum of 500). This leads to around 20,000 active documents at peak times.

• Maximum document size is set at 500 kilobytes; this impacts how edits are processed and stored in the database as well as caching strategies.

Storage Estimation and High-Level Design

Storage Needs

• At peak usage with maximum document sizes, total active data could reach up to 10 gigabytes fitting within one Redis cluster's capacity.

• In worst-case scenarios involving maximum concurrent connections (10 million), approximately 200 servers would be needed for handling WebSocket connections efficiently with auto-scaling capabilities in place.

High-Level System Components

• The system architecture includes clients making requests via an API Gateway that manages authentication and rate limits before connecting through WebSockets for real-time updates on shared documents.

Handling Real-Time Updates

Update Mechanism

• When multiple users edit a document simultaneously, changes are published via Redis pub/sub mechanisms allowing real-time broadcasting across connected clients without significant latency issues from database reads.

Conflict Management Strategies

• If conflicts arise when two users attempt simultaneous edits on overlapping content areas, versioning helps manage these conflicts by rejecting outdated edits based on current document versions stored in the system's backend logic.(1039))

Database Structure Choices

Document Structure

• Core elements include ID, title, content (stored as JSON or JSONB for rich text formatting), collaborators' permissions linked back to user details such as name and email addresses.(739))

Database Options:

1. PostgreSQL: Supports relational integrity with JSONB fields.

2. MongoDB: Offers schema flexibility but lacks relational constraints.

3. Hybrid Approach: Using both databases can introduce complexity due to distributed transactions but allows leveraging strengths of each type.

Optimizing Data Transmission

Delta Updates vs Full Document Updates

• Sending only deltas instead of full documents reduces network load significantly; e.g., sending just a few bytes instead of megabytes when multiple users make small edits concurrently.(922))

Concurrency Techniques

1. Optimistic Locking: Detect conflicts based on version numbers leading to potential re-edit requests from clients.

2. Operational Transformation: Allows both edits from different users by transforming their positions dynamically.

3. Conflict-Free Replicated Data Types (CRDT): Handles merging automatically without needing central coordination.

By understanding these techniques better than basic conflict resolution methods can provide an edge in system design discussions during interviews or practical applications.(1290))