System design deep dive · HLD
Design a real-time chat system (WhatsApp / Slack style): full high-level design.
Requirements, back-of-envelope capacity estimation, a high-level design diagram, a deeper deployment architecture diagram, sequence diagrams for message send/fan-out and reconnect/sync, and an entity-relationship diagram for the data model - with the reasoning an interviewer expects behind every box and arrow.
1. Clarify requirements before drawing any box
A chat system's hardest constraints are latency and ordering, not raw storage volume, so requirements gathering should explicitly separate "must feel instant" from "must never reorder."
Functional requirements
Non-functional requirements
2. Back-of-the-envelope capacity estimation
These numbers decide the two hardest constraints: how many stateful gateway nodes are needed to hold every connection open, and whether a single Kafka partitioning scheme can preserve ordering at this throughput.
| Metric | Assumption | Resulting estimate |
|---|---|---|
| Daily active users | 200 million DAU, ~40 messages sent/user/day | ~8 billion messages/day |
| Message throughput | 8B/day spread over the day, 3x peak multiplier | ~93,000 msgs/sec average, ~280,000 msgs/sec peak |
| Concurrent connections | ~20% of DAU online at any instant | ~40 million concurrent WebSocket connections |
| Gateway fleet size | ~50,000 stable WS connections per gateway node | 40M / 50K ≈ 800 gateway nodes |
| Storage per message | ~250 bytes (text + sender/conversation/timestamp metadata) | 8B/day × 250B ≈ 2 TB/day raw before compaction |
| Group fan-out load | 15% of messages sent in groups, avg group size 8 | adds ~1M extra fan-out writes/sec at peak for large/broadcast groups |
3. High-level design (HLD)
The HLD names the major components and the one-directional data flow between them for a single message send, without yet committing to region counts, partition counts, or replica counts.
What each box owns
WebSocket gateway
Terminates one long-lived, authenticated WebSocket per client device and is deliberately stateful and sticky: a client stays pinned to the same gateway node for the life of the connection. It forwards inbound messages to the Message service and pushes outbound messages to any client connected to it, but holds no business logic of its own.
Connection registry
A Redis-backed map from (user_id, device_id) to the specific gateway node holding that socket. Any gateway node that needs to deliver a message to a user looks up this registry first - this is what makes the gateway fleet horizontally scalable without every node needing to know about every connection.
Message service
Assigns each message a monotonically increasing per-conversation sequence number, validates the sender is a conversation member, writes the message onto the Kafka partition owned by that conversation, and triggers fan-out to online recipients (via the registry) and to the push service for offline recipients - all before the message is guaranteed durable, which is why persistence happens off Kafka rather than synchronously in this service.
Kafka, message store, and push fallback
Kafka is partitioned by conversation_id specifically so all messages for one conversation land on one partition and are consumed in write order. A consumer persists them into Cassandra (the durable message store) and updates each recipient's unread counters. If a recipient has no active WebSocket connection, the push service sends a native mobile push instead, and the message waits in the store for the next sync.
4. Detailed architecture diagram
The architecture diagram answers how connection stickiness, partition ownership, and cross-region delivery actually work in production - the details an interviewer probes once the HLD shape is accepted.
| Decision | Choice | Reasoning |
|---|---|---|
| Connection stickiness | Sticky WebSocket per gateway node, tracked in a Redis registry | WebSockets are stateful by nature; a registry lets any node route a message to the exact node holding the recipient's socket without broadcast. |
| Kafka partitioning key | conversation_id, not user_id or random | Guarantees every message in one conversation is consumed in the order it was produced - the core ordering guarantee - at the cost of one hot partition for very large groups. |
| Conversation "home region" | Each conversation's partition lives in one region; the other region mirrors it | Avoids split-brain ordering across regions; cross-region participants pay one extra hop, which is acceptable given the <200ms budget. |
| Idempotent persistence | Writer workers key writes by client-generated message UUID | At-least-once delivery from Kafka means the same message can be reprocessed after a consumer restart; the UUID makes the write a safe no-op on retry. |
5. Sequence diagrams for the two critical flows
A sequence diagram is where an interviewer checks whether you understand what's synchronous on the critical path versus fire-and-forget, and how a recipient who is offline still ends up caught up.
5.1 Send a message and fan it out (recipient online)
Step 6 - the "sent" acknowledgment to the sender - happens the moment Kafka commits the offset in step 4, not after the recipient has actually received anything. Step 7 is deliberately drawn as a dashed, decoupled fan-out: it happens after the consumer reads the committed message, so a slow or disconnected recipient never adds latency to the sender's "sent" status.
5.2 Reconnect, presence update, and history sync
Step 4's watermark - the highest per-conversation seq_no the client last saw - is exactly what makes sync correct instead of guessing based on a timestamp: clocks drift and can duplicate or skip messages near a boundary, while a monotonic sequence number cannot. Step 7 is fire-and-forget for the same reason the write path is: a slow presence fan-out to hundreds of contacts must never delay the reconnecting client from seeing its own missed messages.
6. Entity-relationship (ER) diagram and schema
The data model has to answer: how are messages fetched in order for one conversation without a full scan, how does group membership stay cheap to check, and where does presence live given it churns far more often than anything else.
Key modeling decisions
| Storage choice | Use when | Watch out for |
|---|---|---|
| Wide-column (Cassandra/HBase), partitioned by conv_id | Access pattern is almost entirely "get ordered messages for one conversation" at very high write volume. | Very large or "celebrity" broadcast groups create a hot partition; needs bucketing (e.g. by conv_id + day) to spread load. |
| Relational (Postgres), sharded by conv_id | You want transactional guarantees around membership changes and simpler operational tooling at moderate scale. | Resharding and cross-shard membership queries are heavier than the wide-column option once volume is very large. |
7. Deep dives interviewers actually probe
How do you guarantee message ordering across multiple gateway nodes?
Ordering is not enforced by the gateway at all - it's enforced by routing every message for a given conversation through the same Kafka partition (keyed by conversation_id) and having the Message service assign a strictly increasing seq_no before producing. Two senders in the same group typing at the same instant simply race for the next seq_no; whichever's produce call lands first wins that slot, and every client renders in seq_no order, so the observed order is consistent for everyone even though it was decided by arrival order, not wall-clock time.
At-least-once delivery: how do clients avoid rendering duplicates?
The client generates a UUID for each message before it is ever sent, and that UUID travels with the message through Kafka into the message store's unique constraint. If the client's network blips right after sending and it retries the same message, the server-side write is a no-op and the original delivery still happens exactly once from the client's point of view - correctness is pushed to idempotent writes rather than trying to prevent retries.
How does a 5,000-member broadcast group avoid melting the fan-out path?
Small groups (the common case) use fan-out-on-write: the consumer looks up every member's gateway node and pushes directly. Past a configurable member-count threshold, the system switches to fan-out-on-read: the message is written once, and each member's client pulls it on its own next sync/poll instead of the server pushing to thousands of sockets synchronously - trading slightly higher perceived latency for a bounded, predictable write amplification.
What happens when a recipient is offline?
The connection registry lookup returns "no active gateway," so the consumer instead calls the push service (APNs/FCM), which delivers a native notification without message content beyond a preview. The message itself waits in the durable store; when the device reconnects, the reconnect/sync flow (5.2) fetches everything since last_seen_seq_no, so push is purely a "wake the user up" signal, never the source of truth for content.
What's the operational risk of a reconnection storm - e.g. after a regional network blip?
Tens of millions of clients reconnecting within seconds all hit the connection auth and registry-write path at once, which can look like a self-inflicted DDoS. Mitigations: client-side reconnect jitter/exponential backoff, connection auth tokens that are cheap to verify (short-lived signed JWTs, not a DB round-trip per connect), and registry writes batched or rate-limited per gateway node rather than one write per socket per second.
Post a Comment
Add