System design deep dive · HLD
Design Twitter / X: full high-level design.
Requirements, back-of-envelope capacity estimation, a high-level design diagram, a deeper deployment architecture diagram, sequence diagrams for posting a tweet and reading a home timeline, and an entity-relationship diagram - with the fan-out-on-write vs fan-out-on-read trade-off and the celebrity-account problem worked through in detail.
1. Clarify requirements before drawing any box
Twitter is a fan-out problem wearing a social-network costume: the hard part is not storing a 280-character tweet, it's delivering it to the right set of home timelines fast enough, at a scale where some accounts have hundreds of millions of followers and others have a dozen.
Functional requirements
Non-functional requirements
2. Back-of-the-envelope capacity estimation
The number that shapes this entire design is not tweets/sec - it's how many timeline-cache writes one tweet can trigger. That single multiplier is why fan-out can't be a single uniform strategy.
| Metric | Assumption | Resulting estimate |
|---|---|---|
| Tweets posted (writes) | 500M tweets/day across 250M DAU | ~5,800 writes/sec average, ~20,000/sec peak (breaking news) |
| Home timeline reads | 250M DAU × ~20 feed refreshes/day | ~58,000 reads/sec average, ~250,000/sec peak |
| Naive fan-out amplification | Mean ~200 followers/account (heavily skewed by celebrities) | 500M tweets/day × 200 ≈ 100B potential cache writes/day if every tweet fanned out to every follower |
| Tweet storage | ~300 bytes/tweet (text + metadata, media stored separately) | 500M/day × 300B × 5yr ≈ 274 TB of tweet rows |
| Timeline cache footprint | Cache last ~800 tweet IDs per active user, ~100 bytes/entry | 250M users × 800 × 100B ≈ 20 TB across the Redis fleet |
3. High-level design (HLD)
The HLD splits the write path (post a tweet, fan it out) from the read path (assemble a home timeline), and introduces the follow graph as a service both paths depend on but neither owns.
What each box owns
Tweet service (write path)
Validates and persists the tweet to the Tweet store, then enqueues a single fan-out job onto the Kafka topic and returns to the caller immediately - it never waits for fan-out to complete. This is what keeps "post a tweet" fast regardless of the author's follower count.
Fan-out workers + queue
Consume fan-out jobs, look up the author's followers from the Follow graph, and push the new tweet ID onto each follower's timeline cache list - but only for authors under a follower-count threshold (see the deep dive on the celebrity problem below). For accounts above the threshold, no push happens at all.
Timeline service (read path)
Reads the requester's pre-computed timeline cache (already-fanned-out tweet IDs), separately fetches recent tweets from any celebrities the user follows (fan-out-on-read), merges and re-sorts the two lists by time, then hydrates full tweet objects for the final page. This hybrid merge is the crux of the whole design.
Follow graph and Tweet store
The Follow graph is a dedicated adjacency-list service (who-follows-whom, who-does-X-follow) since both fan-out and celebrity-pull need fast graph lookups it shouldn't share a database with tweet content. The Tweet store is a wide-column store (Cassandra-style) partitioned by tweet ID, since tweets are immutable and accessed by ID or author, never by complex query.
4. Detailed architecture diagram
The architecture diagram is where the celebrity problem becomes concrete: a threshold check that routes two categories of accounts down structurally different fan-out paths, plus the replica and partitioning counts an interviewer expects to see justified.
| Decision | Choice | Reasoning |
|---|---|---|
| Fan-out strategy | Hybrid: push for <1M followers, pull for ≥1M | Bounds worst-case fan-out writes per tweet to a manageable number while keeping the common-case read cheap (pre-computed). |
| Follow graph as its own service | Separate from Tweet store | Both the write path (who to fan out to) and read path (who am I following) need low-latency graph lookups independent of tweet volume. |
| Tweet store partitioning | Partition by tweet_id | Tweets are immutable and always fetched by ID during hydration; no need to co-locate by author. |
| Engagement counters | Async batched increments via Kafka | A like/retweet burst on a viral tweet must never turn into a hot-row lock contention problem on the primary tweet record. |
5. Sequence diagrams for the two critical flows
These two flows are exactly where the fan-out-on-write vs fan-out-on-read trade-off has to be shown as call order, not just described in prose.
5.1 Post a tweet (fan-out-on-write, normal account)
Step 5 returns to the author before fan-out has even started - the client never waits on it. Step 6/7 run entirely asynchronously, seconds after the response was sent, and are the exact place the celebrity threshold check happens: for a huge account this step is skipped altogether, and those tweets are picked up at read time instead.
5.2 Read a home timeline (hybrid push + pull merge)
Step 2/3 is the fast, common case - most of the timeline was already pushed there when it was posted. Steps 4/5 only add latency proportional to how many celebrities the requester follows, which is small and bounded, unlike the alternative of pulling from every followed account. Step 6 is where fan-out-on-write and fan-out-on-read results are reconciled into one ordered page.
6. Entity-relationship (ER) diagram and schema
The data model has to answer: where does a tweet live, how is an asymmetric follow relationship stored so both directions can be queried, and what exactly does the timeline cache hold (never full tweet bodies).
Key modeling decisions
| Storage choice | Use when | Watch out for |
|---|---|---|
| Wide-column (Cassandra) for tweets | Access pattern is get-by-id and get-by-author at extreme write volume. | No multi-row transactions; fan-out and counters must be handled as separate, eventually-consistent writes. |
| Graph-oriented store for follows | You expect frequent "mutual follows" or "suggested accounts" queries later. | Adds an extra system to operate; a well-indexed relational table is often enough for pure follow/unfollow. |
7. Deep dives interviewers actually probe
Why not just fan out every tweet to every follower, always?
Because follower count is a power-law distribution: most accounts have a few hundred followers, where fan-out-on-write is cheap and gives near-instant timeline reads, but a small number of accounts have tens of millions. Fanning out one tweet from a 100M-follower account would mean 100M cache writes in a burst, overwhelming the fan-out workers and the cache cluster at the exact moment the tweet matters most. The hybrid design pushes for normal accounts and pulls for celebrities at read time, capping worst-case fan-out cost per tweet regardless of who posted it.
What happens to a follower's cached timeline when they follow someone new?
The new followee's recent tweets are not retroactively injected into the follower's existing cache - that would require scanning the followee's whole history. Instead, the follower's cache simply starts receiving future pushes from that account (if non-celebrity) or gets that account included in the read-time pull set (if celebrity) going forward, and the client is expected to backfill a small amount of the followee's recent history via a direct "load their profile" call the first time.
How do you keep like/retweet counters accurate without hot-row contention?
Every like or retweet publishes an event to Kafka rather than issuing a synchronous UPDATE tweets SET like_count = like_count + 1. A stream aggregator batches events per tweet over a short window (e.g. 500ms) and applies one combined increment, so a viral tweet receiving thousands of likes per second produces orders of magnitude fewer writes to the underlying row, and the counter is allowed to be a few hundred milliseconds stale in exchange for not serializing on a single row.
How does deleting a tweet propagate through a system with pre-computed caches?
Because timeline_cache stores only tweet IDs, a delete only needs to write a tombstone in the Tweet store; every timeline that still references the deleted ID simply omits it at hydration time (the hydrate step filters out IDs the Tweet store returns as deleted or not-found). This is exactly why the cache was designed to hold IDs instead of denormalized tweet content in the first place - it avoids having to hunt down and rewrite every fanned-out copy.
What's the single biggest bottleneck as this scales 10x?
Not the read path - the hybrid fan-out already bounds per-tweet cost. The real risk is the Follow graph service during a mass event (e.g. a huge account gaining millions of new followers in hours, or many normal accounts posting during breaking news simultaneously, each triggering their own fan-out fan-out job). The mitigation is the same pattern used elsewhere in the design: keep the graph lookups cheap and cacheable, and let the fan-out queue's consumer group scale out horizontally to absorb backlog rather than trying to make any single write synchronous end-to-end.
Post a Comment
Add