System design deep dive · HLD
Design Instagram: full high-level design.
Requirements, back-of-envelope capacity estimation, a high-level design diagram, a deeper deployment architecture diagram, sequence diagrams for uploading a post and generating a feed, and an entity-relationship diagram - with the async media-processing pipeline and hybrid feed fan-out worked through in detail.
1. Clarify requirements before drawing any box
Instagram is really two systems glued together: a media-processing pipeline that turns one uploaded photo into many delivery-ready variants, and a feed system that decides, for every user, which of those variants to show and in what order.
Functional requirements
Non-functional requirements
2. Back-of-the-envelope capacity estimation
The numbers that matter most here are storage per post (multiple image variants) and the multiplier between one upload and how many feed reads it eventually serves.
| Metric | Assumption | Resulting estimate |
|---|---|---|
| Posts uploaded (writes) | 100M posts/day, mostly photos, ~10% video | ~1,150 writes/sec average, ~6,000/sec peak |
| Feed reads | 500M DAU × ~10 feed page loads/day | ~58,000 reads/sec average, ~200,000/sec peak |
| Storage per photo post | Original ~4MB + 5 derived variants (~1.5MB total) | 100M/day × 5.5MB ≈ 550 TB/day of new media (before compression/tiering) |
| Processing throughput | 90M photos/day need resize + thumbnail + filters | ~1,050 jobs/sec average on the processing queue, needs autoscaled worker fleet |
| Feed cache footprint | Cache last ~500 post IDs per active user, ~80 bytes/entry | 500M users × 500 × 80B ≈ 20 TB across the Redis fleet |
3. High-level design (HLD)
The HLD separates "accept the upload and acknowledge fast" from "do the expensive work," and separates the write path (post) from the read path (feed) the same way a URL shortener separates create from redirect.
What each box owns
Post service (write path)
Issues a pre-signed upload URL so the client uploads the original directly to object storage (never proxied through the app tier), writes a post row in "processing" state, and enqueues a processing job. It returns to the client as soon as the row exists, well before any resized variant is ready.
Processing queue and workers
Pull jobs, download the original, and produce a fixed set of derived variants (thumbnail, feed-resolution, full-screen, story crop) plus a video transcode if applicable. Each variant is uploaded to object storage and pushed to the CDN origin; when all variants exist the post flips to "ready" and fan-out is triggered.
Feed service (read path)
Reads the requester's pre-computed feed cache (post IDs pushed there when followed accounts posted), hydrates post metadata and CDN URLs, and paginates. Like the timeline problem in other social feeds, very large accounts are excluded from push fan-out and merged in at read time instead.
Object storage, CDN, and follow graph
Object storage is the durable source of truth for both originals and derived variants; the CDN caches derived variants at the edge since they are immutable once generated - a post's image never changes without becoming a new object. The follow graph is a dedicated adjacency-list service shared by both the fan-out workers and the feed service.
4. Detailed architecture diagram
The architecture diagram makes the processing pipeline concrete - specific worker pools per media type - and shows how feed fan-out reuses the same push/pull split other feed-shaped systems need.
| Decision | Choice | Reasoning |
|---|---|---|
| Upload path | Direct-to-storage via pre-signed URL | Removes large binary payloads from the app tier entirely; app servers only ever handle small JSON requests. |
| Processing model | Async queue + autoscaled worker pool | Resize/transcode time (seconds) must never block the client's perceived "post uploaded" moment. |
| Feed fan-out | Hybrid: push for normal accounts, pull for >1M followers | Same amplification problem as any social feed - bounds worst-case fan-out cost per post. |
| Storage tiering | Hot tier recent, cold/archival for old originals | Originals are rarely re-processed after initial upload; keeping them all on hot storage indefinitely is unnecessary cost. |
5. Sequence diagrams for the two critical flows
These flows show exactly where "upload finishes" and "post is visible in feeds" diverge in time - a distinction that's easy to gloss over in prose but has to be explicit in a sequence diagram.
5.1 Upload and process a photo post
Step 3 uploads directly from client to storage, never through the Post service - the service only ever handles the small metadata request. Step 5 returns "processing," not "posted" - real apps show the post optimistically in the author's own profile immediately, while step 6/7 (seconds later, fully async) is what actually makes it appear in followers' feeds.
5.2 Load a home feed page
Step 4/5 fetch metadata (caption, like count, CDN URL) but never the image bytes themselves - the Feed service response is a small JSON payload. Step 7 shows the client fetching the actual image directly from the CDN in parallel, which is what keeps a feed page load fast regardless of how many photos are on it.
6. Entity-relationship (ER) diagram and schema
The data model separates the mutable social graph and engagement data from the largely immutable post record, and never stores media bytes in the relational layer at all - only CDN-addressable references.
Key modeling decisions
| Storage choice | Use when | Watch out for |
|---|---|---|
| Relational (sharded by author_id) for posts | You want straightforward "all posts by this author" queries for profile pages. | Cross-shard queries for a mixed feed require the feed cache to already hold the fan-out, not a live join. |
| Wide-column / key-value for likes | Like volume is extremely high and access is always by post_id or (post_id,user_id). | Aggregate "who liked this" listings at very high like counts need pagination, not a full scan. |
7. Deep dives interviewers actually probe
Why process media asynchronously instead of resizing on upload?
Resizing to five-plus variants and transcoding video can take several seconds, and doing it synchronously would tie up an app-tier request thread per upload and make the client wait through the slowest step (video transcode) before it even gets an acknowledgment. Making it async means the client gets a fast "accepted" response, the app tier stays lightweight, and the worker fleet can be scaled and retried independently of request traffic.
How is feed fan-out kept from being overwhelmed by an account with 50M followers?
Same problem as any social feed: push fan-out is used for typical accounts, but accounts above a follower-count threshold are excluded from push entirely and instead pulled at read time by the Feed service, merged with the pushed portion of the feed, and re-sorted. This bounds worst-case fan-out cost per post regardless of the author's audience size.
How do you avoid re-encoding the same video at multiple resolutions from scratch for every request?
Encoding happens exactly once, at upload time, producing a fixed bitrate ladder (e.g. 480p/720p/1080p) that is stored and cached at the CDN; playback never triggers a new encode. This trades upfront processing cost (paid once per upload) for near-zero marginal cost per view, which is the right trade-off given views vastly outnumber uploads.
What happens if a processing worker crashes mid-job?
The original was already durably written to object storage before the job was enqueued, so nothing is lost - the queue's visibility-timeout/retry mechanism redelivers the job to another worker after a timeout. Workers are also idempotent: re-running the resize step simply overwrites the same derived-variant object keys, so a retried job produces the same result rather than duplicate posts.
What's the single biggest bottleneck as this scales 10x?
Not the feed read path - caching and CDN delivery scale horizontally. The real pressure point is the media processing pipeline during traffic spikes (e.g. a major event driving a burst of video uploads), since video transcoding is CPU/GPU-bound and far more expensive per job than image resizing. The mitigation is prioritizing image jobs ahead of video jobs in the queue and autoscaling the GPU transcode pool independently, so a video backlog never delays photo posts from appearing in feeds.
Post a Comment
Add