Dropbox

System design deep dive · HLD

Design Dropbox (or Google Drive): full high-level design.

Requirements, back-of-envelope capacity estimation, a high-level design diagram, a deeper deployment architecture diagram, sequence diagrams for chunked upload with dedup and for multi-device sync, and an entity-relationship diagram for files, blocks, file versions, and devices - with the reasoning an interviewer expects behind every box and arrow.

Jump to HLD Architecture Sequence diagram ER diagram

4MBBlock chunk size

~35%Storage saved by dedup

<2sCross-device sync latency target

1. Clarify requirements before drawing any box

A file-sync system is fundamentally a distributed-consistency problem wearing a file-manager UI: the interesting engineering is entirely in how bytes are chunked, deduplicated, versioned, and reconciled across devices that were briefly offline - not in the upload button itself.

Functional requirements

Upload & store filesAny file type/size is uploaded, split into fixed-size blocks, and stored durably.

Sync across devicesA change on one device (add, edit, delete, rename) propagates to every other linked device.

Version historyEvery save creates a new file version; users can restore a prior version.

Conflict resolutionOffline edits on two devices to the same file are reconciled without silently discarding data.

Non-functional requirements

Bandwidth efficiencyA one-line edit to a huge file must not re-upload the whole file - only changed blocks.

Storage efficiencyIdentical content across users/files (common templates, duplicate uploads) should be stored once.

Eventual consistency across devicesEvery online device converges to the same file state within a few seconds of a change.

DurabilityA stored block must survive disk, node, and even full data-center failure.

Explicitly out of scope Real-time collaborative co-editing within a single document (a fundamentally different operational-transform/CRDT problem), sharing/permissions ACL UI, and virus scanning pipelines are called out as extensions rather than core requirements, so the core design stays focused on chunking, dedup, and sync.

2. Back-of-the-envelope capacity estimation

These numbers decide the block size trade-off, how much storage dedup realistically saves, and how many metadata writes the sync-notification path must sustain during a mass "everyone's back online Monday morning" wave.

Metric	Assumption	Resulting estimate
Active users	700 million registered, ~150 million daily active	Baseline for sync fan-out and metadata query volume
Average file size	Mixed workload, average ~2MB per file	4MB block size means most files are 1 block; larger files (video, ISOs) split into many
Daily file writes (saves)	~150M DAU × ~5 file saves/day	~750M file-version events/day ≈ 8,700 writes/sec average, ~40,000/sec peak (business-hours overlap)
Block-level dedup savings	Common OS files, templates, duplicate uploads across users	Empirically 30-40% of blocks already exist somewhere in the system - meaningful storage and upload-bandwidth savings
Storage growth	150M DAU × ~5 file saves/day × ~4MB average new/changed data	~3 PB/day of new unique block data before dedup and compression reduce it further

Why this matters Splitting files into fixed-size, content-hashed blocks is the one design decision that pays for both non-functional requirements at once: it makes incremental sync possible (only changed blocks re-upload) and makes cross-user deduplication possible (identical blocks, even across unrelated users' files, are only ever stored once).

3. High-level design (HLD)

The HLD separates the block storage layer (dumb, content-addressed, dedup-friendly) from the metadata layer (which file owns which blocks, in what order, at what version) from the sync/notification layer that tells other devices something changed.

Stateless servicesDedup / notification infraDurable block storageMetadata / async

What each box owns

Client chunking & dedup index

The client splits every file into fixed-size (e.g. 4MB) blocks and computes a content hash (SHA-256) per block. Before uploading a single byte, it asks the dedup index which of those hashes already exist in block storage - identical blocks (a duplicate file, a common template, an unmodified region of a large file re-saved) are simply referenced, never re-uploaded. This is the single mechanism that satisfies both the bandwidth-efficiency and storage-efficiency requirements at once.

Block storage & metadata DB

Block storage is a dumb, content-addressed blob store (key = block hash, value = bytes) - it has no concept of "files" at all. The metadata DB is what turns an ordered list of block hashes into a file: it stores, per file version, the ordered list of block references, so reconstructing any historical version is just resolving that version's block list.

Notification service & device registry

When device A commits a new file version, the sync service publishes a lightweight "file X changed, new version N" event; the notification service pushes (or the client long-polls for) that event to every other device registered for the same account. Devices then pull only the metadata diff and any new blocks they don't already have locally - never a full-file re-download.

Version & conflict resolver

Every commit is versioned, never overwritten in place. If two devices commit conflicting changes to the same file while offline from each other, the resolver detects the version-history fork (both claim the same parent version) and keeps both as a "conflicted copy" rather than silently picking one and discarding the other's edits.

4. Detailed architecture diagram

The architecture diagram shows block storage sharded by content hash (making dedup lookups and storage placement trivially parallel), metadata sharded by account, and the notification fan-out path kept fully asynchronous from the upload path.

Decision	Choice	Reasoning
Block size	Fixed 4MB chunks (not variable-size content-defined chunking)	Simpler to implement and reason about; the trade-off is that inserting a single byte near the start of a file shifts every subsequent fixed-size block's hash, which content-defined chunking would avoid at the cost of implementation complexity.
Block storage sharding	By content hash (consistent hashing)	Naturally distributes load evenly regardless of which files are popular, and makes dedup lookups a simple hash-to-shard routing decision with no hot spotting.
Deletion / garbage collection	Reference-counted, async GC worker, never immediate delete	A block can be referenced by many files across many users; only when its reference count reaches zero (a background sweep, not the delete request itself) is it safe to reclaim storage.
Sync transport	Long-lived connection (WebSocket/long-poll) per online device, not polling	Achieves near-real-time propagation (<2s) without the overhead and latency of every device polling on a fixed interval.

5. Sequence diagrams for the two critical flows

Upload shows exactly where dedup avoids redundant bytes on the wire; sync shows how a second device converges to the new state without ever re-downloading blocks it already has.

5.1 Chunked upload with block-level dedup

Step 2 is the entire point of the design: the client never uploads a byte until it knows whether the server already has that exact block, which is why re-saving a large file with one small change uploads only the changed blocks (step 4), not the whole file. Step 5 increments reference counts for all N blocks (both the M newly uploaded ones and the N-M already-existing ones this file now also references) - this is what makes later garbage collection safe.

5.2 Multi-device sync

Step 6 is where dedup pays off a second time: device B compares the incoming block list against what it already has cached locally (perhaps from an earlier version of the same file, or an unrelated file that happened to share a block) and only fetches the handful of genuinely new blocks - not a full re-download. If device B was offline when step 3 fired, it simply queries "give me all changes since my last known version" upon reconnecting, using the same metadata diff mechanism.

6. Entity-relationship (ER) diagram and schema

The schema has to answer: how is a file reconstructed from its blocks in the right order, how does version history avoid duplicating unchanged blocks, and how does the system know which devices are still owed a sync notification.

Key modeling decisions

block_hashes is an ordered reference list, not a join tableA JSON array of hashes in commit order is cheap to read/write per version and avoids a separate many-to-many table for what is an append-mostly, read-heavy relationship.

blocks.ref_count enables safe, async deletionDeleting a file_version decrements ref_count on its blocks; only a background GC sweep physically removes blocks whose count hits zero, so a race can never delete a still-referenced block.

file_versions.parent_ver_id detects conflictsIf two versions are committed with the same parent_ver_id from different devices, the resolver recognizes a fork and creates a "conflicted copy" instead of silently overwriting.

devices.last_synced_ver drives catch-up syncAn offline device reconnecting simply asks for every file_versions row newer than its own last_synced_ver per file, rather than the notification service tracking a queue per device.

Storage choice	Use when	Watch out for
Relational (Postgres/MySQL) for files, file_versions, devices	Version history and conflict detection benefit from transactional guarantees around parent-version checks.	file_versions can grow large per file; old versions need periodic archival to cold storage.
Key-value / content-addressed store for blocks	Access pattern is purely "get bytes by hash" - no relational structure needed for the blob itself.	Reference counting must be handled carefully under concurrent writers to avoid premature garbage collection.

7. Deep dives interviewers actually probe

Fixed-size chunking vs content-defined chunking - which and why?

Fixed 4MB chunking is simple, but has a well-known weakness: inserting a single byte at the start of a file shifts every subsequent block boundary, so every block hash downstream changes even though the actual content barely changed - dedup and incremental sync both fail in this case. Content-defined chunking (using a rolling hash like Rabin-Karp to pick block boundaries based on content, not fixed offsets) fixes this by re-synchronizing boundaries after an insertion, at the cost of variable block sizes and a more complex chunking algorithm. Many real systems start with fixed-size chunking for simplicity and only add content-defined chunking once the insertion-shift problem is shown to matter in practice.

How exactly does conflict resolution avoid silently losing data?

Every file_version commit includes the version it was based on (parent_ver_id). If device A commits version 5 based on version 4, and device B - having been offline - also commits a version based on version 4, the server sees two children of the same parent and cannot merge them automatically (it doesn't understand file content). Instead it accepts the first commit as the new current version and stores the second as a "Copy (conflicted, device B's edits, <timestamp>)" - both sets of edits survive, and the human resolves the conflict, exactly like Dropbox's real behavior.

How is storage reclaimed when a file or version is deleted?

Deleting a file (or pruning an old version past the retention window) never directly deletes blocks - it only decrements ref_count on the blocks that version referenced and marks the file_versions row deleted. A separate, low-priority background garbage collector periodically scans for blocks with ref_count = 0 and only then removes the bytes from block storage, which avoids a race where a concurrent read or a not-yet-committed reference to the same block gets deleted out from under it.

Why deduplicate at the block level instead of the whole-file level?

Whole-file dedup (hash the entire file, skip upload if the hash matches) only helps when two files are byte-identical - it does nothing for a 500MB video edited to add ten seconds, or a document with one paragraph changed. Block-level dedup captures both cases: identical whole files trivially dedup at the block level too, but partially-changed large files also get the benefit, uploading and storing only the handful of blocks that actually differ.

What is the single biggest bottleneck as this scales 10x?

Not block storage - it shards cleanly by content hash and scales near-linearly. The real bottleneck becomes the dedup index's hot-key problem for extremely common blocks (empty file blocks, common OS/template headers) that are referenced by an enormous fraction of all files - a single hash's ref_count row becomes a write hotspot under concurrent uploads. The fix is either sharding the ref-count updates across multiple counter cells per hash (sum-sharded counters) or making ref-count increments asynchronous and eventually consistent rather than synchronous on the upload path.

8. Summary: what a strong answer covers

Separated block storage from file metadataJustified every number with a calculationExplained fixed vs content-defined chunking Made conflicts explicit, never silent data lossUsed reference counting for safe garbage collectionCompared relational vs content-addressed storage honestly

Interview tip When asked to design Dropbox, the strongest signal is going deep on block-level chunking and content-addressed dedup as the foundation for everything else - incremental sync, storage efficiency, and version history are all just different views over the same "files are ordered lists of block hashes" idea, and articulating that connection clearly is what separates a strong answer from a shallow one.