System design deep dive · HLD
Design a Digital Wallet / E-Wallet System: full high-level design.
Requirements, back-of-envelope capacity estimation, a high-level design diagram, a deeper deployment architecture diagram, sequence diagrams for a peer-to-peer transfer and a card top-up, and an entity-relationship diagram for a double-entry ledger - with the reasoning an interviewer expects behind every box, arrow, and dollar amount.
1. Clarify requirements before drawing any box
A digital wallet is a ledger system wearing a friendly UI - the moment real money is involved, "roughly correct" is not an option. Scope has to separate what must happen from how fast and how safely it must happen, because the safety requirements are what shape every later diagram.
Functional requirements
Non-functional requirements
2. Back-of-the-envelope capacity estimation
These numbers decide whether one Postgres primary can carry the write path, how much the ledger grows under a multi-year regulatory retention requirement, and how much headroom the design needs above steady-state for a flash-sale-style spike.
| Metric | Assumption | Resulting estimate |
|---|---|---|
| Active wallets | 50 million monthly active wallets | Baseline population for every estimate below |
| Transactions per wallet | ~15 transfers + top-ups / wallet / month | 50M × 15 = 750 million transactions/month |
| Throughput | 750M/month over ~2.59M seconds | ~289 TPS average, ~1,750 TPS at normal daily peak (~6x avg) |
| Ledger volume (double-entry) | 2 ledger_entries rows per transaction | 750M × 2 = 1.5 billion rows/month (~18 billion/year) |
| Storage per ledger row | ~150 bytes, fixed-width (ids, type, amount, balance_after, timestamp) | 18B/year × 150B ≈ 2.7 TB/year raw |
| 7-year retention | Financial audit trails typically must be retained 7 years | 2.7TB × 7 ≈ 19 TB raw, ~25 TB including indexes/replicas |
| Flash-sale peak TPS | ~5x the normal peak during a high-traffic promo/sale event | ~9,000 TPS - sizes the transactional DB's write throughput and connection-pool headroom |
3. High-level design (HLD)
The HLD names the major components and the direction money and data flow between them, without yet committing to replica counts, regions, or which specific queueing technology carries the async fraud escalation - that belongs in the architecture diagram next.
What each box owns
API gateway
Terminates client requests and enforces the idempotency contract at the edge: every mutating request carries a client-generated idempotency key, and the gateway checks a key-value store before letting the request reach the wallet service. A duplicate key returns the previously cached result instead of reprocessing - the request never even reaches the fraud engine twice.
Wallet service
The orchestrator for every balance-affecting operation. It validates the sender has sufficient balance, calls the fraud/velocity engine synchronously, and then opens the single database transaction that debits one wallet, credits another, and writes the matching ledger_entries rows - or rolls all of it back if any step fails.
Ledger service
Owns the append-only ledger_entries table - the actual source of truth for every dollar in the system. It never updates or deletes a row; corrections happen by inserting new offsetting entries, never by editing history, which is what makes the ledger auditable after the fact.
Wallets table (balance cache)
Holds a denormalized current-balance column so reads (checking "can I afford this transfer?", showing a balance in the app) are a single indexed lookup instead of summing potentially thousands of ledger rows. It is a cache in the strict sense: it must always be reconcilable against the ledger, never authoritative on its own.
Fraud / velocity check service
A fast, rules-based engine that checks a small recent-activity window per wallet (e.g. "more than $2,000 or 5 transfers in the last hour") using cached counters, fast enough to run inline on every transfer. Anything borderline is also queued for a heavier, asynchronous ML risk-scoring pass that can flag a transaction for hold or reversal after the fact.
Payment gateway integration
The only external dependency in the design, called strictly for top-up funding (never for P2P transfers, which stay fully internal). Charging a card or bank account takes real time, so this call is asynchronous end-to-end: the gateway confirms settlement later via a webhook rather than in the original request/response cycle.
Reconciliation job
A periodic batch job that recomputes every wallet's balance by summing its ledger_entries and compares that to the cached balance column, alerting on any mismatch. It is the safety net that catches bugs, race conditions, or partial failures the atomic transaction design is supposed to prevent - and it always treats the ledger as correct when reconciling a drift.
4. Detailed architecture diagram
The architecture diagram answers the questions an interviewer actually probes: where does the idempotency key live, what exactly is "atomic" here, what happens to a flagged transaction, and why does the reconciliation job never touch the primary database.
| Decision | Choice | Reasoning |
|---|---|---|
| Transfer atomicity | Single local DB transaction, not a distributed saga | As long as both wallets are co-located/co-partitioned in the same database, one ACID transaction covers the debit, credit, and both ledger rows. A saga with compensating actions only becomes necessary once wallets are sharded across separate databases. |
| Balance storage | Denormalized cached column, not summed live from the ledger | Summing potentially thousands of ledger rows on every balance check is too slow for the read path; a periodic reconciliation job keeps the cache honest instead. |
| Idempotency key storage | Redis with a short TTL (chosen), vs a dedicated idempotency table in the same DB transaction | Redis is fast and simple and matches the retry window clients actually need; an in-transaction idempotency table gives stronger guarantees (survives a Redis outage) at the cost of coupling every write to the same DB, which is the fallback if Redis-level guarantees prove insufficient. |
5. Sequence diagrams for the two critical flows
A sequence diagram is where an interviewer checks whether you actually understand call order, what is synchronous versus fire-and-forget, and exactly where a failure means nothing gets written at all.
5.1 Peer-to-peer transfer, with idempotency and fraud check
If step 3 instead finds a HIT, the gateway returns the cached prior result immediately and skips steps 4-8 entirely - a retried request never re-runs the fraud check or re-moves money. On the failure path, if step 5 comes back FAIL, or the balance check inside step 6 finds insufficient funds, the database transaction in step 6 is either never opened or is rolled back before COMMIT: no wallet row and no ledger row is ever written, and the client receives an error response instead of step 9.
5.2 Wallet top-up via external payment gateway (async settlement)
Real payment rails cannot confirm settlement synchronously, so steps 1-4 deliberately return a "pending" response without waiting - the client is told the top-up was accepted, not that funds have landed. Step 6 keeps the ledger's double-entry invariant even for external money entering the system: the credit to the user's wallet is balanced by a debit against an internal "gateway settlement" clearing account, so debits still equal credits system-wide, and step 8 is the only place the client learns the money actually arrived.
6. Entity-relationship (ER) diagram and schema
The data model has to answer three questions: what proves a transfer's two sides always balance, what makes a retried request safe, and how a wallet's balance can always be reconstructed from history alone.
Key modeling decisions
| Storage choice | Use when | Watch out for |
|---|---|---|
| Relational (PostgreSQL), ACID | You need a single multi-row transaction to guarantee the debit, credit, and both ledger rows commit or fail together. | Vertical scaling limits eventually require sharding wallets, at which point cross-shard transfers need a saga. |
| NoSQL (DynamoDB/Cassandra) | Extreme horizontal read/write scale is the priority and per-item operations are acceptable. | No native multi-row ACID transaction across two wallets - the atomic debit+credit invariant has to be rebuilt at the application layer, which is exactly the guarantee a ledger cannot afford to get wrong. |
7. Deep dives interviewers actually probe
Why double-entry bookkeeping instead of just incrementing/decrementing a balance column?
A single mutable balance column has no memory of how it got there - a bug or a race condition can silently corrupt it and nothing detects that. Double-entry means every transaction writes a balanced debit+credit pair, so the ledger is self-auditing: summing all ledger_entries for any wallet always reconstructs its true balance independent of the cached column. Reconciliation can then compare the two and catch drift instead of trusting a number that could have quietly drifted for months.
How do you guarantee atomicity for a transfer touching two different wallets without a distributed transaction?
Keep both wallets' rows in the same database (the same shard), so a single local ACID transaction covers the debit, the credit, and both ledger_entries inserts - if anything fails, the whole transaction rolls back and nothing partial is ever visible. The moment wallets are sharded across separate databases, this stops being free: you either introduce a saga with compensating actions (debit here, then credit there, with an explicit reversal step if the second half fails) or design a partitioning scheme that co-locates wallets that transact with each other frequently.
BEGIN;
SELECT balance FROM wallets WHERE id = :from_id FOR UPDATE;
-- application checks balance >= amount, else ROLLBACK
UPDATE wallets SET balance = balance - :amount WHERE id = :from_id;
UPDATE wallets SET balance = balance + :amount WHERE id = :to_id;
INSERT INTO ledger_entries (transaction_id, wallet_id, entry_type, amount, balance_after)
VALUES (:txn_id, :from_id, 'debit', :amount, :from_balance_after);
INSERT INTO ledger_entries (transaction_id, wallet_id, entry_type, amount, balance_after)
VALUES (:txn_id, :to_id, 'credit', :amount, :to_balance_after);
COMMIT;How do you make a transfer request safe to retry after a client-side timeout, given the server might have actually succeeded?
The client generates an idempotency key once and sends it with every retry of the same logical request. The API gateway checks that key against a store before doing any work; if the key has been seen before, it returns the original cached result instead of re-executing the transfer. A unique constraint on transactions.idempotency_key backs this up at the database level in case the cache is ever bypassed or evicted early - a duplicate insert simply fails rather than moving money twice.
How does the velocity/fraud check stay fast enough to run inline on every transfer?
The inline check is deliberately cheap: a rules engine reading a handful of cached counters (recent transaction count and total amount per wallet over a rolling window, kept in Redis) rather than invoking a full ML model synchronously. That keeps it well under the latency budget of the surrounding request. Genuinely borderline or high-risk transactions are still allowed through immediately but are also pushed onto a queue for a heavier, asynchronous ML scoring pass; if that later scoring comes back bad, the system places a hold on the wallet or reverses the transaction rather than blocking every legitimate transfer with a slow model in the hot path.
How do you detect and recover if a bug or partial failure ever causes a wallet's cached balance to drift from its ledger total?
A periodic reconciliation job runs against a read replica (never the primary, so it never competes with live traffic) and, for every wallet, recomputes the balance as the sum of its ledger_entries and compares it to the cached balance column. Any mismatch pages an on-call engineer rather than silently correcting itself, because a drift usually means a bug is actively running; when a correction is made, the ledger is always treated as the source of truth and the cached column is the one that gets fixed, never the other way around.
Post a Comment
Add