System design deep dive · HLD
Design a Payment Processing System (Stripe-like): full high-level design.
Requirements, back-of-envelope capacity estimation, a high-level design diagram, a deeper deployment architecture diagram, sequence diagrams for the payment-creation and refund flows, and an entity-relationship diagram built around a double-entry ledger - with the reasoning an interviewer expects behind every box and arrow.
1. Clarify requirements before drawing any box
A payment system lives or dies on one property that most systems don't need: money must never be created, destroyed, or duplicated by a retry, a crash, or a race condition. Every requirement below is written with that constraint in mind.
Functional requirements
Non-functional requirements
2. Back-of-the-envelope capacity estimation
These numbers decide whether a single-region relational database with strong consistency is realistic (it is, at this scale), how big the idempotency key store needs to be, and how fast the ledger table grows.
| Metric | Assumption | Resulting estimate |
|---|---|---|
| Payments created | 10 million/day | ~116 TPS average, ~1,200 TPS peak (holiday sales, flash promos) |
| Ledger entries | Double-entry: 2 rows per successful payment leg, 2 more per refund | ~20-24M ledger rows/day |
| Storage per payment row | ~600 bytes (amount, currency, token ref, metadata, indexes) | 10M/day × 5 years × 600B ≈ 11 TB for payments alone |
| Idempotency key store | Keys retained 24h, ~10M new keys/day | ~10M concurrent keys × 200 bytes ≈ 2 GB - comfortably fits a Redis cluster |
| Refund volume | ~2% of payments are refunded | ~200K refunds/day, ~2.3 refunds/sec average |
3. High-level design (HLD)
The HLD names the major components and the one-directional data flow between them, without yet committing to region layout, replica counts, or sharding keys - that belongs in the architecture diagram in the next section.
What each box owns
Payment service
Receives the charge request, looks up the Idempotency-Key header in the idempotency store first. On a new key, it calls the card network to authorize/capture, then hands off to the ledger service. On a repeated key (client retry after a timeout), it returns the already-stored response verbatim without contacting the card network again - this single check is what prevents double charges.
Ledger service
Takes a successful (or reversed) payment and writes a balanced set of double-entry rows - a debit somewhere and an equal credit somewhere else - inside a single database transaction. It never exposes a way to write an unbalanced entry; the invariant "sum of all entries for a transaction = 0" is enforced at write time, not checked later.
Idempotency store
A Redis-backed (or DB-backed with a unique index) store keyed on idempotency_key, holding the request fingerprint and the final response for 24 hours. It is consulted synchronously and is on the critical path, but reads/writes are single-key operations, so it stays fast even under retry storms.
Card network, event bus, and webhook dispatcher
The card network (Visa/Mastercard rails via an acquiring processor) is the external source of truth for whether money actually moved between banks. Once the payment service and ledger service agree the local state changed, an event is published so the webhook dispatcher can notify the merchant asynchronously, with retries and backoff, without making the merchant's uptime a dependency of the charge API's response time.
4. Detailed architecture diagram
The architecture diagram takes every HLD box and answers "how is this actually deployed, and why not active-active?" - the single most interviewer-tested decision in a payments design.
| Decision | Choice | Reasoning |
|---|---|---|
| Region topology | Active-passive, single writer region | Money movement needs a single consistent order of operations per account; active-active would risk split-brain double-processing during a network partition, which is worse than a few minutes of failover downtime. |
| Ledger sharding key | Hash of account_id | Nearly every ledger read/write pattern ("show me this account's balance/history") is scoped to one account, so this key keeps double-entry transactions single-shard and ACID without cross-shard coordination. |
| Fraud/risk check placement | Synchronous, before card network call | Blocking on risk scoring adds latency but is cheaper than authorizing a fraudulent charge that then has to be manually reversed and disputed. |
| Webhook delivery | Async via Kafka + retry workers, never inline | A slow or down merchant endpoint must never hold open the charge API's HTTP connection or delay the ledger commit. |
5. Sequence diagrams for the two critical flows
A sequence diagram is where an interviewer checks whether the idempotency key is actually consulted before any external call, and whether a refund correctly reverses rather than deletes ledger history.
5.1 Create payment with an idempotency key
If the merchant's network call times out after step 4 and it retries the exact same request with the same key, step 3 instead returns the cached response from step 8, and steps 4-7 never happen again - the card is never charged twice for one logical intent, even though the HTTP request was sent twice.
5.2 Refund a payment
Step 5 never edits or deletes the original charge's ledger rows from step 6 of the create flow - it inserts new rows that debit and credit the opposite accounts, so the ledger stays append-only and the full history of "what happened and when" is preserved for audit and dispute purposes.
6. Entity-relationship (ER) diagram and schema
The data model has to answer three questions: how is a retried request detected before it reaches the card network, how is the double-entry invariant enforced by the schema rather than by hope, and how does a refund relate back to its original payment.
Key modeling decisions
| Storage choice | Use when | Watch out for |
|---|---|---|
| Relational (Postgres), ACID transactions | You need the debit and credit of a double-entry post to succeed or fail together, with a unique constraint on idempotency_key. | Sharding must be chosen so that a single payment's entries don't span shards, or you need a distributed transaction. |
| NoSQL (DynamoDB/Cassandra) | Almost never for the ledger itself - the lack of multi-row ACID transactions makes enforcing "debit and credit both commit" an application-level problem prone to partial-write bugs. | Can still be reasonable for read-only denormalized views (e.g. a per-account transaction history feed) built from the ledger via CDC. |
7. Deep dives interviewers actually probe
How does the idempotency key actually prevent a double charge?
The key is written to the idempotency store (or a unique DB column) inside the same transaction boundary as the initial "processing" state, before the card network is ever called. A concurrent or retried request with the same key either finds the row already in "processing" (and is told to wait/poll) or finds it "completed" (and gets the cached response immediately) - the card network call itself only ever happens once per key, by construction, not by a best-effort check.
-- Enforced by the schema, not application logic
CREATE UNIQUE INDEX ux_payments_idem_key ON payments(idempotency_key);
-- INSERT ... ON CONFLICT (idempotency_key) DO NOTHING RETURNING id;
-- if 0 rows returned: this key already exists, fetch and return the existing rowWhy a double-entry ledger instead of a single balance column?
A single mutable balance column can only ever tell you the current number, not how it got there, and a bug that increments the wrong row silently corrupts money with no trace. Double-entry gives a mechanical, schema-enforceable invariant - the sum of every entry tied to one logical transaction must equal zero - which turns most classes of accounting bugs into a failed assertion at write time instead of a silent discrepancy discovered during a customer dispute months later.
What happens if the card network approves the charge but the ledger write then fails?
The card network call and the ledger write cannot be wrapped in one distributed transaction, so the design treats the card network as external and idempotent-by-auth-id: if the ledger insert fails (deadlock, connection drop), the payment service retries the ledger write using the same idempotency key and the same auth_id it already has from the card network, rather than re-authorizing. The payment sits in a "pending_ledger" status until the ledger write succeeds, and a background reconciliation job also catches and repairs any payment that gets stuck in that status past a timeout.
How do you shard the ledger without breaking the double-entry invariant across accounts?
Sharding by account_id means a payment that moves money between a customer account and a platform/merchant account could, in principle, touch two different shards. The practical fix used by most payment systems is to keep a lightweight "platform revenue" account replicated into every shard as a local reference point, so the two entries of a given payment's core debit/credit pair land in the same shard as the customer or merchant account that owns them, and only asynchronous, eventually-consistent rollups (not the atomic entry pair) cross shard boundaries.
Why active-passive across regions instead of active-active?
Active-active would mean two regions could each accept and process a charge for the same idempotency key during a network partition, and reconciling two independently-committed ledger histories after the fact is close to impossible to do safely for real money. Active-passive with a synchronous (or tightly-bounded async) replica accepts a few minutes of unavailability during failover in exchange for a guarantee that there is always exactly one writer deciding the order of operations for any given account.
How do you catch bugs that silently break the ledger over time?
A nightly reconciliation batch job sums every account's ledger_entries and compares the result against the card network's settlement report and against the cached balance column; any mismatch pages the on-call engineer rather than being caught only when a customer complains. This is treated as a first-class part of the design, not an afterthought, precisely because the ledger schema prevents intra-transaction bugs but not cross-system drift against the external card network.
Post a Comment
Add