System design deep dive · HLD
Design an Online Code Judge (LeetCode / Codeforces style): full high-level design.
Requirements, back-of-envelope capacity estimation, a high-level design diagram, a deeper deployment architecture diagram, sequence diagrams for submission execution and contest-time verdict aggregation, and an entity-relationship diagram for the data model - with the reasoning an interviewer expects behind every box and arrow.
1. Clarify requirements before drawing any box
A strong system design answer starts by pinning down scope out loud. For an online code judge, that means separating "run untrusted code" from "run untrusted code safely, fairly, and reproducibly at contest scale" - sandboxing and resource limiting are the actual crux of the problem, not compiling code.
Functional requirements
Non-functional requirements
2. Back-of-the-envelope capacity estimation
These numbers decide how many sandbox worker containers must be warm and ready, how test-case data should be distributed to workers, and whether a single queue is enough or partitioning by contest is required.
| Metric | Assumption | Resulting estimate |
|---|---|---|
| Daily submissions | 2 million/day across practice + contests | ~23 submissions/sec average |
| Contest burst | 50,000 contestants, 40% submit in the last 2 minutes of a round | ~20,000 submissions in 120s ≈ 170/sec sustained, spiking past 400/sec |
| Execution cost per submission | ~15 hidden test cases × ~200ms each (compile + run) | ~3s of sandbox time per submission × 400/sec peak ≈ 1,200 concurrent sandbox slots needed |
| Storage per submission record | ~2 KB (source code, per-test results, verdict, timing) | 2M/day × 2KB × 3yr retention ≈ 4.4 TB |
| Test-case data per problem | ~5 MB average (input/output files across difficulty tiers) | 50,000 problems × 5MB ≈ 250 GB - fits object storage with a hot-tier cache |
3. High-level design (HLD)
The HLD names the major components and the one-directional data flow between them, without committing yet to specific infrastructure, regions, or replica counts - that level of detail belongs in the architecture diagram in the next section.
What each box owns
Submission service (write path)
Accepts the source code and language, does a cheap static check (size limit, disallowed syscalls if statically detectable, basic syntax sanity), writes a submission row with status PENDING, and enqueues a judge job. It never executes code itself - keeping the public-facing API surface free of anything that touches the sandbox pool directly.
Judge dispatcher and sandbox pool
The dispatcher pulls jobs from the appropriate queue (contest-isolated or general practice pool), assigns each to a pre-warmed sandbox container, and streams the compiled binary plus each hidden test case's input in turn, enforcing CPU time, wall-clock time, and memory limits per test via cgroups and a gVisor (or Firecracker) isolation boundary. Sandboxes are single-use and destroyed after judging one submission, so no state or side effect can leak between submissions even if the code being judged is actively malicious.
Problem/test store
Holds problem statements, per-test input/output files, and per-problem resource limits, kept strictly separate from the submissions database - a hidden test case leaking into submission-facing storage would compromise every contest using that problem. Test data is content-addressed and cached at the sandbox-host layer so repeated judging of the same problem doesn't re-fetch large test files from object storage each time.
Submissions DB, per-contest queues, and leaderboard cache
The submissions database is the durable source of truth for verdicts and is written once judging completes. Per-contest queues exist specifically so that one contest's last-two-minutes submission storm cannot delay another contest's or practice mode's turnaround time. The leaderboard cache is updated asynchronously from verdicts so a burst of scoring recalculation never blocks the judging pipeline itself.
4. Detailed architecture diagram
The architecture diagram takes every HLD box and answers "how is this actually deployed?" - replica counts, sandbox host pools, regions, and the specific technology choice, which is what an interviewer is checking for once they've accepted the high-level shape.
| Decision | Choice | Reasoning |
|---|---|---|
| Sandbox isolation technology | gVisor (user-space kernel) or Firecracker microVMs, not bare Docker | A plain container shares the host kernel; untrusted code with a kernel exploit could escape. gVisor/Firecracker add a real isolation boundary between the sandbox and the host kernel. |
| Queue partitioning | Per-contest isolated queues, not one global FIFO | A 50,000-contestant last-minute rush must not delay verdicts for an unrelated practice-mode user or a different concurrent contest. |
| Container lifecycle | Pre-warmed pool, single-use, destroyed after one submission | Cold-starting a container per submission adds hundreds of milliseconds of unacceptable latency; reusing a container across submissions risks state leakage between untrusted code runs. |
| Test data distribution | Content-addressed, host-local cache in front of object storage | The same problem's test cases are fetched repeatedly across thousands of submissions; caching by content hash avoids re-fetching immutable data on every judge run. |
5. Sequence diagrams for the two critical flows
A sequence diagram is where an interviewer checks whether you actually understand call order, what is synchronous versus fire-and-forget, and exactly how a resource-limit violation short-circuits the rest of the test run.
5.1 Code submission and sandboxed execution
Step 3 returns immediately with a submission ID rather than blocking on execution, since judging can take several seconds; the client polls or subscribes for the verdict separately. Step 7 shows the judge stopping at the first failing test in this example (a common mode for practice problems to save compute), though many judges instead run all N tests to completion and report full per-test detail - a real design trade-off worth naming out loud between "fail fast" and "full diagnostic output."
5.2 Contest-time verdict aggregation under load
Step 3's async publish is what keeps the judging pipeline itself from ever being slowed down by leaderboard-update volume - the sandbox pool's job is only to produce a verdict, never to wait on standings math. Step 8's explicit "~2s staleness under burst" is a deliberate trade-off: contestants briefly see a slightly stale leaderboard during the heaviest submission spike rather than the judging pipeline itself backing up to keep the leaderboard perfectly real-time.
6. Entity-relationship (ER) diagram and schema
The data model has to answer three questions: where do hidden test cases live so they can never leak to a client, how is a submission's per-test detail stored without one giant JSON blob, and how does the schema support both "practice" and "contest, ranked by time" scoring modes.
Key modeling decisions
| Storage choice | Use when | Watch out for |
|---|---|---|
| Relational (Postgres), sharded by user_id | You need range queries ("all my submissions"), joins for leaderboards, and transactional verdict writes. | Cross-shard contest-wide leaderboard queries need a separate read path (the Redis sorted-set cache) rather than hitting the shards directly. |
| Document/wide-column for problems + test_cases | Problem statements and test data are read far more often than written and rarely need relational joins. | Strict separation from submissions storage must still be enforced at the access-control layer, not just by data-modeling convention. |
7. Deep dives interviewers actually probe
How do you sandbox untrusted code safely - containers vs microVMs?
A plain Docker container shares the host's Linux kernel, so a kernel-level exploit in the submitted code could escape to the host or to other tenants' containers. gVisor intercepts syscalls in a user-space kernel shim, and Firecracker runs each sandbox in a lightweight microVM with its own kernel - both add a real isolation boundary at the cost of a few milliseconds of overhead per syscall, which is an acceptable trade for the security guarantee at judge scale. Network access is disabled entirely inside the sandbox (no network namespace), since a submission has no legitimate reason to make outbound network calls.
// Simplified per-test execution with resource limits (Linux cgroups v2)
cgroup.set("cpu.max", "2000000 1000000"); // 2 CPU-seconds per 1s period
cgroup.set("memory.max", "268435456"); // 256 MB
Process p = sandbox.exec(compiledBinary, testInput, timeoutMs=2000);How do you keep judging deterministic and reproducible?
Fixed compiler/interpreter versions pinned per language (never "latest"), a fixed CPU frequency/governor setting on judge hosts (to avoid turbo-boost variance affecting timing-sensitive verdicts), and test execution order that never depends on wall-clock scheduling artifacts. Floating-point comparisons in expected output use an epsilon tolerance rather than exact string match, since bit-for-bit float reproducibility across hardware is not guaranteed even for correct solutions.
Fail-fast vs run-all-tests - which should the judge do?
Fail-fast (stop at the first failing test) saves compute and is common for high-volume practice platforms; run-all-tests gives the submitter full diagnostic detail (useful in a learning context, or for contest problems with partial-credit scoring per subtask) at higher compute cost. Many production judges use a hybrid: fail-fast for the compile/first-test smoke check, then run all remaining tests only if that passes, balancing feedback quality against sandbox-time cost.
How do you prevent one contestant's runaway process from affecting others?
Every resource limit (CPU time, wall-clock time, memory, process/thread count, output size) is enforced at the cgroup/microVM level, not just measured after the fact - a submission that tries to fork-bomb or allocate unbounded memory is killed by the kernel-level limit itself, not by application code politely checking a counter. Because each sandbox is single-use and torn down after judging, a submission that somehow evades all limits and crashes its sandbox still cannot affect the next submission, which runs in a freshly created sandbox.
What is the single biggest bottleneck during a live contest?
Not the database writes - those are cheap, small rows. The real constraint is sandbox execution throughput during the last-two-minutes submission spike, since compute-bound judging cannot be cached or batched away like a read path can. The mitigation is pre-warming a sandbox pool sized for the expected peak (based on registered contestant count, known in advance) before the contest starts, plus autoscaling headroom, rather than reactively scaling once the queue is already backing up.
Post a Comment
Add