Design LeetCode

Three tiers: submission API (accept + queue), judge farm (sandboxed execution), and result store (verdicts + leaderboard). The judge farm is the interesting part — each worker is a short-lived sandbox that compiles, runs, and compares output, then dies.

The fundamental tension: you're running arbitrary untrusted code on your servers. User code can fork-bomb, allocate terabytes, read /etc/passwd, open sockets, or try to escape to the host. The sandbox must make all of these impossible while still allowing normal computation.

Sandbox options (from strongest to lightest):

1. VM per submission (Firecracker). Full kernel isolation. ~125 ms boot. Strongest. AWS Lambda uses this. Expensive at 10K concurrent.
2. gVisor (Google's user-space kernel). Intercepts syscalls, reimplements them in user space. Containers run normally but kernel calls go through gVisor. Moderate overhead (~10% slower), excellent isolation. LeetCode-class services use this.
3. Container with seccomp + namespaces. Standard Linux containers with restricted syscall set. Lighter but shares host kernel — kernel exploits can escape. Fine for trusted code; risky for arbitrary.

Judge pipeline, step by step:

1. Submission lands in queue. Two priority lanes: contest (higher) and practice (lower). During a contest, practice submissions are throttled but not dropped.
2. Worker picks job. Spins up a gVisor sandbox with the language runtime (pre-warmed container pool for Python, Java, C++). Pulls user code from S3.
3. Compile (if needed). C++/Java/Rust compiled inside sandbox. Time limit on compilation (e.g., 30 s). Compile error → verdict = CE.
4. Run against test cases, one by one. Test cases streamed from S3 (not loaded all at once — some problems have 200+ cases, each with large input). For each: set stdin, run, capture stdout, compare against expected. On first Wrong Answer in contest mode → stop (fail-fast). In practice mode → run all for feedback.
5. Resource enforcement. cgroups for memory limit (kill if exceeds). Wall-clock timer for time limit (SIGKILL after TL × 1.5 for safety). No network access (sandbox drops all outbound).
6. Verdict emitted. Worker writes {submission_id, verdict, runtime_ms, memory_kb, test_cases_passed} to verdict service. Worker sandbox destroyed. Leaderboard updated if contest.

sequenceDiagram
    participant U as User
    participant S as Submission svc
    participant Q as Judge Queue
    participant W as Worker (gVisor)
    participant TC as S3 (test cases)
    participant V as Verdict svc
    participant L as Leaderboard

    U->>S: POST /submissions {code}
    S->>S: persist to Postgres + S3
    S->>Q: enqueue (priority=contest)
    S-->>U: submission_id, status=queued
    Q->>W: dequeue job
    W->>W: spin gVisor sandbox
    W->>W: compile (C++ → binary)
    loop each test case
        W->>TC: stream test input
        W->>W: run with time/mem limits
        W->>W: compare output
        Note over W: Wrong Answer → stop (contest)
    end
    W->>V: verdict {AC, 45ms, 12MB}
    V->>L: update leaderboard
    V-->>U: push verdict via WS
          

Deterministic execution. Same code must produce the same verdict every time. Threats: random() in user code (seed it), floating-point non-determinism (accept epsilon), time-dependent code (mock system clock). Workers run on identical hardware specs; container images are pinned versions.

Plagiarism detection. Offline batch job after each contest. Uses MOSS-style (Measure of Software Similarity) fingerprinting: normalize code (strip whitespace, rename variables), extract k-gram hashes, compare Jaccard similarity. Flag pairs above threshold for manual review. AST-based comparison catches deeper structural plagiarism than text-based.

• VM (Firecracker) vs gVisor vs bare container. Firecracker: strongest isolation, ~125 ms boot, expensive at scale. gVisor: strong (user-space kernel), ~30 ms overhead, good balance. Bare container: lightest, shared kernel risk. For a judge: gVisor is the sweet spot.
• Fail-fast vs run-all. Contest: stop on first wrong answer (saves compute, faster verdicts). Practice: run all test cases (better feedback — "you passed 47/50"). Configurable per submission type.
• Pre-warmed pool vs cold-start. gVisor containers take ~1–2 s to cold-start (pull image, init runtime). Pre-warmed pool: keep 500+ idle containers ready. Costs memory but gives sub-second start. During contest: burst beyond pool → cold-start for overflow.
• Test case storage. S3 for the source of truth; local SSD cache on judge nodes for hot problems (top-100 problems account for 80% of submissions). Cache hit → no S3 round-trip.
• Leaderboard freshness. Redis sorted set updated on each verdict → real-time. For 10K-person contest: sorted set with score = (problems_solved × 10000) - total_time_seconds. O(log N) per update.