Marketplace & Booking
Design URL Shortener
01
What the system must do
What the system must do
- A signed-in user or an API client submits a destination and gets back a 7-character code on our domain.
- Opening the short link sends the browser to the destination with an HTTP redirect.
- Paid workspaces can pick a vanity alias such as
/spring-sale, first come first served. - Owners can set an expiry date, change the destination, or disable a link.
- Owners see click totals broken down by day, country, referrer and device, with a few minutes of delay being acceptable.
- Destinations that point at known phishing or malware are refused at creation and can be blocked later.
- Redirect latency of p99 under 50 ms measured at our edge, not counting the destination page.
- The redirect path stays up at 99.99%; creating links may degrade before redirects do.
- A code, once issued, is never reissued to another destination, even after the link is deleted.
- Codes should not be guessable in sequence, so nobody can walk the keyspace and scrape private links.
- Click logging must never add latency to the redirect itself.
Settle four things in the first two minutes: how many links are created per day, how long links live, whether analytics must be real time, and whether a changed destination must take effect instantly. The last one quietly decides your cache TTLs and whether you can use permanent redirects at all. Also ask whether the same long URL submitted twice should return the same code. Most products say no, because two campaigns pointing at one page still want separate click counts.
02
Sizing the load
Sizing the load
Assume 8 million new links a day and a read ratio of about 150 clicks per link created. Keep links for five years by default. The numbers below follow from those three choices.
| Quantity | Working | Result |
|---|---|---|
| Links created per day | assumption | 8M |
| Average create rate | 8M / 86,400 s | about 93 / s |
| Peak create rate | 3x average for bulk API imports | about 280 / s |
| Redirects per day | 8M x 150 | 1.2B |
| Average redirect rate | 1.2B / 86,400 s | about 13,900 / s |
| Peak redirect rate | 4x average, evening plus a viral spike | about 56,000 / s |
| Links retained | 8M x 365 x 5 years | about 14.6B |
| Bytes per link row | 180 B URL + code, owner, dates, flags, index overhead | about 400 B |
| Link store size | 14.6B x 400 B, then 3 replicas | 5.8 TB raw, 17.5 TB stored |
| Click events per day | 1.2B x about 120 B each | about 144 GB, roughly 15 GB compressed |
| Hot cache footprint | 20M busiest links x about 300 B in Redis | about 6 GB, provision 24 GB |
| Redirect egress | 56K / s x about 450 B response | about 25 MB / s at peak |
Writes are under 300 per second, which a single well-tuned database could absorb. The interesting number is 56,000 lookups per second at peak, each of which a person is waiting on. Almost every decision in this design serves that read path, and the write path only has to be correct.
17.5 TB across five years is a modest cluster. The click log grows faster than the link table, so it goes to a columnar analytics store with its own retention policy instead of living next to the links.
03
API design
API design
A 301 tells browsers they may remember the answer forever. That saves you traffic, but the owner can never retarget the link for anyone who already clicked it, and you stop seeing their repeat clicks. This design returns 302 with a short private cache lifetime. Offer 301 only as an opt-in for links whose owners explicitly want maximum caching and do not need analytics.
04
Architecture overview
Architecture overview
flowchart LR
U["Browser"] --> E["Edge PoP cache"]
E -->|miss| R["Redirect service"]
R --> L1["In-process LRU"]
R --> RC[("Redis cluster")]
R --> KV[("Link store, KV partitioned by code")]
R -. "click event" .-> Q[["Kafka: clicks"]]
C["Client or API user"] --> W["Link API"]
W --> A["ID range allocator"]
W --> S["URL safety scanner"]
W --> KV
W -. "invalidate" .-> E
Q --> AG["Stream aggregator"]
AG --> CH[("ClickHouse")]Split the system into two services that never share a deploy: a tiny redirect service that only reads, and a link API that handles creation, edits and the dashboard. The redirect binary has no dependency on the safety scanner, the allocator or the analytics store, so none of them can take redirects down. If the link API is broken for an hour, people cannot make new links. If the redirect service is broken for an hour, every link ever printed on a flyer is dead. Those deserve different blast radii.
A click travels through up to four layers, cheapest first. The edge answers popular codes without touching our origin. On a miss, the redirect service checks a small in-process cache, then Redis, then the link store. Every answer, including "not found", is written back to the layers above it with a TTL. The service then drops a click event on a local buffer and returns the 302. A background thread ships buffered events to Kafka in batches, so a slow broker costs memory, not latency.
CDN or edge workers keep each redirect for 60 seconds. With a strongly skewed popularity curve, that alone absorbs most peak traffic, and a purge API call on edit or delete closes the staleness gap.
Stateless, horizontally scaled, a few hundred lines of code. It keeps an in-process LRU of about 200,000 entries and uses single-flight so a thousand concurrent misses for one code cause one Redis read.
Redis holds the working set with LFU eviction. Behind it, a KV store such as DynamoDB or Cassandra keyed by code holds every link. Lookups are single-key gets, so there are no joins and no secondary indexes on the hot path.
The allocator leases blocks of numeric IDs to link API pods. Clicks flow through Kafka into a stream job that rolls them up per minute and writes to ClickHouse, which serves the dashboard queries.
05
Deep dive: minting codes and surviving hot links
Deep dive: minting codes and surviving hot links
There are only two hard parts here. How do many writers produce unique, unguessable codes without asking each other? And what happens when one link suddenly gets a large share of all traffic? Everything else is plumbing.
flowchart TD
P["Link API pod starts"] --> LQ{"Local block empty?"}
LQ -->|yes| LEASE["Lease next 10,000 IDs from allocator"]
LEASE --> LQ
LQ -->|no| N["Take next integer n"]
N --> F["Permute n with a keyed 42-bit Feistel"]
F --> B["Encode result in base62, pad to 7"]
B --> PUT["Conditional put: code must not exist"]
PUT --> OK["Return short URL"]Leased ID ranges instead of hashing
The obvious approach is to hash the long URL and keep the first seven base62 characters. It fails twice. Truncated hashes collide well before the keyspace fills, by the birthday bound, so every insert needs a read to check, and a retry with a salt when it hits. And identical URLs map to identical codes, which is exactly what the product does not want.
A counter avoids collisions entirely, as long as no two writers ever hold the same number. So we make the counter a lease. A small allocator, backed by a strongly consistent store such as a single Postgres row or an etcd key, hands each link API pod a block of 10,000 integers with one atomic increment. The pod mints codes from memory until the block runs out, then leases another. At 280 creates per second the allocator sees about two lease calls a minute across the whole fleet. If a pod crashes mid-block, the unused numbers are simply skipped. With 3.5 trillion codes available, wasting a few thousand per crash costs nothing.
Raw counters are sequential, which breaks the "not guessable" requirement. The fix is to pass each integer through a keyed permutation before encoding it. A four-round Feistel network over a 42-bit domain is a bijection: distinct inputs always give distinct outputs, so uniqueness survives, but neighbouring integers land far apart in the keyspace. Since 242 is about 4.4 trillion and 627 is about 3.5 trillion, use cycle-walking: if the output is at or above 627, permute again until it fits.
Vanity aliases bypass the allocator and go straight to a conditional write that fails if the key exists. To stop an alias from ever clashing with a minted code, reserve a namespace: minted codes are exactly seven characters, and aliases must be at least eight or contain a hyphen. The conditional write stays on minted codes too, as a cheap safety net against a bug in the permutation or a reused key.
Hot links and the read path
Link popularity follows a power law. On a normal day the top 1% of links might take 70% of clicks, and on a bad day a single code from a celebrity post or a TV advert takes tens of thousands of requests per second for twenty minutes. Three mechanisms handle this, each cheaper than the next one down.
First, the edge. A 60-second edge TTL means a code getting 20,000 requests per second reaches origin about once per minute per PoP. Second, the in-process LRU with request coalescing: when a miss is already in flight for a code, later callers wait on the same future instead of issuing their own read. Third, Redis itself, where a single hot key lives on one shard. If monitoring shows one shard running hot, the redirect service can read that key from replicas or keep a short-lived local copy, which turns a cluster problem back into a cache-hit problem.
Do not forget negative caching. Bots probe random codes all day. Without caching "not found" for a minute, each probe goes all the way to the link store, and a scanner sending 5,000 junk codes per second becomes your heaviest database client.
Choosing seven characters
Base62 gives 626, about 56.8 billion codes, at six characters, and 627, about 3.52 trillion, at seven. Five years of our traffic needs 14.6 billion, which is already a quarter of the six-character space. Growth or a single bulk-import customer could exhaust it. At seven characters the same five years use about 0.4% of the space, which leaves decades of headroom and keeps random guessing hopeless: an attacker hitting a valid code would need roughly 240 tries per hit, and rate limiting makes that slow and loud.
06
Decisions and what they cost
Decisions and what they cost
No collision checks, no coordination per write, and codes that look random. The cost is an allocator to run and a secret permutation key that must never change, because changing it would make new codes collide with old ones. Store it like a signing key.
Uniqueness by constructionSimple and stateless, and fine for a prototype. Both need a read-before-write uniqueness check that gets more expensive as the table fills, and both need retry logic for the rare clash. A pre-generated pool of random codes also works but means storing and serving billions of unused keys.
Fine at small scaleEvery click still reaches us or our edge, so analytics stay complete and edits take effect within a minute. You pay for that in origin and edge requests, which the cache tiers already make cheap.
Default for all linksBrowsers and some proxies cache it with no expiry. Repeat visits vanish from the stats, and a retargeted or taken-down link keeps working for anyone who clicked it once. That is a support ticket and, for a malicious link, a safety problem.
Opt-in onlyEvery redirect is a single-key read, and every create is a conditional put. A partitioned KV store scales both linearly by adding nodes and needs no manual resharding at 17.5 TB. Owner listings use a secondary table keyed by workspace.
Matches the access patternGives you transactions and ad hoc queries, which help the dashboard. But you own the sharding scheme, the rebalancing and the failover. A reasonable choice if the team already runs Postgres at scale, keeping the hot path cache-first either way.
Workable with more opsThe redirect returns before the event is durable. You can lose a few seconds of clicks if a pod dies, which is acceptable for marketing analytics, and in return the broker can be slow or down without touching latency.
Latency firstWriting clicks = clicks + 1 to the link row puts a write on the hottest read path and turns each viral link into a single-row contention point. It also cannot answer "clicks by country" without more writes.
07
Failure modes
Failure modes
A bad config push restarts every cache node at once. Hit rate drops to zero and 56,000 requests per second fall onto a link store sized for a few thousand. Latency climbs, retries pile on, and the store starts throttling. This is the outage that turns a cache problem into a database problem.
Roll cache changes one shard at a time, keep the edge and in-process tiers as a buffer, coalesce misses, and cap the redirect service's concurrent store reads so excess requests get a fast 503 instead of a slow pile-up.A code created thirty seconds ago appears in a push notification to millions of phones. Every PoP misses at the same moment and thousands of requests race for the same row.
Write new links into Redis at creation time, so the first read is already warm. Single-flight at the redirect service and request collapsing at the edge reduce a stampede to one origin fetch per PoP.A storage node fails and one range of codes cannot be read until replicas take over. Links in that range return errors while every other link works.
Replicate three ways across availability zones and read at a consistency level that tolerates one replica down. Serve cached entries past their TTL while the store is unhealthy; a slightly stale redirect beats an error page.The ClickHouse ingest job stalls during a migration. Kafka keeps accepting events, but dashboards show numbers from an hour ago and customers file bugs about missing clicks.
Size Kafka retention for at least 72 hours of events so nothing is lost, show an "as of" timestamp in the dashboard, and alert on consumer lag rather than on dashboard staleness.The store behind the allocator fails over. Pods that drain their current block cannot lease another and link creation starts failing.
Let each pod hold a spare block on top of its active one, which covers several minutes of creation at peak. The allocator only needs to be up most of the time, not all of the time, and redirects never depend on it.Trust and safety disables a malicious code, but the edge, Redis and in-process caches still hold the old destination. Victims keep landing on the phishing page for the length of the TTL.
On disable, delete the Redis key, publish an invalidation that each redirect pod applies to its LRU, and call the CDN purge API. Keep TTLs short enough that a missed purge self-heals within a minute.A cloud region goes dark. Users routed there see timeouts until DNS or anycast moves them, and links created in the last few seconds may not have replicated out.
Run the redirect path active-active in at least two regions with the link store replicated between them, and health-check routing at the edge. Accept that a link minted moments before the failure may 404 briefly; ID blocks are leased per region, so no code is ever issued twice.08
How to handle the interview
How to handle the interview
Say early that this is a read problem. Do the arithmetic out loud, land on tens of thousands of reads per second against a few hundred writes, and let that number justify every cache you add.
Compare at least two ways to make codes. Hashing, random codes and counters each have a failure you should name. Interviewers want to hear why truncated hashes collide and how you would make a counter non-sequential.
Have an opinion on 301 versus 302. Tie it to analytics and to taking down abusive links. It is a small detail that shows you understand caching outside your own servers.
Keep analytics off the hot path. Draw the event queue explicitly and state what you lose if a pod crashes before flushing. Owning the trade-off is better than hiding it.
Expect the hot-key question. When asked what happens if one link gets half the traffic, walk the layers: edge TTL, request coalescing, replica reads. Mention negative caching before they ask about bots.
Justify the code length with numbers. Show 62 to the sixth and seventh powers next to your five-year link count. One line of maths ends the discussion.
Bring up abuse yourself. Shorteners are a favourite tool for hiding phishing URLs. A creation-time safety scan, a disable switch that purges caches, and rate limits per workspace show product sense, not just plumbing.
Do not over-build the first answer. Start with one service, one database and one cache, then add the edge, the allocator and multi-region only when your numbers ask for them. The ordering is part of what is being graded.
09
Growing the design in stages
Growing the design in stages
Each stage below exists because a specific number crossed a line. Add the next component when monitoring shows you need it, not because the final diagram has it.
A single service with an auto-increment ID, base62 encoded, and a unique index on the code column. Postgres serves every redirect from memory. This is enough for most internal tools and costs almost nothing.
Put Redis in front of the database, move redirects into their own small service, and add read replicas. Click counting moves to a queue so it stops writing to the link table.
Popular links now dominate, so a 60-second edge TTL with purge on edit takes most traffic off origin. Codes switch to the permuted counter so the public keyspace cannot be crawled.
The link table outgrows one primary. Move it to a KV store partitioned by code and introduce the range allocator so many writers can mint without coordinating. Analytics move to ClickHouse. The single Postgres primary is gone.
Run the redirect path in several regions with replicated link data and region-scoped ID blocks. At this size you may also answer redirects in edge workers from a replicated KV, so most clicks never reach a region at all.
Watch and read
Further watching and reading
Try next
Free to read · better with landed
Whiteboard this with landed
landed runs it as a live system design round on the whiteboard and grades your trade-offs.