Social & Feed
Design Instagram
01
Scope and targets
Scope and targets
- Post one or more photos or a short video with a caption, location and tagged accounts.
- Follow and unfollow accounts; private accounts approve followers.
- A home feed of recent posts from followed accounts, ranked by likely interest.
- Like, comment and save posts; see counts on every post.
- Profile grid of an account's posts, newest first.
- Delete a post or make an account private, with the change reflected everywhere quickly.
Out of scope here: stories, direct messages, live video and the explore tab, though the media pipeline serves all of them.
- Feed loads in under 250 ms p95 on the server; first images visible within a second on 4G.
- Uploads survive network drops and app restarts without starting over.
- A photo post is visible to followers within 10 seconds; video within a minute.
- Uploaded media is never lost: originals stored with eleven nines of durability.
- Feeds may be slightly stale, and counts may lag by a few seconds.
- Deleted or privacy-restricted posts stop being served within a minute, including from caches.
Clarify the ratio of photos to video, the maximum video length, and whether the feed is chronological or ranked. Video changes storage and processing cost by an order of magnitude. A ranked feed changes how much work happens at read time. For this walkthrough: about one post in five is a short video under 90 seconds, and the feed is ranked.
02
Sizing it
Sizing it
Assume 600 million daily users who each open the feed about twelve times a day, and 70 million new posts a day, of which 15 million are videos. People follow about 250 accounts on average.
| Quantity | Working | Result |
|---|---|---|
| Posts per second | 70M / 86,400, peak 3x | about 810 average, 2.4K peak |
| Photo originals per day | 55M posts x 1.4 photos x 1.8 MB | about 140 TB |
| Video originals per day | 15M x 18 MB | about 270 TB |
| Derived renditions | photo sizes about 0.3x, video ladder about 0.8x of originals | about 260 TB / day |
| Yearly storage growth | 670 TB x 365, erasure coded at 1.4x | about 340 PB |
| Feed loads per day | 600M x 12 | 7.2B |
| Peak feed loads | 7.2B / 86,400 x 3 | about 250K / s |
| Feed entries written by fan-out | 70M posts x about 300 follower inboxes | about 21B / day, 240K / s |
| Media egress | 7.2B loads x 8 items x about 220 KB | about 12 PB / day, 1.1 Tbps average |
| Likes per second | about 4B likes a day | about 46K / s, very skewed |
Storage grows by about a third of an exabyte a year, so storage tiering and erasure coding are not optional. And reads outnumber posts by roughly a hundred to one, so feeds must be cheap to read even if that makes writes more expensive.
03
Endpoints and payloads
Endpoints and payloads
04
System overview
System overview
flowchart LR
APP["Mobile app"] -->|"chunks, signed URLs"| UPL["Upload edge"]
UPL --> RAW[("Originals bucket")]
APP --> API["API gateway"]
API --> PS["Post service"]
RAW -. "upload complete" .-> MQ[["Media job queue"]]
MQ --> IMG["Image workers"]
MQ --> VID["Video transcoders"]
IMG --> DER[("Renditions bucket")]
VID --> DER
DER --> CDN["CDN"]
PS -. "post published" .-> FO["Fan-out workers"]
FO --> INB[("Feed inboxes")]
API --> FS["Feed service and ranker"]
FS --> INB
FS --> GR[("Follow graph")]The media path never goes through the API servers. The app asks for an upload session, receives signed URLs, and sends chunks straight to an upload edge that writes into an originals bucket. When the last chunk lands, an event goes onto a job queue. Image workers produce the sizes and formats the app needs; video transcoders produce a bitrate ladder and segments for adaptive streaming. Renditions go to their own bucket, fronted by a CDN.
The social path handles small records. The post service stores the post row, its media references and a state field that moves from processing to live when the media pipeline reports success. Going live emits an event that fan-out workers use to push the post id into followers' feed inboxes. The feed service reads a user's inbox, merges in posts from very large accounts the user follows, ranks the candidates and hydrates them with author data, counts and media URLs.
Supporting stores are each picked for one access pattern. The follow graph is a sharded wide-column table with two directions, followers of X and accounts X follows. Feed inboxes are capped lists in a cache cluster backed by a durable store. Counters for likes and comments are sharded counters with a cached total.
Accepts 4 MB chunks in any order, verifies each with a checksum, and tracks progress per session. Uploads go to the nearest region, then replicate.
Stateless, autoscaled on queue depth. Photos take well under a second; a 60-second video takes tens of seconds of CPU or a fraction of that on hardware encoders.
Posts sharded by author id so a profile grid is a single-shard range read. Post ids are time ordered, which gives cheap "newest first" sorting everywhere.
Read follower lists page by page and append the new post id to each follower's inbox. Accounts above a follower threshold are skipped and pulled at read time instead.
Gathers about 500 candidates, scores them with a model that predicts likes, comments, saves and time spent, and returns the top dozen with a cursor.
Serves nearly all media bytes. Rendition URLs include the post id and a version, so they are immutable and can be cached for a long time.
05
Deep dive: getting media from a phone to every screen
Deep dive: getting media from a phone to every screen
The input is a 20 MB file on a phone with an unreliable connection. The output is a dozen renditions, scanned for abuse, replicated across regions and cached at the edge, with the post appearing in feeds only once every one of them exists. Each step can fail independently, and the user should only ever see "posting" and then "posted".
Step 1: resumable, direct upload
Routing large uploads through API servers ties up threads and memory for the length of a slow upload and makes those servers the bottleneck. Instead, the API only creates an upload session and hands back signed URLs, one per chunk. The phone uploads chunks in parallel, two or three at a time, straight to the upload edge. If the app is killed or the train goes into a tunnel, it asks which chunks arrived and sends only the rest. A session expires after a day, and a lifecycle rule deletes incomplete uploads so orphaned chunks do not pile up.
Two tricks make posting feel instant. The app starts uploading while the user is still writing the caption, so by the time they tap share most bytes are already on our side. And the app compresses on the device first, often cutting a photo from 6 MB to under 2 MB with no visible loss, which saves the user's data plan and our ingress.
Step 2: a processing state machine
When the last chunk is confirmed, the upload edge assembles the file, checks the full checksum and writes one job to the queue. Processing is a small workflow with explicit states rather than one long function, so a crash resumes from the last completed step.
flowchart TD U["Chunks uploaded, checksum verified"] --> S["Safety checks: hash match, nudity and abuse models"] S -->|blocked| X["Reject and notify the author"] S -->|ok| M["Strip location metadata, fix rotation"] M --> P["Images: 4 sizes in AVIF and JPEG"] M --> V["Video: bitrate ladder, 4 second segments, poster frames"] P --> R["Write renditions, record versions"] V --> R R --> L["Mark post live"] L --> F["Emit event for fan-out, search and notifications"]
Safety first. Compare the perceptual hash against databases of known abusive content and run classifiers before anything else. A blocked upload never produces renditions and never reaches a CDN.
Images. Strip GPS metadata unless the user attached a location on purpose, apply the orientation flag, then render widths such as 150, 320, 640 and 1080 pixels. Encode in AVIF or WebP for apps that support them and JPEG as a fallback. The app picks the size for its screen.
Video. Transcode into a ladder, for example 240p, 480p, 720p and 1080p at increasing bitrates, cut into four-second segments with an HLS or DASH manifest. The player starts on a low rung and climbs as bandwidth allows, which keeps start-up fast on weak networks. Shorter segments start faster; longer ones compress better. Videos are processed in parallel by splitting them at keyframes and encoding the parts on different workers.
Going live. The post service flips the state to live only after every required rendition is recorded. For video, you can go live once the lowest two rungs exist and add higher ones as they finish, which gets most posts into feeds within half a minute.
Step 3: delivery and storage tiers
Rendition URLs are immutable, so the CDN can keep them for weeks. A post that goes viral is pre-warmed: once its view rate passes a threshold, the CDN is told to fetch its renditions into more edge locations before requests arrive there.
Storage cost is dominated by the long tail. Most views happen in the first few days after posting. After 30 days, move originals to an infrequent-access tier; after a year, to cold storage, keeping only the renditions hot. Use erasure coding rather than three full copies for anything not hot, which cuts raw bytes stored from 3x to about 1.4x. Content-addressed storage of originals by hash also removes exact duplicates, such as the same meme uploaded by thousands of people.
Treat media as immutable blobs produced by an idempotent, restartable pipeline, and treat the post as a small record whose state says whether those blobs are ready. Everything else, caching, retries, tiering, follows from that split.
06
Choices and their costs
Choices and their costs
Push new post ids into followers' inboxes for ordinary accounts, which makes reads a single list fetch. For accounts with more than about a million followers, skip the push and merge their recent posts in at read time. Reads stay cheap and one celebrity post does not trigger a hundred million writes.
Fast reads, bounded writesNo fan-out writes at all, but each feed load queries recent posts from hundreds of accounts and merges them. At 250,000 loads a second, that is tens of millions of lookups a second.
Reads too expensiveThe only questions asked are "who follows X" and "whom does X follow", both answered by a range scan on one partition. Scales horizontally and handles billions of edges.
Fits the access patternGreat for multi-hop questions such as friends of friends, which the feed does not need. Harder to operate at this size and slower for the simple adjacency lookups that dominate.
For recommendations, maybeA fixed set of renditions is ready before the post goes live, so the first viewer never waits. Costs CPU and storage for sizes some posts never need.
Predictable latencySaves work for posts nobody sees, and suits rare sizes. But a viral post triggers many concurrent resizes on cache miss, and first views are slower.
For rare sizes onlyPeople see the posts they care about first, which matters once they follow hundreds of accounts. Ranking about 500 candidates per load adds model serving cost and makes debugging "why did I see this" harder.
Better engagementSimple and transparent, but heavy posters bury everyone else and people miss posts from close friends. Offer it as an option rather than the default.
Optional mode07
When things break
When things break
If it were treated like a normal account, fan-out would try to write hundreds of millions of inbox entries, delaying every other post queued behind it for many minutes.
Mark large accounts for pull-based delivery, cache their latest posts in a small hot set every feed server can reach, and give fan-out separate queues by follower count so big jobs never block small ones.Video uploads jump five times in an hour. The queue grows, and posts sit in "processing" for twenty minutes.
Autoscale on queue age, not just depth. Publish once the low rungs are ready and encode high rungs later. Keep a pool of spare capacity, and degrade by skipping the 1080p rung temporarily rather than delaying posts.A 30 MB video over a congested connection times out at 80%. Without resume, the user retries from zero and gives up.
Chunked, resumable uploads with server-side progress, background upload on the device, and on-device compression before the transfer starts.A viral reel takes 40,000 likes a second. Incrementing one row serialises every write on one shard.
Split the counter into many sub-counters with random assignment, sum them periodically into a cached total, and record the per-user like edge separately so "did I like this" stays exact while the count is approximate.The post row is gone, but its id sits in millions of inboxes and its images are cached at the edge.
Filter inbox entries against post state during hydration, so deleted posts drop out on the next load. Issue CDN purges for the rendition URLs, and for privacy-sensitive removals require signed short-lived media URLs on private accounts.Three full copies of every original in hot storage, forever, turns a third of an exabyte a year into the largest line on the infrastructure bill.
Tier by age, erasure-code cold data, deduplicate by content hash, and delete abandoned uploads and unused intermediate files quickly.08
Designs that look right and are not
Designs that look right and are not
It works in a demo. At scale, every slow phone holds a server thread for a minute, and API capacity is sized by upload bytes instead of requests. Use signed direct uploads.
The user waits for a transcode, and a timeout loses both the post and the work. Accept the post, return quickly, and process asynchronously.
Blobs bloat backups and replication and turn every image view into a database read. Store bytes in object storage and only references in rows.
Pure push breaks on celebrities, and pure pull breaks on read volume. The follower distribution is skewed, so the policy must be too.
Strongly consistent like counts force serialised writes on hot posts. Nobody can tell 1,904 likes from 1,907. Make the count eventually consistent.
Reusing a URL for a re-encoded file means purging caches every time. Version the URL and caches never serve the wrong bytes.
09
Interview notes
Interview notes
Split media from metadata in the first five minutes. Two paths, two scaling stories. It frames the rest of the interview and stops you mixing concerns.
Do the storage arithmetic. Showing that originals add hundreds of terabytes a day leads naturally to tiering, erasure coding and deduplication.
Explain resumable uploads concretely. Session, chunks, signed URLs, a progress check. This is where many candidates wave their hands.
Know your fan-out answer cold. Push for most accounts, pull for very large ones, merge at read time. Give a threshold and say why the follower distribution forces it.
Describe adaptive streaming. A bitrate ladder, short segments and a manifest explain how video starts fast on bad networks.
Handle hot posts. Sharded counters and CDN pre-warming show you have thought past the average case.
Remember deletion and privacy. Inboxes and caches hold copies. Explain how a deleted post disappears everywhere.
10
Growing the design in stages
Growing the design in stages
Postgres for users, posts and follows, images in object storage, resized in the request. The feed is a query over followed accounts. Fine for the first million users.
Signed uploads straight to storage, a job queue for resizing, and a CDN in front of renditions. Posts gain a processing state.
Shard posts and follows by user id, precompute feeds into cached inboxes with fan-out on write, and move counts to sharded counters.
Add video with adaptive streaming, switch large accounts to pull-based delivery, and introduce storage tiers and erasure coding as costs climb.
Upload and serve from the nearest region, rank the feed with a model trained on engagement, and move to newer codecs on capable devices to cut egress.
Watch and read
Try next
Free to read · better with landed
Whiteboard this with landed
landed runs it as a live system design round on the whiteboard and grades your trade-offs.