Communication
Design Slack / Discord
01
What the chat system must do
What the chat system must do
- Users belong to workspaces or servers, which contain channels. They can also send direct messages to one person or a small group.
- Sending a message delivers it in real time to everyone in the channel who is online, and records it so anyone can scroll back through history later.
- Show who is online, who is typing, and which channels have unread messages or mentions for you.
- Support threads, reactions, edits, deletes, file attachments and full-text search over history the user can see.
- Reach people who are offline through mobile push or email when they are mentioned or receive a DM.
- A message reaches online recipients in the same region in under 250 ms at p99.
- Messages in a channel appear in the same order for every member, and an acknowledged message is never lost.
- Hold about 25 million concurrent connections at peak, and recover from losing a gateway server within seconds.
- Channels range from two people to several hundred thousand members, and the big ones must not slow down the small ones.
- Search results may lag new messages by a few seconds; history and delivery may not.
Ask how big the largest channel is and whether every member gets every message in real time. A Slack-style workspace with 20,000 people and a Discord community with 800,000 members need different fan-out strategies. Also ask about retention: some plans keep history forever, others for 90 days, and that changes storage by an order of magnitude.
02
Sizing the load
Sizing the load
Assume 60 million daily active users, about 25 million connected at the busiest hour, and 2.5 billion messages a day. An average message reaches about 40 online clients: most channels are small, but a few large ones pull the mean up.
| Quantity | Working | Result |
|---|---|---|
| Messages per day | assumption | 2.5B |
| Average send rate | 2.5B / 86,400 s | about 29,000 / s |
| Peak send rate | 3x average during working hours in two big regions | about 87,000 / s |
| Deliveries to sockets at peak | 87K / s x 40 online recipients | about 3.5M pushes / s |
| Concurrent connections | assumption at the busiest hour | 25M |
| Gateway servers | 25M / 50K connections each, plus 40% headroom | about 700 |
| Heartbeats | 25M / one every 40 s | about 625,000 / s |
| Bytes per stored message | text plus IDs, author, flags, reactions summary | about 400 B |
| New message data per day | 2.5B x 400 B | about 1 TB, 3 TB with 3 replicas |
| History after a year | 365 x 1 TB, before compression | about 365 TB raw |
Sends are tens of thousands per second, which one partitioned database handles. Deliveries are millions per second, and connections are tens of millions. This is a fan-out and connection-management problem first and a storage problem second.
Presence is quietly the most expensive feature. If every status change were sent to everyone who shares a server with you, a 500,000-member community would generate billions of presence events an hour. Large servers must get presence only on demand.
03
API design
API design
Clients use a persistent WebSocket for everything real time and plain HTTPS for actions that should be retried safely, such as sending a message or loading history.
The connection-level seq and the per-channel channel_seq are different numbers with different jobs. The first lets a reconnecting client ask for exactly the events it missed. The second gives every member the same order inside a channel and makes gaps easy to detect.
04
Architecture overview
Architecture overview
flowchart LR
C["Clients"] --> LB["L4 load balancer"]
LB --> GW["Gateway servers"]
C --> API["HTTP API"]
API --> MS["Message service"]
MS --> CH["Channel servers, hashed by channel_id"]
CH --> DB[("Message store, ScyllaDB")]
CH --> GW
GW --> PS["Presence service"]
MS -. "message events" .-> K[["Event log"]]
K --> NT["Push and email notifier"]
K --> SR[("Search index")]
K --> RS[("Read state and unread counts")]There are three layers, and each one scales on a different number. Gateways scale with connections. They terminate WebSockets, authenticate, keep a small buffer of recent events per session for resumes, and know which channels each of their local users is subscribed to. They do no business logic.
Channel servers scale with message volume. Every channel has one owner, chosen by consistent hashing on channel_id. The owner assigns the next channel_seq, writes the message to the store, and then sends it to every gateway that has at least one subscriber to that channel. Because one process owns each channel, ordering inside a channel is trivial: it is the order the owner processed sends.
The event log carries every message to consumers that are allowed to lag: the notifier that decides who gets a phone push, the search indexer, and the read-state service that keeps unread counts and mention badges. None of them sits on the path between the sender and the people reading the channel.
When a user connects, the gateway registers interest in that user's channels with their channel servers. A channel server therefore holds a set of gateway IDs, not user IDs, which keeps its fan-out list short even for big channels.
A wide-column store partitioned by (channel_id, time bucket) and clustered by message ID. Recent history for one channel is one partition read, and no partition grows without limit.
Holds online, idle and offline state with a TTL refreshed by heartbeats. Small contexts get pushes; large servers get presence only for members currently on screen.
Reads message events, checks each recipient's notification settings and online state, and sends mobile pushes only to people who are mentioned or in DMs and not active on any device.
05
Deep dive: sending, ordering, fan-out and reconnects
Deep dive: sending, ordering, fan-out and reconnects
Can you get a message to tens of thousands of open sockets in order, exactly once from the user's point of view, while gateways crash and phones drop off Wi-Fi? And can you stop the largest channels from starving everyone else?
The send path, step by step
flowchart TD A["Client sends with client_msg_id"] --> B["Message service checks dedupe cache"] B --> C["Route to channel owner"] C --> D["Assign channel_seq"] D --> E["Write to message store, quorum"] E --> F["Ack to sender with message_id"] E --> G["Look up subscribed gateways"] G --> H["Push to each gateway"] H --> I["Gateway writes to each local socket"] E -. "async" .-> J["Event log for search, push, unread"]
The client generates client_msg_id before sending and shows the message immediately in a pending state. If the request times out, it retries with the same ID. The message service keeps recent IDs per channel in a short-lived dedupe cache, so a retry returns the original result instead of creating a second message.
The channel owner assigns the sequence number and writes to the store at quorum before it acknowledges anyone. Acknowledging before the write is durable is the classic way to lose messages, so do not. Once the write succeeds, the owner looks up which gateways have subscribers and sends each of them one copy. Each gateway then writes to every local socket that is subscribed. A message for a channel with 20,000 online members spread over 300 gateways costs the channel server 300 sends, not 20,000.
Keeping order and spotting gaps
Clients track the highest channel_seq they have rendered for each open channel. If message 50,124 arrives and the last one seen was 50,122, the client knows it missed one and fetches it from history. That makes the real-time path allowed to be best effort: a dropped push is repaired by a cheap history read, and nobody needs exactly-once delivery across the network.
Channel ownership moves when a channel server fails. The new owner reads the latest channel_seq from the store before accepting sends, and the store write is conditional on that sequence not existing yet. Two owners briefly believing they own the same channel therefore cannot both write message 50,123.
Big channels need a different plan
| Channel size | Real-time delivery | Presence | Typing indicators |
|---|---|---|---|
| Up to 1,000 members | Push to every subscribed gateway | Pushed on change | Shown |
| 1,000 to 50,000 | Push, batched per gateway every 50 ms | Only for members visible in the sidebar | Shown, rate limited |
| Over 50,000 | Push only to clients with the channel open; others get an unread bump | Online count only | Hidden |
The largest communities break the simple model in two ways. A single announcement to 600,000 members can produce hundreds of thousands of socket writes from one send, and members who are not looking at the channel do not need the full message right now. So for very large channels, gateways push the full event only to sockets where the channel is currently on screen. Everyone else gets a tiny "unread in channel X" signal and fetches the message when they open it. This is lazy delivery, and it cuts fan-out for big servers by an order of magnitude.
Hot channels can also overload their owner. Rate limit sends per channel, give very large channels a dedicated owner process, and let that owner batch outgoing events per gateway over a short window so a burst of 2,000 messages a minute becomes a few hundred network writes.
Reconnects and resume
Phones drop connections all day. On reconnect, the client sends its session_id and the last connection seq it processed. If the client lands on a gateway that still has that session's buffer, typically the last two minutes of events, it replays only the missing ones. If the buffer is gone because the gateway restarted, the client falls back to a fresh session and asks each open channel for messages after its last channel_seq. Either way the user sees no duplicates and no holes.
Unread counts without a counter per message
Storing an unread flag per user per message would mean writing to every member on every send. Instead, store one read marker per user per channel: the last channel_seq they have read. Unread count is then the channel's latest sequence minus the marker, which is a subtraction, not a query. Mentions are the exception, because they must be counted precisely; the read-state service keeps a small per-user mention counter updated from the event log.
Gateways hold WebSockets and register interest per channel. Each channel has a single owner chosen by consistent hashing, which assigns a per-channel sequence, writes to a store partitioned by channel and time, and pushes one copy to each subscribed gateway. Clients detect gaps using the sequence and repair them from history. Big channels get lazy delivery and on-demand presence, and everything slow hangs off an event log.
06
Decisions and what they cost
Decisions and what they cost
Ordering, sequence numbers and fan-out lists all live in one place, so they are simple and fast. The cost is a failover step when an owner dies, and very hot channels need their own process.
Simple orderingSpreads load evenly but needs a distributed sequencer or a consensus step per message to agree on order, which adds latency to every send.
Ordering gets expensiveMessages are pushed once per gateway as they are written. Readers who are online see them immediately, and history is read only when scrolling or repairing gaps.
Real-time firstMakes "what is new for me" one read, like a feed. For a 600,000-member channel it means 600,000 writes per message, which is unaffordable.
Only for DMs and small groupsWrites are append-only, recent history is one partition, and time buckets keep busy channels from producing giant partitions. Edits and deletes are rare updates.
Built for this patternFamiliar, with transactions for edits and threads. Workable for a Slack-sized product, but resharding a few hundred terabytes of history is painful and write amplification grows with indexes.
Works at smaller scaleUsers get live status for people they are actually looking at. Everything else shows a count.
Bounded costAccurate everywhere, and the presence traffic alone would exceed message traffic for large communities.
Explodes with size07
Failure modes
Failure modes
A kernel panic takes one gateway down. Fifty thousand clients notice within one heartbeat and all reconnect within the same few seconds, hammering the load balancer and the auth service.
Clients reconnect with exponential backoff plus random jitter, the auth service accepts a cheap resume token instead of a full login, and gateways are sized to absorb the sockets of two failed peers.A rolling restart moves too quickly and drops several million connections in a few minutes. Presence flaps from online to offline and back for millions of users, generating a storm of status events.
Drain gateways slowly, telling clients to reconnect elsewhere over several minutes. Suppress presence changes for a grace period after a disconnect, so a user who is back within 30 seconds never appears offline.A moderator posts in the main channel of a huge community and mentions everyone. The owner process, the gateways and the notifier all spike at once, and small channels on the same channel server slow down.
Lazy delivery for large channels, a dedicated owner for them, and a notifier that spreads mass mention pushes over a minute or two. Restrict mass mentions to admins and rate limit them.A user on a train taps send, the request times out after the server accepted it, and the client retries. Without protection the channel shows the same message twice.
The client-generated message ID is checked against a dedupe cache before the owner assigns a sequence, so the retry gets the original message ID back.A network partition makes the old channel owner unreachable, the new owner takes over, and the old one keeps accepting sends from gateways on its side.
Ownership is backed by a lease with a fencing token, and the store write is conditional on the sequence number. The old owner's writes fail as soon as the new owner writes the same sequence.The search indexer stalls during a cluster upgrade, and users cannot find messages from the last hour.
The event log retains days of messages, so the indexer catches up without loss. Show a notice in search when lag exceeds a minute, and fall back to scanning recent channel history for very fresh queries.08
Mistakes to avoid
Mistakes to avoid
Every client asking every few seconds wastes most requests and still feels slow.
Hold a persistent connection and push events, with HTTP only for sends and history.Large channels turn one send into hundreds of thousands of writes.
Store each message once per channel and track a read marker per member.Clocks on different servers disagree by milliseconds, which is enough to reorder a fast conversation.
Let a single channel owner assign a sequence number.Presence traffic grows with the square of community size.
Push presence for small groups and fetch it on demand for large ones.A slow push provider then slows down every message.
Drive notifications from an event log consumer.09
How to handle the interview
How to handle the interview
Separate connections from logic. Say early that gateways only hold sockets and that channel servers own ordering and fan-out. It makes the rest of the design easy to explain.
Quote the fan-out number. Sends per second times online recipients gives millions of deliveries per second. That is the number your architecture has to survive.
Solve ordering with a single writer. One owner per channel and a sequence number is the standard answer. Explain how you fence an old owner after failover.
Treat delivery as best effort and repair gaps. Sequence numbers plus history reads are cheaper and more robust than trying to guarantee every push arrives.
Bring up giant channels yourself. Describe lazy delivery and on-demand presence before the interviewer asks what happens with 700,000 members.
Cover reconnects. Session resume with a replay buffer, then a fallback to per-channel catch-up. Mention jittered backoff for mass reconnects.
Keep slow work off the hot path. Search, push notifications and unread counts read an event log and are allowed to lag.
10
Growing the design in stages
Growing the design in stages
Every stage below is a response to one number: connections, sends per second or the size of the largest channel. Change the design when that number changes, not before.
One process holds all WebSockets and writes messages to a table. Fan-out is a loop over connected users. Fine for a few thousand users.
Connections spread across gateways behind a load balancer. A Redis or NATS pub/sub channel per conversation lets any gateway receive messages from any other.
Pub/sub topics become owned channel processes that assign sequence numbers and track gateway subscriptions. History moves to a store partitioned by channel and time.
Channels with hundreds of thousands of members push full events only to active viewers. Presence becomes subscription based. Search, push and unread counts move behind an event log.
Users connect to the nearest region. Each workspace or server has a home region that owns its channels, and other regions relay. Cross-region sends add latency, so place the home region near most members.
Watch and read
Try next
Free to read · better with landed
Whiteboard this with landed
landed runs it as a live system design round on the whiteboard and grades your trade-offs.