Design Slack / Discord

Two surfaces: a REST API for stateful operations (create channel, fetch history) and a WebSocket gateway for real-time events. All real-time flows use the WebSocket; REST is for things you'd do from any tab.

The defining piece is the gateway tier — a fleet of stateful WebSocket servers. Each gateway holds tens of thousands of connections. A session store (Redis) maps user_id → gateway_id + conn_id so any service can push to any user. A fan-out bus (Redis pub/sub or Kafka) delivers messages to the right gateways.

Sending a message in a 50k-member channel means one write + 50k pushes. The critical path is: persist → fan-out → edge-deliver. The trick is keeping it fast while staying correct under reconnects, dedup, and order preservation.

1. Client sends over WebSocket to its gateway with a client-generated nonce (for dedup) and the target channel_id.
2. Gateway validates auth + rate limits (e.g., 5 msgs / 5 s per channel per user). Relays to Message service via internal gRPC.
3. Message service writes to Cassandra with partition key = channel_id, clustering key = message_id (Snowflake/ULID — monotonic, sortable). The message_id IS the ordering; no per-row timestamps needed.
4. Publish to Redis pub/sub on topic channel:{channel_id}. Every gateway subscribes to topics for channels it has active viewers in.
5. Gateways receive the pub/sub event and push over WS to each local subscriber. At 10M conns × 500 gateways, one channel publish touches only the ~10 gateways where actual viewers sit — not all 500.
6. Offline users route through Push service: APNs / FCM notification if mobile; queued in inbox if web.

sequenceDiagram
    participant C as Client
    participant G1 as Gateway A
    participant MS as Message svc
    participant DB as Cassandra
    participant PS as Redis pub/sub
    participant G2 as Gateway B
    participant R as Recipient

    C->>G1: WS send { nonce, text }
    G1->>G1: auth + rate-limit
    G1->>MS: gRPC createMessage
    MS->>DB: INSERT by channel_id, msg_id
    DB-->>MS: ok
    MS->>PS: PUBLISH channel:X {envelope}
    MS-->>G1: ack { message_id, ts }
    G1-->>C: ACK (echoes nonce)
    PS-->>G2: deliver envelope
    G2->>R: WS push MESSAGE_CREATE
          

Ordering guarantee. Clients see messages in the order the Message service wrote them to Cassandra, because message_ids are monotonic and clients sort by message_id. Out-of-order arrivals (e.g., reconnected client replaying) are reordered on the client before render.

Dedup. Client-generated nonce lets the Message service detect and reject replays of the same send. Server-assigned message_id lets clients detect duplicate delivery (pub/sub at-least-once) and collapse.

Huge channels. A 500k-member channel can't fan out to 500k WS conns per message — that's 50k msgs/sec × 500k = 25B events/sec in aggregate. Real answer: only users currently viewing the channel get real-time pushes; everyone else gets an unread-count update and fetches history when they open the channel. Discord calls this "lazy loading" of large guilds.

• Redis pub/sub vs Kafka for fan-out. Redis pub/sub: sub-ms latency, zero durability, easy ops. Kafka: durable, replayable, higher latency (~5–10 ms). Chat demands latency; the "durable replay" lives in Cassandra history, so Redis pub/sub is the right pick. Kafka is still used upstream for analytics + notifications (audit trail).
• Cassandra vs Postgres for messages. Per-channel write patterns are append-only by message_id; Cassandra's partition-by-chan_id gives free sharding + cheap range scans for history. Postgres fails at 4B-row monotonic tables.
• Presence is cross-product expensive. 10M online users × ~200 mutuals = 2B presence events per state change wave. Batch + coalesce on gateway egress: only send presence updates that changed in the last few seconds, bundled per tick.
• Sticky WS sessions. Can't round-robin each WS frame — they belong to a specific conn. Load balancer uses consistent hashing on conn_id; reconnect lands on a different gateway fine (session store holds user→gateway).

1. Name the hard parts fast. "Persistent WebSockets, fan-out, presence." Don't start with sharding the user table — that's trivial here. The gateway tier and fan-out is the actual design.
2. Distinguish durability from delivery. Cassandra holds the durable record; pub/sub is fire-and-forget notify. This decoupling is the whole design — explain it.
3. Huge channels ≠ huge fan-out. Show that you know Discord doesn't actually push messages to 500k WS conns on every send. Viewers get pushed; others get unread counts + history pull on open.
4. Pick the right storage. Cassandra (messages) + Postgres (users/guilds) + Elasticsearch (search) + S3 (attachments). Don't jam it all in one DB.
5. Presence is its own system. It's read-heavy, write-heavy, and not durable. Separate service, separate scale story.