Design a Real-Time Chat Backend

Design the backend for a real-time 1:1 chat product (think a popular consumer messaging app) that carries text messages between two users over persistent connections and reflects delivery and read status back to the sender within a second.

Operating context. You are building the messaging core, not the UI. Each user runs one or more clients (phone, web) that hold a live connection when the app is open. A conversation is strictly between two people. Users are frequently offline: when the recipient is not connected, their messages must be queued durably and drained in order the moment they reconnect. The system carries about 40,000 messages/sec sustained with peaks near 120,000/sec, has roughly 5 million concurrently connected sockets, and must keep the send-to-delivered path under p99 300 ms for online recipients. Store the full message history durably.

Out of scope. Large group chats and broadcast channels, end-to-end encryption / key exchange, media upload and transcoding (assume a separate blob service returns a URL you just reference), spam and abuse detection, voice / video calls, and push-notification vendor integration (assume a notifier component exists that you call).

What to produce. A high-level architecture covering: the connection / gateway tier that terminates the persistent sockets and how clients are routed to their session; the send path from sender to the recipient's device(s), including how you decide fan-out on write versus fan-out on read for a two-party thread; the durable offline queue and how it is drained in per-conversation order on reconnect; how delivery and read receipts propagate back to the sender; the message data model and its partitioning key; and how you achieve at-least-once delivery with client-side dedupe and stable per-conversation ordering. Sketch the major components and the request flow between them; we will probe specifics during checkpoints.