Problem statement
Design a message-search service that lets each user run full-text search over their own chat history, every one-to-one and group conversation they belong to, and jump straight to a matching message. Results must reflect messages sent moments ago, and must drop messages that were later edited or deleted.
Operating context. You are indexing a firehose of chat messages: roughly 50,000 new messages/sec, roughly four billion per day, arriving from the messaging core as an ordered event stream. Search is strictly private, so a query only ever matches messages in conversations the querying user is a member of, and a single popular group's messages must appear in the searchable set of every member. Users expect a query to return in a few hundred milliseconds and to surface a message they sent seconds ago. Edits, deletes, and legal redactions must remove or update the corresponding index entry promptly.
Out of scope. Relevance-ranking ML beyond keyword match plus recency, global or cross-user search, indexing of media contents (OCR or transcription), and the message-delivery path itself. Assume membership and access checks are provided by a separate service you can call.
What to produce. A high-level architecture covering: the ingestion pipeline from the message stream into a search index, how the index is partitioned and how a group message reaches every member's searchable set without exploding storage, the query path and how you enforce per-user access, how edits and deletes propagate to the index, and the freshness-versus-cost trade-off. Sketch the components and the data flow; checkpoints will probe the group-message fan-out and the delete path.
Requirements
This assessment is a Premium feature.
The statement above is free to read. The functional and non-functional requirements, and the graded canvas that scores your design against them, come with Premium.
Topics
- System Design HLD
- Messaging Search
- Data Inverted-Index
- Scaling Write-Heavy
- Data Partitioning