Design a Message Search Index

Design a message-search service that lets each user run full-text search over their own chat history, every one-to-one and group conversation they belong to, and jump straight to a matching message. Results must reflect messages sent moments ago, and must drop messages that were later edited or deleted.

Operating context. You are indexing a firehose of chat messages: roughly 50,000 new messages/sec, roughly four billion per day, arriving from the messaging core as an ordered event stream. Search is strictly private, so a query only ever matches messages in conversations the querying user is a member of, and a single popular group's messages must appear in the searchable set of every member. Users expect a query to return in a few hundred milliseconds and to surface a message they sent seconds ago. Edits, deletes, and legal redactions must remove or update the corresponding index entry promptly.

Out of scope. Relevance-ranking ML beyond keyword match plus recency, global or cross-user search, indexing of media contents (OCR or transcription), and the message-delivery path itself. Assume membership and access checks are provided by a separate service you can call.

What to produce. A high-level architecture covering: the ingestion pipeline from the message stream into a search index, how the index is partitioned and how a group message reaches every member's searchable set without exploding storage, the query path and how you enforce per-user access, how edits and deletes propagate to the index, and the freshness-versus-cost trade-off. Sketch the components and the data flow; checkpoints will probe the group-message fan-out and the delete path.