Problem statement
Design the system that produces the periodic activity digest email a collaboration product sends — the "here's what happened while you were away" summary bundling your mentions, comment replies, and task updates into one message on a daily or weekly cadence.
Operating context. Activity events stream in continuously across the whole user base. Periodically, a batch run assembles one digest per eligible user from everything they have not yet been told about, ranks and groups the items, and sends at a time appropriate to the user's local morning. Users with nothing new get nothing. The run is huge and time-boxed, and it will be retried on partial failure — so it must be idempotent: a re-run must not double-send anyone or drop them, and no activity item may appear in two digests.
Out of scope. Instant/real-time notifications (a separate path), the email template design and rendering, the SMTP/delivery transport internals, and spam/bounce reputation management. Assume a delivery service accepts a rendered per-user message and best-effort sends it.
What to produce. A high-level architecture covering: the activity-accumulation store and how you window "since the last digest"; the batch orchestration across hundreds of millions of users within a send window, including time-zone waves; per-user assembly, ranking, and empty-skip; the idempotency/watermark scheme that guarantees at-most-once inclusion under retries; and the send fan-out at high throughput. Sketch the components and the flow of one daily run; we will probe specifics in checkpoints.
Must do 5
Functional requirements
- Collect the per-user activity events accumulated since that user's last digest.
- Assemble a per-user digest that groups and ranks items by relevance for a scheduled send.
- Send each user's digest on their configured cadence and at their local send-time.
- Skip users with no new activity and respect their notification and digest preferences.
- Ensure each activity item appears in at most one digest, with no duplicates across runs.