Problem statement
Design the backend that captures user-interaction events from web and mobile clients at very high volume, moves them through a resilient pipeline, and turns them into both near-real-time dashboards and accurate daily rollups (think the behavioral-analytics platform a large consumer product runs on). The core tension is that the same raw event stream must feed a fast, approximate live view AND a slow, precise accounting view, while surviving duplicate deliveries, out-of-order arrival, and events that show up minutes or hours late.
Operating context. Around 80 million monthly active users generate clicks, page views, scrolls, and custom events across web and native apps. Steady state is roughly 1.5 million events/sec, averaging ~400 bytes each (~600 MB/sec), with 4-5x spikes during product launches or marketing pushes. Clients batch events and retry on flaky networks, so the same event can arrive several times; mobile clients that were offline can flush a backlog hours later, so events routinely arrive out of order and late. Analysts need funnels, active-user counts, and per-page engagement; the business needs exactly-once daily counts that reconcile with billing and reporting.
Out of scope. The client-side SDKs and their on-device batching, the consent/privacy and PII-redaction policy (assume a redaction hook runs at the edge), the BI/dashboard rendering layer, the experimentation/A-B assignment engine, and ML recommendation models built on the events. You own everything from the ingest endpoint onward.
What to produce. A high-level architecture covering: the ingest edge and the durable buffer that absorbs bursts; the streaming path for low-latency approximate metrics and the batch path for exact reprocessable counts, and how you reconcile the two (lambda vs kappa — pick and defend); how raw events are deduplicated given at-least-once delivery and how you handle late and out-of-order events with watermarks and bounded lateness; how you group a user's events into sessions with an inactivity gap and repair sessions when a late event lands; the aggregation store that serves live dashboards and the tables that hold reconciled daily rollups; and how the daily accounting achieves exactly-once results even though the transport is at-least-once. Sketch the components and the end-to-end flow; checkpoints will probe sessionization, late-event handling, and exactly-once accounting specifically.
Requirements
This assessment is a Premium feature.
The statement above is free to read. The functional and non-functional requirements, and the graded canvas that scores your design against them, come with Premium.
Topics
- System Design HLD
- Infra Queue
- Scaling Write-Heavy
- Consistency Eventual
- Reliability Idempotency