Design a Recommendation Service

Design a recommendation service that picks, in real time, a small ranked list of items to show each user (think the "For You" feed of a large media or shopping app) out of a catalog far too big to score exhaustively per request.

Operating context. The catalog holds ~50M live items and churns constantly — thousands of new items appear per minute and must become recommendable within minutes, not the next day. Around 40M daily active users generate 30k feed requests/sec at peak, each asking for the top ~20 items. User and item signals (clicks, dwell, purchases) stream in continuously; some are behavioral (updated per event) and some are heavy embeddings (recomputed on a batch cadence). The serving path has a strict p99 latency budget of 150 ms end to end, and every request must return something — a blank feed is worse than a mediocre one.

Out of scope. The actual ML model architectures and loss functions, the training-data labeling pipeline, A/B experiment assignment and metrics, ads/sponsored insertion, and content moderation. Assume separate teams own those; treat models as trained artifacts you deploy and query.

What to produce. A high-level architecture for the online serving path and the offline/near-line pipelines that feed it. Cover: the two-stage funnel (cheap candidate generation over the full catalog, then expensive ranking of a shortlist), the feature store and how the same features stay consistent between offline training and online serving, how freshly created items and brand-new users are handled (cold-start and fallback), how signal freshness is kept within budget, and how the 150 ms request is spent across stages. Sketch the major components and the request flow between them; we will probe specifics at the checkpoints.