Problem statement
Design the pipeline that consumes a global firehose of public posts, tallies hashtag usage over rolling time windows, and surfaces the top trending hashtags per region in near-real-time.
Operating context. Posts arrive as a continuous stream; each carries zero to a handful of hashtags and an origin region. Trends must reflect momentum, not just raw popularity — a tag jumping from a few hundred to tens of thousands of uses in minutes is trending, while a tag that is steadily huge every day is not. Rankings are recomputed on a tight cadence and read by a very large client population. The tag vocabulary is enormous and long-tailed, so keeping an exact live counter per distinct tag is not affordable.
Out of scope. Bot and spam detection, personalized or per-follower trends, semantic clustering of near-duplicate tags, translation, and the durable storage of the posts themselves. Assume upstream teams own those.
What to produce. A high-level design of the whole path from ingest to serving: how you shard and buffer the incoming stream, your windowing model (sliding vs tumbling and how old counts decay), how you compute a trend/velocity score, what approximate-counting structures keep memory bounded while staying exact for the true heavy hitters, how per-region counts roll up into a global view, and the serving and caching layer that answers top-N reads. Sketch the components and the data flow between them; checkpoints will probe the counting and scoring choices.
Requirements
This assessment is a Premium feature.
The statement above is free to read. The functional and non-functional requirements, and the graded canvas that scores your design against them, come with Premium.
Topics
- System Design HLD
- Stream Processing
- Data Approximate-Counting
- Scaling Write-Heavy
- Social Trends