Problem statement
Design a service that aggregates massive geolocated event streams into spatial heatmaps — density by cell — queryable at multiple zoom levels and time windows for operational dashboards and map overlays.
Operating context. Events (demand, incidents, activity) stream in continuously and are geotagged. Dashboards ask for a heatmap tile: give me cell densities for this bounding box, at this zoom, over this time window, optionally filtered to one event type. Both a live view (the last few minutes) and historical views (arbitrary past windows) are needed. The volume forbids scanning raw events per query, so aggregates are precomputed at several resolutions.
Out of scope. Client-side rendering and color mapping of the tile, the raw event producers, per-user personalization of what is shown, and any predictive/forecasting model. Assume producers publish clean geotagged events to your ingest.
What to produce. A high-level architecture covering: event ingestion; the spatial binning scheme (geohash / H3 / S2) and how cells roll up across zoom levels; time-windowed streaming aggregation (tumbling/sliding windows) for the live path versus batch rollups for history; the aggregate storage model keyed by cell, zoom, window, and type; tile query serving with caching and how a bounding box maps to cells; and how the system degrades under load. Sketch the components and the flow from event to a served tile; we will probe specifics in checkpoints.
Must do 5
Functional requirements
- Ingest geotagged events (lat, lng, type, weight, timestamp) at high volume.
- Aggregate event counts and weights into spatial cells at several zoom resolutions.
- Serve a heatmap tile of cell densities for a bounding box, zoom, and time window.
- Support both a live window (last few minutes) and arbitrary historical windows.
- Let a caller filter aggregation to one or more event types before serving the tile.