Problem statement
Design the service that ingests parcel-tracking updates from many shipping carriers, normalizes their differing status vocabularies into one canonical timeline, and serves customer-facing tracking plus proactive notifications.
Operating context. Dozens of carriers report progress differently: some push webhooks, some must be polled on a schedule, and each has its own status codes, formats, and reliability. The service maps every update onto a canonical shipment state (label created, in transit, out for delivery, delivered, exception) and maintains an ordered timeline per tracking number. Updates arrive late, out of order, and duplicated. Customers open a tracking page expecting the latest state, and the service pushes a notification on meaningful transitions. Scale: about 100 million active shipments and roughly 40000 tracking events/sec at peak.
Out of scope. Carrier rate-shopping and label purchase, the notification rendering/delivery channels themselves, returns logistics, and address validation. Assume carriers provide webhook and/or polling endpoints and a separate service delivers the notifications you enqueue.
What to produce. A high-level architecture covering: how heterogeneous carrier inputs (webhook and polled) are ingested uniformly, how raw updates are normalized to the canonical timeline, how out-of-order and duplicate updates are ordered and de-duplicated, how the customer read path serves current status at low latency, how meaningful transitions fan out to notifications, and how an unreliable carrier is handled (webhook stall to polling fallback). Sketch the components and trace a raw carrier update from ingestion to updated timeline and a fired notification.
Functional requirements
- Ingest tracking updates from many carriers via both webhooks and scheduled polling under one internal model.
- Normalize each carrier's status vocabulary into a canonical shipment state and per-tracking-number timeline.
- Order out-of-sequence updates and de-duplicate repeated carrier events on the timeline.
- Serve the current status and full history for a tracking number to customers.
- Emit a notification on meaningful state transitions (shipped, out for delivery, delivered, exception).
- Fall back to polling for a carrier whose webhooks stall, and back off when a carrier is unreliable.
Non-functional requirements
- Track 100 million active shipments and ingest 40000 tracking events/sec at peak.
- Customer tracking read p99 < 150 ms; notification fired within 60 s of a carrier update.
- Tolerate late, out-of-order, and duplicate updates without corrupting the timeline.
- 99.9% availability; at-least-once notification delivery with per-transition de-duplication.
- Retain shipment history 18 months (~30 TB) and support lookup by tracking number.
Topics
- System Design HLD
- Commerce Shipping
- Integration Webhooks
- Data Normalization
- Events Streaming