Problem statement
Design a distributed unique ID generator: a service that hands out 64-bit identifiers to every other service in a large backend, so that rows, events, and messages get a globally unique key the moment they are created (think the ID stamped on a new chat message or order the instant it is written).
Operating context. Dozens of application services call this generator directly (embedded library) or over a thin RPC to mint IDs at row-creation time. Fleet-wide demand is roughly 50,000 IDs/sec sustained with bursts to 200,000/sec at peak, spread across two regions and many data-center racks. Consumers store IDs as a signed 64-bit integer and often sort or range-scan by them, so IDs should be roughly time-ordered (k-sorted), not random. The generator must keep minting even if a coordination store, a whole rack, or an entire region goes dark.
Out of scope. The databases that consume the IDs, authentication of callers, a human-readable slug/short-code layer on top, cross-service schema, and any analytics on ID usage. Assume a separate team owns those.
What to produce. A high-level design that names the ID minting components and the request flow from a caller to a minted ID. Lay out the 64-bit layout you choose (how many bits for time, node/worker identity, and per-tick sequence), how a node gets a unique worker identity without a human hand-assigning it, and your throughput ceiling per node. Then compare the main families — timestamp+node+sequence (snowflake-style), a central range/block allocator, and random 128-bit-truncated-to-64 UUIDs — and justify your pick against the ordering and single-point-of-failure requirements. Finally, address clock skew and backwards clock jumps: what a node does when its wall clock moves backward, and what ordering guarantee you actually promise (strict monotonic per node vs. k-sorted globally). Sketch the components and the mint flow; we will probe specifics at checkpoints.
Functional requirements
- Mint a 64-bit unique ID on request and return it to the caller with sub-millisecond added latency.
- Guarantee every minted ID is unique across the whole fleet, both regions, and all nodes.
- Produce IDs that are roughly time-ordered so consumers can range-scan and sort by them.
- Assign each generator node a distinct worker identity automatically, without a human configuring per-node IDs.
- Keep minting IDs on a node even while the central coordination store is temporarily unreachable.
Non-functional requirements
- Sustain 50,000 IDs/sec fleet-wide with bursts to 200,000/sec; a single node mints at least 4,000 IDs/sec.
- Added mint latency p99 < 1 ms for the embedded path; p99 < 5 ms for the RPC path.
- No single point of failure: loss of any one node, rack, or coordination store must not stop minting.
- Availability of the mint path >= 99.99%; IDs remain unique even during a region failover.
- IDs fit in a signed 64-bit integer for the design's full stated epoch lifetime (decades), with no overflow in that window.
- Ordering guarantee is explicit and honored: strict-monotonic per node, k-sorted globally within clock-skew bounds.
Topics
- System Design HLD
- Reliability Replication
- Consistency Eventual
- Scaling Write-Heavy
- Reliability Idempotency