Problem statement
Design the platform service that meters and enforces per-customer API usage quotas for a large multi-tenant SaaS (for example, a plan that grants 5 million API calls per billing month). Unlike a per-second rate limiter, this service tracks cumulative consumption against a period budget, drives both soft warnings and hard cut-offs, and feeds billing — so its counts must be accurate enough to charge money against.
Operating context. Every tenant belongs to a plan tier with a monthly quota; some endpoints cost more than one unit per call. The enforcement check sits inline on the request path across a global fleet of gateway nodes, so the added latency must be tiny. Counters roll over at each tenant's billing anchor, which may sit in a different timezone. Usage must ultimately reconcile with the invoicing system to the unit.
Out of scope. Per-second burst throttling (a separate rate limiter owns that), the billing and invoicing system itself, plan/pricing configuration UI, and fraud or abuse detection. Assume other teams own those.
What to produce. A high-level architecture covering: the inline check path and how it stays fast, how usage is counted and aggregated across many gateway nodes without a global lock, the storage and partitioning of counters, how you trade freshness against throughput (local pre-aggregation, sync interval, and the resulting over-shoot window), how billing-period resets are handled per tenant, how the service degrades when the counter store is unreachable, and how counts reconcile with billing. Sketch the components and the flow between them; we will probe specifics in checkpoints.
Functional requirements
- Given a tenant and a weighted call, return allow or deny plus remaining quota for the current billing period.
- Aggregate consumption reported by many gateway nodes into an authoritative per-tenant, per-period counter.
- Support weighted endpoints where one call debits more than one quota unit.
- Reset each tenant's counter at its own billing-period anchor and timezone.
- Emit finalized usage records that reconcile with the billing system at period close.
Non-functional requirements
- Sustain 500,000 enforcement checks/sec globally at steady state, 1,000,000/sec at peak.
- Inline check adds p99 < 5 ms to the request path.
- 99.99% availability for the enforcement decision; fail-open or fail-closed must be a deliberate, stated choice.
- Bounded over-shoot: a tenant may exceed quota by at most a stated small margin before the hard cut-off engages.
- Finalized period totals accurate to the unit for billing; interim reads may be eventually consistent within a few seconds.
- Support 1,000,000 active tenants with counters retained for at least 13 billing periods.
Topics
- System Design HLD
- Platform Metering
- Data Counters
- Scaling Multi-Tenant
- Consistency Eventual