Design the shared API gateway that sits in front of a fleet of internal microservices (think the single front door a large consumer app puts between the public internet and its dozens of backend teams). Every external request lands here first; the gateway authenticates it, enforces per-caller limits, routes it to the right upstream, and shapes the response before it goes back out.

Operating context. The platform runs ~200 backend services owned by ~40 teams, each service registered in a service-discovery system that changes membership continuously (deploys, autoscaling, health flaps). The gateway fronts ~80k RPS at steady state and ~200k RPS at peak, with a long-tail of upstreams from 5 ms to 800 ms. Callers are a mix of first-party mobile/web clients (JWT) and third-party partners (API keys with per-plan quotas). Teams must be able to attach cross-cutting behavior (auth, transforms, header injection, WAF rules) per route without redeploying the gateway binary, and a bad upstream must never take the whole front door down.

Out of scope. The upstream services' own internals, the identity provider that mints tokens (assume it exists and exposes a JWKS endpoint), the CDN / static-asset edge, TLS certificate issuance, and the billing system that turns quota overages into invoices. Assume a separate team owns each.

What to produce. A high-level architecture covering: the request/response flow through the gateway (the ordered pipeline of stages), how routes map to upstreams and how the gateway stays in sync with service discovery, how authentication and per-caller rate limiting work at this scale, how request/response transforms and the pluggable per-route middleware are modeled and configured, how circuit breaking and load balancing protect the gateway from a sick upstream, and the observability story (metrics, tracing, logs). Sketch the major components and the data/control planes; we will probe specifics during checkpoints.