Design a Publish-Subscribe System

Design a multi-tenant publish-subscribe messaging service that lets producers publish messages to named topics and lets many independent subscribers each consume those messages reliably (think the eventing backbone that other engineering teams build their pipelines on).

Operating context. Internal teams create topics; each topic has zero or more subscriptions, and every subscription is a fully independent consumption cursor — a message published once must be delivered to each subscription at least once. Fan-out is wide and skewed: a hot topic can have 1000+ subscriptions while most have a handful. Producers push roughly 50k messages/sec aggregate, average payload 4 KB, with a topic-level burst factor of 10x for a few minutes. Subscribers vary wildly in speed: some drain within milliseconds, others are slow HTTP endpoints or fall offline for minutes. Delivery is push (the service POSTs to a subscriber endpoint) or pull (the subscriber long-polls); design for at-least-once with visible tradeoffs. Retained backlog per subscription is bounded (7 days or a size cap, whichever first).

Out of scope. Exactly-once end-to-end semantics, the topic/subscription admin UI, cross-region replication and disaster recovery, authn/authz and quota billing, and message schema validation/registry. Assume separate teams own those.

What to produce. A high-level architecture covering: the publish and consume request paths (both push and pull), the data model for topics/subscriptions/messages and how they are partitioned, the fan-out mechanism that turns one publish into per-subscription delivery, the per-subscriber ack + retry + redelivery machinery, the ordering guarantee you offer and its blast radius, how you apply backpressure when a subscription cannot keep up, and where poison messages go (dead-letter). Sketch the major components and the flow between them; we will probe specifics at checkpoints.