Design a distributed message queue: a durable, horizontally scalable log service that lets producer applications publish records to named topics and lets independent consumer applications read them back in order (think the backbone that decouples microservices at a busy commerce company).

Operating context. This is a multi-tenant internal platform. Roughly 500 topics, the busiest carrying 1,000,000 records/sec at peak (~1 KB each, so ~1 GB/sec ingest on the hot topic) while most are near-idle. Records must be retained for 7 days by default, with some compacted-forever topics that keep only the latest value per key. Producers and consumers run in three availability zones; a single broker or disk loss must not lose an acknowledged write. Read traffic is a mix of real-time tail consumers (milliseconds behind head) and batch consumers that replay from an arbitrary offset hours in the past.

Out of scope. The client SDK ergonomics, the exact on-disk file format, cross-datacenter (multi-region) mirroring, authn/authz and encryption, schema registry, and the dead-letter-queue application semantics. Assume a separate team owns those.

What to produce. A high-level architecture covering: the partitioned-log data model and how a topic maps to partitions and brokers; how producers route and how ordering + monotonic offsets are guaranteed within a partition; the consumer-group model and how partitions are assigned and rebalanced when members join or leave; the replication + durability story (leader/follower, in-sync replicas, acknowledgement levels) and what "committed" means; the delivery-semantics spectrum (at-least-once vs exactly-once, and what each costs); and the retention + log-compaction lifecycle. Sketch the major components and the write/read/commit flows between them; we will probe specifics at checkpoints.