Problem statement
Design the platform service that reliably turns a service's local database writes into published events. Producer services follow the transactional-outbox pattern: inside the same database transaction that changes business state, they insert an event row into an outbox table. Your dispatcher is what reads those rows and publishes them to the message bus, guaranteeing that every committed change is eventually delivered — solving the classic dual-write problem where a crash between the DB commit and the publish would otherwise lose an event.
Operating context. There are many producer databases, sharded, each with its own outbox table. Because a database transaction cannot atomically also write to the message bus, the only source of truth is the committed outbox row. Delivery is at-least-once, so the same event may be published more than once and consumers dedupe. Events for the same aggregate (say, one order) must arrive in the order they were written. The dispatcher and the bus can each fail and restart without dropping a committed event.
Out of scope. The producers' business logic and schema, the message bus internals, consumer-side processing, and schema-registry / event-format governance. Assume other teams own those.
What to produce. A high-level architecture covering: how outbox rows are detected and read (polling vs log-tailing/CDC) and the trade-off, how at-least-once publish is guaranteed with no lost committed event, how per-aggregate ordering is preserved through publish, how rows are marked done and cleaned up, how the dispatcher scales across many shards without double-publishing the same row, and how it behaves when the bus is down or the dispatcher crashes mid-batch. Sketch the components and flow; we will probe specifics in checkpoints.
Requirements
This assessment is a Premium feature.
The statement above is free to read. The functional and non-functional requirements, and the graded canvas that scores your design against them, come with Premium.
Topics
- System Design HLD
- Platform Messaging
- Patterns Outbox
- Delivery At-Least-Once
- Consistency Ordering