Design a Saga Coordinator

Design the platform service that runs distributed business transactions across many microservices using the saga pattern. A single logical operation — say, booking a trip that reserves a flight, a hotel, and a car — spans several services that each own their own database, so there is no shared transaction. The coordinator drives the steps in order and, if a later step fails, invokes compensating actions to undo the earlier ones.

Operating context. Steps call independently-owned services that can be slow, can fail transiently, and can be temporarily down; a step may take milliseconds or hours (awaiting a human approval or a third party). The coordinator itself can crash and restart at any point and must resume every in-flight saga exactly where it left off. Step handlers are expected to be idempotent because the coordinator may retry them. There is no global rollback — only forward progress or compensation.

Out of scope. The internal logic of each participant service, the saga-definition authoring tooling, exactly-once messaging infrastructure (assume an at-least-once bus exists), and analytics on saga outcomes. Assume other teams own those.

What to produce. A high-level architecture covering: how a saga definition is modeled and executed, how saga state is persisted so it survives a coordinator crash mid-saga, how steps are dispatched and retried against flaky services, how compensation is triggered and ordered on failure, how long-running steps are handled without holding resources, how duplicate dispatches are made safe, and how you scale to many concurrent sagas across coordinator instances. Sketch the components and flow; we will probe specifics in checkpoints.