Design a backfill orchestrator that reprocesses large historical datasets — re-running a pipeline over months of past data after a bug fix or a new derived column — without disrupting the live pipeline that is still processing current data.

Operating context. A production pipeline continuously processes fresh data. Occasionally the team must recompute historical output over a large range — thousands of partitions, petabytes — using updated logic. The backfill runs alongside live processing on shared, finite compute; it must be resumable, must not corrupt or block live output, must replace historical results atomically, and must be throttled so live latency barely moves. Progress tracking and observability are essential over a job that may run for days.

Out of scope. The transformation logic, the compute engine, the dataset catalog, and how downstream consumers are notified of restated data.

What to produce. A high-level architecture covering: chunking the backfill range into independently schedulable, idempotent, resumable units, the orchestration state machine tracking each chunk's progress durably, throttling that yields to live-pipeline priority and is adjustable at runtime, writing output to a staging location and atomically publishing it over the historical output per partition, idempotency and resume-after-failure, and progress / ETA reporting. Sketch the components and flow; we will probe specifics at checkpoints.