Problem statement
Design a secret rotation service that generates, distributes, and rotates credentials (database passwords, API keys, certificates) for a large workload fleet. Rotation must be zero-downtime — the new secret is provisioned and accepted before the old one is revoked — and a compromise must trigger an emergency rotation that revokes the old material fast, with every operation audited.
Operating context. The service manages secrets for about 100,000 workloads. Each secret has a rotation policy (for example, every 30 days, or on demand). Workloads fetch secrets via short-lived, renewable leases rather than permanent copies. Rotation keeps a two-valid-versions overlap so in-flight clients on the old secret are not broken mid-rotation. Backends (a database, a cloud IAM system, a certificate authority) are pluggable behind a common adapter.
Out of scope. The underlying key-management/HSM internals, the application code that consumes secrets, network policy and firewalling, and human SSO/login. Assume those are handled elsewhere.
What to produce. A high-level architecture covering: secret storage and encryption-at-rest; lease-based distribution and renewal to workloads; the rotation state machine with the two-version overlap and provision-then-revoke ordering; revocation and emergency rotation on compromise; the pluggable-backend adapter; and the immutable audit trail — plus how you avoid an outage when a rotation half-fails. Sketch the components and the flow; we will probe specifics in checkpoints.
Requirements
This assessment is a Premium feature.
The statement above is free to read. The functional and non-functional requirements, and the graded canvas that scores your design against them, come with Premium.
Topics
- System Design HLD
- Observability Secrets
- Security Rotation
- Distribution Lease
- Audit Trail