Idempotency
Doing the same thing twice should be the same as doing it once.
Introduction
An operation is idempotent if performing it multiple times has the same observable effect as performing it once. In distributed systems, where retries and duplicate deliveries are unavoidable, idempotency is what stops a retry from becoming a second charge, a duplicate order, or a doubled balance.
Why it exists
Networks are unreliable. A client sends a request, the server processes it, but the response is lost — so the client retries. Message brokers deliver at-least-once, so consumers see duplicates. Without idempotency, these normal, expected events cause real damage: charged twice, shipped twice, emailed twice. Idempotency exists to make retries safe, which in turn makes reliable systems possible.
Analogy
A well-designed elevator button is idempotent: pressing 'call' five times summons one elevator, not five. Pressing it again while it's already coming changes nothing. Contrast a naive 'add $10' button pressed five times — you'd owe $50. Good operations behave like the elevator button.
How it works
Some operations are naturally idempotent: SET balance = 100 (absolute) or DELETE user 42 produce the same state no matter how many times you apply them. Others are not: balance = balance + 10 (relative) or INSERT order create new effects each time.
To make a non-idempotent operation safe, the client sends an idempotency key — a unique token per logical operation (e.g. a UUID per checkout attempt). The server records the key with the result of the first execution. On a retry with the same key, it returns the stored result instead of executing again. The key must be stored atomically with the effect (same transaction) or you re-open the duplicate window. HTTP semantics reflect this: GET/PUT/DELETE are defined as idempotent; POST is not, which is why payment POSTs carry an idempotency key.
- 1
Client generates a unique idempotency key for the operation and sends it with the request.
- 2
Server checks the store: has this key been seen?
- 3
If new, it executes the operation and records (key → result) atomically with the effect.
- 4
If seen, it returns the previously stored result without re-executing.
- 5
The client can safely retry indefinitely; the outcome is identical each time.
Interactive
Architecture
Idempotency key deduplicates a retried request
Check the key
Dedupe store (atomic with effect)
First call: execute + store (key, result) atomically. Retry with the same key: return the stored result, don't charge again.
In code
export async function charge(req: ChargeRequest, idempotencyKey: string) {
return db.transaction(async (tx) => {
// Atomic claim: unique constraint on idempotency_key.
const existing = await tx.idempotency.findByKey(idempotencyKey)
if (existing) return existing.result // retry → return stored result
const result = await processPayment(tx, req) // the real side effect
// Stored in the SAME transaction as the effect — no duplicate window.
await tx.idempotency.insert({ key: idempotencyKey, result })
return result
})
}public async Task Handle(OrderPlaced evt)
{
// Dedupe on the message/event id; INSERT ... ON CONFLICT DO NOTHING.
var isNew = await _processed.TryMarkAsync(evt.EventId);
if (!isNew) return; // duplicate delivery → no-op
await _orders.CreateAsync(evt); // safe: runs at most once per EventId
}In the real world
Where you've seen this
Stripe's API requires an Idempotency-Key header on charge requests exactly for this reason: a client whose connection drops can safely retry the charge, and Stripe returns the original result rather than charging twice. The same pattern underlies 'exactly-once' processing in event systems, which is really at-least-once delivery plus idempotent consumers.
When to use it
Reach for it when
- Any operation with side effects reachable over an unreliable network (payments, orders).
- Consumers of at-least-once message brokers (Kafka, SQS, RabbitMQ).
- Public APIs where clients will retry on timeout.
- Retryable background jobs and webhooks.
Avoid it when
- Naturally idempotent operations (absolute SET, DELETE) that need no extra machinery.
- Pure reads with no side effects.
- When the added dedupe store genuinely isn't worth it for a low-stakes, easily-corrected action.
Trade-offs
Advantages
- Makes retries and duplicate deliveries safe — the foundation of reliability.
- Enables 'exactly-once' semantics on top of at-least-once infrastructure.
- Simplifies clients: they can retry blindly on failure.
Disadvantages
- Requires storing and expiring keys (extra state).
- The key must be scoped and generated correctly by the client.
- Atomicity between key and effect is easy to get subtly wrong.
- Keys have a retention window; very late retries may re-execute.
Idempotency trades a little extra state and complexity for the ability to retry safely — almost always worth it for side-effecting operations. Key design is itself a trade-off: client-generated keys are correct but require client cooperation; natural keys (order id) avoid storage but only work when a stable business identifier exists. Retention is a trade-off too: keep keys long enough to cover realistic retries, but not forever. And 'exactly-once' is really 'at-least-once delivery + idempotent processing' — pretending otherwise leads to fragile designs.
| Delivery/processing guarantees | At-most-once | At-least-once | Exactly-once (effective) |
|---|---|---|---|
| Duplicates | Never | Possible | Deduped away |
| Lost messages | Possible | Never | Never |
| How | Fire and forget | Retry until ack | At-least-once + idempotent consumer |
| Use when | Metrics you can lose | Most business events | Payments, orders, money |
Common mistakes
Watch out for
- Storing the idempotency key in a separate step from the effect, re-opening the duplicate window.
- Generating the key server-side per request (so retries look new).
- Reusing one key for logically different operations.
- No unique constraint, allowing two concurrent first-attempts to both execute.
- Confusing at-least-once delivery with exactly-once — the consumer must still dedupe.
Failure thinking
What breaks it
A request times out but actually completed successfully. What happens with and without idempotency?
The client retries. Without idempotency, the server executes again — double charge. With an idempotency key, the server recognizes the key, skips execution, and returns the original result. This exact scenario (lost response, not lost request) is why idempotency matters more than tuning timeouts.
A consumer processes the same message twice (at-least-once delivery). How do you stay correct?
The consumer must be idempotent: dedupe on the message/event id before applying the effect, ideally recording the id atomically with the write. Then a redelivery is a no-op. Trying to achieve exactly-once purely at the broker is a trap — the practical answer is at-least-once + idempotent consumer.
Think like a senior
Senior Engineer Insight
The classic senior line: 'retries without idempotency can duplicate side effects.' If you propose retries anywhere, the very next sentence should be how the target is made idempotent.
Senior Engineer Insight
Push idempotency to the layer that owns the side effect and store the key in the same transaction as the effect. Bolting dedupe onto a separate cache leaves a crash window that reintroduces duplicates.
Remember
Retries without idempotency can duplicate side effects.
Remember
Store the idempotency key atomically with the effect, or the dedupe is a lie.
Interview questions
Design an idempotent payment endpoint.
Active recall
Check yourself
Your payment API receives the same request twice because the client timed out and retried. What is the best protection against a double charge?
Practical challenges
Idempotency-Key payment endpoint
Build a payment endpoint that is safe under duplicate and concurrent retries.
Requirements
- Accept an Idempotency-Key header.
- Execute the charge and persist the key+result in one transaction.
- Return the original result on any retry.
Constraints
- A unique constraint on the key.
- Must handle two concurrent first-attempts.
Acceptance criteria
- Sequential retries → one charge, same response.
- Concurrent duplicates → one charge, both callers get the same result.
Edge cases
- Crash between charge and key insert (should be impossible if atomic).
- Key reused for a different amount.
Bonus
- Add a 24h retention/expiry for keys.
- Return a clear response for an in-flight duplicate.
Reflection
- Why must the key be stored in the same transaction as the charge?
Architecture challenge
A checkout calls Payment, which emits a PaymentSucceeded event consumed by Order and Email services over an at-least-once broker. Design idempotency end to end so that a client retry, a Payment redelivery, and an Order redelivery together can never double-charge, create two orders, or send two receipts. Where does each dedupe live, and what is the key at each hop?
Flashcards
Summary
Idempotency makes repeating an operation harmless, which is essential because retries and duplicate deliveries are guaranteed in distributed systems. Naturally idempotent operations need nothing; for the rest, a client-supplied idempotency key stored atomically with the effect deduplicates retries. 'Exactly-once' processing is simply at-least-once delivery plus an idempotent consumer.
Key takeaways
- Idempotent = doing it N times equals doing it once (observably).
- Use an idempotency key stored in the same transaction as the side effect.
- At-least-once delivery + idempotent consumer = effective exactly-once.
- Never add retries without making the target idempotent.
Your notes
Saved to this device