Idempotency keys
The customer taps Pay, the request times out, and the app retries. Is the customer charged once or twice?
In 30 seconds
- The problem
- A timeout looks the same whether the bank never got the request, charged and lost the reply, or is just slow. A plain retry can charge the customer twice.
- Option A: retry the call
- Call the bank again on a timeout. Simple, and it charges twice whenever the first call actually went through.
- Option B: idempotency key, three phases
- Record the key before the call, store the result after, and answer a retry with the stored result. This is what Airbnb runs.
- Common choice
- An idempotency key on every payment, with Airbnb's three phases: record, call, record the result.
- In an interview
- Every payment carries an idempotency key the client creates once and reuses on retry. Claim the key in the database before calling the bank, keep the bank call outside any transaction, store the result after, treat a timeout as unknown, and read keys from the primary.
Why it matters
A charge is the one write you cannot cheaply take back. The network between your server and the bank can drop the request, drop the reply, or be slow, and from your side all three are the same timeout.
Airbnb built a library called Orpheus for exactly this, used across its payment services, and reports five nines of consistency while its payment volume doubled. Each of this week's traps is one of its rules being skipped.
Words you need
- Idempotent
- Safe to repeat. A retried request ends in the same state as the first attempt, instead of an error or a second effect.
- Idempotency key
- A unique id the client creates once per payment and sends with every retry of it. The server uses it to recognise a retry and return the first result instead of charging again.
- Timeout
- Giving up on waiting for a reply. It tells you the reply did not arrive in time, not whether the work was done.
- Transaction
- A group of changes the database treats as one: all of them are saved, or none of them.
- Unique index
- A database rule that rejects a second row with the same value. A safety net: it turns a silent duplicate into an error.
- Read replica
- A copy of the database that follows the primary a moment behind. Fine for most reads, wrong for anything that must see the latest write.
Picture it
The customer pays ₹2,000. The bank charges the card, but the reply is lost and the app times out.
Option A: just retry
- Pay → bank charges ₹2,000
- The reply is lost: timeout
- The app retries
- Bank charges ₹2,000 again
₹4,000 taken for one order. The classic "paise kat gaye" support ticket.
Option B: idempotency key
- Pay with key k1 → key recorded
- Bank charges ₹2,000, reply lost
- The app retries with the same k1
- Server sees k1, returns the stored result
Charged once. The retry is safe and cheap.
The options
Option A: call the bank, retry on timeout
charge(); on timeout → charge() againGood: Nothing to build. Works whenever the first call really failed.
Cost: When the first call went through and only the reply was lost, the customer pays twice. Not retrying doesn't help: they tap Pay again.
Option B: idempotency key with three phases
claim key → call bank → store resultGood: A retry with the same key gets the first result back. The customer is charged once however many times the request arrives.
Cost: Every rule has to hold: claim the key atomically, no bank call inside a transaction, timeouts treated as unknown, keys read from the primary. Skipping any one is a trap this week.
The choice and its cost
The client generates an idempotency key once per payment, saves it before calling, and sends the same key with every retry. The server runs Airbnb's three phases: record the request and key (no network calls), call the bank (no database work), record the result (no network calls).
Airbnb's rules make it hold: keys live on the primary database, never a replica; each request holds an expiring lease on its key, longer than the bank call's timeout; 5XX and infrastructure errors are retryable, 4XX validation errors are not; and the client never changes the payload between retries.
What it costs
Interviewers also listen for: what the key is
Check yourself
1Why is a timeout dangerous on a payment?Show answerHide
It doesn't tell you whether the bank charged the card. The request may have failed, or only the reply was lost.
2Why must the key be claimed before the bank call, not after?Show answerHide
If two retries arrive together and both check first, both see nothing and both charge. Claiming the key in one statement lets only one through.
3Why does Airbnb read idempotency keys from the primary only?Show answerHide
A replica can lag behind. A retry that reads it may not see the stored result and charge again.
The code
Every trap in this world is runnable
The design notes are in the repo, and each trap that follows ships with the bug, the fix and a test that proves it.
Remember
A timeout means unknown, not failed. Give every payment a key, claim it before the bank call, and answer retries from what you stored.