Let’s talk

Idempotency keys

The customer taps Pay, the request times out, and the app retries. Is the customer charged once or twice?

In 30 seconds

The problem
A timeout looks the same whether the bank never got the request, charged and lost the reply, or is just slow. A plain retry can charge the customer twice.
Option A: retry the call
Call the bank again on a timeout. Simple, and it charges twice whenever the first call actually went through.
Option B: idempotency key, three phases
Record the key before the call, store the result after, and answer a retry with the stored result. This is what Airbnb runs.
Common choice
An idempotency key on every payment, with Airbnb's three phases: record, call, record the result.
In an interview
Every payment carries an idempotency key the client creates once and reuses on retry. Claim the key in the database before calling the bank, keep the bank call outside any transaction, store the result after, treat a timeout as unknown, and read keys from the primary.

Why it matters

A charge is the one write you cannot cheaply take back. The network between your server and the bank can drop the request, drop the reply, or be slow, and from your side all three are the same timeout.

Airbnb built a library called Orpheus for exactly this, used across its payment services, and reports five nines of consistency while its payment volume doubled. Each of this week's traps is one of its rules being skipped.

Words you need

Idempotent
Safe to repeat. A retried request ends in the same state as the first attempt, instead of an error or a second effect.
Idempotency key
A unique id the client creates once per payment and sends with every retry of it. The server uses it to recognise a retry and return the first result instead of charging again.
Timeout
Giving up on waiting for a reply. It tells you the reply did not arrive in time, not whether the work was done.
Transaction
A group of changes the database treats as one: all of them are saved, or none of them.
Unique index
A database rule that rejects a second row with the same value. A safety net: it turns a silent duplicate into an error.
Read replica
A copy of the database that follows the primary a moment behind. Fine for most reads, wrong for anything that must see the latest write.

Picture it

The customer pays ₹2,000. The bank charges the card, but the reply is lost and the app times out.

Option A: just retry

  1. Pay → bank charges ₹2,000
  2. The reply is lost: timeout
  3. The app retries
  4. Bank charges ₹2,000 again

₹4,000 taken for one order. The classic "paise kat gaye" support ticket.

Option B: idempotency key

  1. Pay with key k1 → key recorded
  2. Bank charges ₹2,000, reply lost
  3. The app retries with the same k1
  4. Server sees k1, returns the stored result

Charged once. The retry is safe and cheap.

The options

Option A: call the bank, retry on timeout

charge(); on timeout → charge() again

Good: Nothing to build. Works whenever the first call really failed.

Cost: When the first call went through and only the reply was lost, the customer pays twice. Not retrying doesn't help: they tap Pay again.

Option B: idempotency key with three phases

claim key → call bank → store result

Good: A retry with the same key gets the first result back. The customer is charged once however many times the request arrives.

Cost: Every rule has to hold: claim the key atomically, no bank call inside a transaction, timeouts treated as unknown, keys read from the primary. Skipping any one is a trap this week.

The choice and its cost

The client generates an idempotency key once per payment, saves it before calling, and sends the same key with every retry. The server runs Airbnb's three phases: record the request and key (no network calls), call the bank (no database work), record the result (no network calls).

Airbnb's rules make it hold: keys live on the primary database, never a replica; each request holds an expiring lease on its key, longer than the bank call's timeout; 5XX and infrastructure errors are retryable, 4XX validation errors are not; and the client never changes the payload between retries.

What it costs

One more table, a write before and after every charge, and a pruning job for old keys. Airbnb notes the three-phase split also makes the code harder to read.

Interviewers also listen for: what the key is

A random UUID for one request, reused only by its retries. A deterministic key like payment-1234-refund for things that must happen once per entity, so two different clicks on Refund still refund once.

Check yourself

  1. 1Why is a timeout dangerous on a payment?Show answer

    It doesn't tell you whether the bank charged the card. The request may have failed, or only the reply was lost.

  2. 2Why must the key be claimed before the bank call, not after?Show answer

    If two retries arrive together and both check first, both see nothing and both charge. Claiming the key in one statement lets only one through.

  3. 3Why does Airbnb read idempotency keys from the primary only?Show answer

    A replica can lag behind. A retry that reads it may not see the stored result and charge again.

The code

GitHub

pay.naive.ts

import type { Pool } from "pg";

// Option A: what most people draw. Client → server → bank.
export async function pay(db: Pool, bank: Bank, orderId: string, amount: number) {
  const charge = await bank.charge(orderId, amount); // times out? the app retries
  await db.query(
    "INSERT INTO payments (order_id, amount, charge_id) VALUES ($1, $2, $3)",
    [orderId, amount, charge.id],
  );
  return charge;
}

Example code. The tested versions land in the repo with each trap.

Every trap in this world is runnable

The design notes are in the repo, and each trap that follows ships with the bug, the fix and a test that proves it.

Remember

A timeout means unknown, not failed. Give every payment a key, claim it before the bank call, and answer retries from what you stored.