Let’s talk
World 1 · URL Shortener
All steps
  1. 1What we are building
  2. 2Hash or counter
  3. 3The ID collision
  4. 4The stale redirect
  5. 5Analytics on the hot path · soon
  6. 6The vanity alias race
  7. 7Cleanup at scale · soon
  8. 8Build it · soon

The stale redirect

The owner changed where the link points. Why do visitors still land on the old page?

Before this: The ID collision·Next: The vanity alias race

In 30 seconds

The problem
After the owner edits a link, visitors keep going to the old destination for as long as the cached copy lives.
Why it happens
A visitor misses the cache and reads the old target from Postgres. The edit and the cache delete happen. Then the visitor writes the old target into Redis.
The fix
Fill the cache with a lease, so a fill that started before an edit is refused. Keep a TTL as the backstop.
In an interview
Update the database, then delete the cache, and expect a slow reader to put the old value back. Fill the cache with a lease, keep a TTL as the safety net, and redirect editable links with 302.

Why it matters

Redirects are read-heavy, so they go through a cache: about 4,000 redirects a second against a handful of edits. Without the cache, Postgres would serve every click.

The usual recipe, update the database and then delete the cached copy, is right almost every time. The gap is small: a cache miss and an edit on the same link at the same moment. But when it happens nothing repairs it. The wrong copy stays until its TTL runs out, a whole day with our settings.

Words you need

Cache
A fast copy of data kept close by, here in Redis, so most reads skip the database.
Cache-aside
Read from the cache; on a miss, read the database and put the result in the cache. The most common way to cache.
Invalidation
Removing or replacing a cached copy after the real data changed, so nobody reads the old one.
TTL (time to live)
How long a cached copy lives before it expires on its own. It caps how long a wrong copy can survive.
Race condition
A bug that only appears when two requests overlap in an unlucky order. Each request is correct on its own.
Lease
A marker a request leaves in the cache on a miss. Its later fill only counts if the marker is still there.

Picture it

sho.rt/g8 points to old-site.com, and its cached copy has just expired. A visitor opens it while the owner changes it to new-site.com.

The usual order

  1. Owner saves new-site.com
  2. Owner deletes the cache
  3. Visitor misses
  4. reads new-site.com
  5. caches it

Correct. This is why the recipe looks safe.

The unlucky order

  1. Visitor misses
  2. reads old-site.com
  3. Owner saves new-site.com
  4. Owner deletes the cache
  5. Visitor caches old-site.com

Postgres says new-site.com and Redis says old-site.com. Every visitor gets the old page for the next 24 hours.

Break it yourself

Play: The stale redirectYou decide which request moves next. Find the order that breaks it, then try the fixes below. The game checks every possible order, not just yours.

The fixes people try

  • Delete first, then update

    Fails
    redis.del(); db.update()

    A visitor can miss right after the delete, read the old value, and write it back. The gap moves; it does not close.

  • Delete twice

    Fails
    del; update; sleep(500); del

    Shrinks the gap. A visitor slower than the sleep still writes the old value after the second delete.

  • Write the new value into the cache

    Fails
    db.update(); redis.set(new)

    Two writers, and the last one wins. The slow visitor's old value can land after the owner's new one.

  • A short TTL

    Half a fix
    redis.set(key, target, "EX", 60)

    Caps how long a wrong copy lives, but every link expires sooner, so Postgres serves more misses. A good backstop, not a fix.

  • Lock the row for misses and edits

    Works
    SELECT target … FOR UPDATE

    Correct, but every cache miss holds a database lock while it talks to Redis, on your busiest path.

  • Fill with a lease

    Works
    SET key fill:<id> NX, fill only if still ours

    An edit's delete removes the marker, so a fill that started before the edit is refused. No locks on the redirect path.

The fix and its cost

Keep the usual recipe, update Postgres and then delete from Redis, and add a lease to the fill. On a miss, the visitor first writes a marker into the empty key. Then it reads Postgres and fills the cache with a tiny Lua script that writes only if its own marker is still there.

If an edit happened in between, its delete removed the marker and the old value is thrown away. The next visitor misses and reads the new target. Facebook described this idea as leases in its 2013 paper on scaling memcache.

Keep a TTL as the safety net for anything else that goes wrong, like a failed delete. Pick it from how long a wrong redirect is acceptable, not from habit.

What it costs

More code on the miss path, and while a marker is in place other visitors read Postgres directly. Give the marker a short expiry, a few seconds, so a crashed request cannot block the cache.

Interviewers also listen for: 301 or 302

Browsers can cache a 301 (permanent) redirect for a long time, so repeat visitors skip your servers and your cache entirely. Links that can be edited should redirect with 302 or 307.

Check yourself

  1. 1Update the database, then delete the cache. How can the cache still end up wrong?Show answer

    A visitor that missed the cache reads the old target before the update, then writes it into the cache after the delete.

  2. 2Why doesn't deleting twice fix it?Show answer

    It shrinks the gap, but a visitor slower than the wait still writes the old value after the second delete.

  3. 3How does a lease stop the stale fill?Show answer

    The edit's delete also removes the visitor's marker, and the fill only writes while its marker is there. A value read before the edit can't be written after it.

The code

GitHub

redirect.bug.ts

import type { Pool } from "pg";
import type Redis from "ioredis";

const TTL = 86_400; // one day

// Redirect: serve from Redis, fill it from Postgres on a miss.
export async function resolve(db: Pool, redis: Redis, code: string) {
  const key = `link:${code}`;
  const hit = await redis.get(key);
  if (hit) return hit;

  const { rows } = await db.query(
    "SELECT target FROM links WHERE code = $1", [code],
  );
  const target: string | undefined = rows[0]?.target;
  // If an edit landed after the SELECT, this writes the old target back.
  if (target) await redis.set(key, target, "EX", TTL);
  return target;
}

// Edit: update Postgres, then drop the cached copy.
export async function updateTarget(
  db: Pool, redis: Redis, code: string, target: string,
) {
  await db.query(
    "UPDATE links SET target = $2 WHERE code = $1", [code, target],
  );
  await redis.del(`link:${code}`);
}

Example code. The tested version, with a proof test against real Postgres, lands in the repo.

Star the repo for the runnable version

Each trap in this world ships with the bug, the fix and a proof test that fires concurrent requests at a real database.

Watch the 60-second version

Remember

Deleting the cache after a write is not enough: a slow reader can put the old value back. Make the fill prove nothing changed since it read.

NextThe vanity alias raceTwo users claim /launch at once, and both are told it is theirs.