How to prevent Cache Stampede (Thundering Herd) in Redis using Distributed Locks?

Question

0

A cache stampede (also known as the thundering herd problem) occurs when multiple concurrent client requests experience a cache miss at the exact same time. This causes all requests to hit the backend database simultaneously, leading to sudden CPU spikes and database exhaustion.

Solution using Distributed Locks

By implementing a distributed lock alongside a standard Cache-Aside pattern, only the first thread that experiences a cache miss is allowed to query the database and rebuild the cached value, while subsequent concurrent requests wait or retry.

import redis
import time

# Initialize Redis client connection
r = redis.Redis(host='localhost', port=6379, db=0)

def get_cached_data(key):
    # Try fetching data directly from cache
    val = r.get(key)
    if val:
        return val.decode('utf-8')
    
    # Try acquiring distributed lock with 10 sec timeout
    lock_acquired = r.set('lock:' + key, 'true', nx=True, ex=10)
    if lock_acquired:
        try:
            # Fetch fresh data from persistent database
            data = "computed_db_result"
            # Cache fresh value with 60 second TTL
            r.setex(key, 60, data)
            return data
        finally:
            # Release distributed lock safely
            r.delete('lock:' + key)
    else:
        # Sleep briefly and retry cache lookup
        time.sleep(0.1)
        return get_cached_data(key)

1 Answer

Write Your Answer

Answer 1

A cache stampede happens when a popular key expires and many requests try to rebuild it at once. A Redis distributed lock lets one request refresh the value while the others wait briefly and check the cache again.

Lock, recheck, then refresh

Use a separate lock key for each cache key. Acquire it with Redis SET key value NX PX: NX only creates the lock if it does not already exist, and PX gives it a lease so it can expire if the worker crashes. The lock value should be a unique lock token, not a shared string.

After acquiring the lock, check the cache again. Another request may have filled it between your first cache lookup and lock acquisition. Requests that fail to acquire the lock should wait for a short, randomized interval and retry the cache lookup rather than immediately running the expensive loader.

import random
import time
import uuid

RELEASE_LOCK = """
if redis.call("get", KEYS[1]) == ARGV[1] then
    return redis.call("del", KEYS[1])
end
return 0
"""

def get_or_load(redis_client, cache_key, loader, ttl_seconds=60):
    lock_key = f"lock:{cache_key}"
    lock_token = uuid.uuid4().hex
    lock_ttl_ms = 10_000
    wait_deadline = time.monotonic() + 5

    while True:
        cached = redis_client.get(cache_key)
        if cached is not None:
            return cached

        # Only one worker should become the cache refresher.
        acquired = redis_client.set(
            lock_key, lock_token, nx=True, px=lock_ttl_ms
        )

        if acquired:
            try:
                # A previous worker may have filled the cache just before this lock was acquired.
                cached = redis_client.get(cache_key)
                if cached is not None:
                    return cached

                value = loader()
                redis_client.set(cache_key, value, ex=ttl_seconds)
                return value
            finally:
                # Delete the lock only if it still belongs to this worker.
                redis_client.eval(RELEASE_LOCK, 1, lock_key, lock_token)

        if time.monotonic() >= wait_deadline:
            raise TimeoutError(f"Timed out waiting for cache key: {cache_key}")

        # Jitter spreads retries out instead of making waiting workers wake together.
        time.sleep(random.uniform(0.02, 0.08))

The Lua release script matters: if the lease expires and another worker acquires the lock, an unconditional delete from the original worker could remove the new worker’s lock.

Things to watch

Choose the lease carefully. The example uses 10 seconds; set it longer than the normal refresh time. If loading can exceed the lease, the lock may expire while the first worker is still running, allowing a second refresh. For long jobs, use a carefully implemented lease renewal or another coordination strategy.
Keep waiters from overwhelming Redis. Use short waits with jitter, and set a sensible timeout. If stale data is acceptable, a stale-while-revalidate approach can serve the old value while one worker refreshes it.
A lock is not a general guarantee of exactly-once work. This pattern coordinates cache refreshes; it does not make external side effects safe. For operations where an expired lock could let an older worker overwrite newer results, use fencing tokens or another mechanism that the destination can validate.

For many cache-aside workloads, the combination of a per-key lock, a second cache check, and jittered retries is enough to prevent the usual herd without making every request wait for the refresh.