Redis: eviction policies, hash slots, hot keys, and persistence

What it is

Redis is a single-threaded in-memory data structure server, and four operational properties follow from that description:

Single-threaded command execution means one slow command blocks everything. KEYS * on a 10-million-key database is not slow for the caller, it is slow for every other client too.

In-memory means a bounded capacity and an eviction policy that decides what happens at the bound. Getting that policy wrong is the difference between a cache and an outage.

Cluster mode partitions by hash slot, not by key range, and the slot assignment determines which operations are possible: multi-key commands require all keys in the same slot.

Persistence is optional and its guarantees are weaker than a database's, which is the source of the most consequential misunderstanding about Redis.

What it is confused with: a database. Redis persists, and its persistence is designed to speed recovery rather than to guarantee durability. The default AOF configuration can lose up to one second of writes on a crash, and the default RDB configuration can lose minutes. For a cache that is correct and unremarkable; for a system of record it is not, and "Redis is durable, it has AOF" is the sentence that precedes an incident.

The problem it solves

Memory is bounded and traffic is not. A cache with no eviction policy fills, and what happens then is a configuration decision most people never make:

maxmemory-policy noeviction     (the DEFAULT)
-> when memory is full, WRITES FAIL with OOM
-> reads still work
-> the cache stops accepting new entries and the application starts erroring

noeviction is the default and it is wrong for a cache. It is right for a queue or a session store where silently dropping data is worse than failing, and it is the setting that turns a full cache into an application outage.

The second problem is hot keys, which single-threadedness makes acute:

A single key receiving 200,000 requests/second:
  - all of them hit ONE shard, because the key hashes to one slot
  - that shard is ONE thread
  - adding shards does nothing: the key does not move
  - the shard saturates at ~100k-150k ops/s and the rest queue

A hot key is a limit you cannot scale past by adding nodes, which is the property that makes it worth its own mitigation strategy.

Mechanics

Eviction policies

noeviction        writes fail with OOM. The DEFAULT.
allkeys-lru       evict least-recently-used, from ALL keys
allkeys-lfu       evict least-FREQUENTLY-used, from all keys
allkeys-random    evict at random from all keys
volatile-lru      evict LRU, but only from keys WITH A TTL
volatile-lfu      evict LFU, only from keys with a TTL
volatile-random   evict at random, only from keys with a TTL
volatile-ttl      evict the keys with the shortest remaining TTL first

The volatile-* family only evicts keys that have a TTL, which means a database containing keys without TTLs can still fill and fail, because those keys are ineligible. That is the trap in volatile-lru: it looks safe and it is noeviction for any key you forgot to expire.

LRU versus LFU is the more interesting choice. Redis's LRU is approximate (it samples maxmemory-samples keys, default 5, and evicts the oldest of the sample) and LFU tracks a probabilistic access counter with decay:

Workload: a catalogue where 5% of items are accessed constantly and
          95% are accessed once each during a crawl.

allkeys-lru:  the crawl evicts the hot 5%, because each crawled item is
              more recently used than a hot item accessed 200 ms ago.
              Hit rate collapses during every crawl.
allkeys-lfu:  the crawled items have a frequency counter of 1 and are
              evicted first. The hot set survives.

LFU is the right default for a cache with a stable hot set and scanning traffic, which describes most caches in front of a catalogue. LRU is right when recency genuinely predicts reuse, such as session data.

maxmemory 24gb
maxmemory-policy allkeys-lfu
lfu-log-factor 10          # how fast the counter saturates
lfu-decay-time 1           # minutes before a counter halves

maxmemory must be set well below the container limit, because Redis's own overhead (replication buffers, client output buffers, fragmentation) is not counted in it:

container limit:      32 GB
maxmemory:            24 GB      (75%)
headroom for:         copy-on-write during BGSAVE (can be up to the dataset size
                      under heavy writes), replication backlog, client buffers,
                      and fragmentation (typically 1.1-1.5x)

maxmemory at the container limit is an OOM kill, because a BGSAVE fork copies pages as they are written and the process can transiently use far more than maxmemory.

Cluster: hash slots, and what they forbid

16,384 hash slots, distributed across shards.
slot = CRC16(key) mod 16384

Shard A: slots 0-5460
Shard B: slots 5461-10922
Shard C: slots 10923-16383

Multi-key commands require every key in the same slot, so MGET user:1 user:2 fails with CROSSSLOT if they hash differently. Hash tags force co-location:

MGET user:1 user:2                    -> CROSSSLOT error
MGET {user}:1 {user}:2                -> same slot: only "user" is hashed

Hash tags are the mechanism and they are also how you create a hot slot, because everything sharing a tag shares a shard:

{tenant:4471}:sessions
{tenant:4471}:cart
{tenant:4471}:prefs
-> all on one shard. Convenient for MGET and MULTI.
-> a large tenant is now a hot shard you cannot split.

Hash tags trade scalability for multi-key operations, and the trade is permanent for the key's lifetime. Use them where the co-location is genuinely required and not as a default naming convention.

Hot keys

Detection first:

# Redis 4.0+: track the hottest keys by frequency (needs an LFU policy)
$ redis-cli --hotkeys
# Or sample the command stream (do NOT leave this running):
$ redis-cli --lru-test 1000
# Per-key stats from the slowlog and from MONITOR (expensive, brief samples only)

Three mitigations, in increasing order of complexity:

1. A client-side local cache for the very hottest keys:

// A small in-process cache with a short TTL, in front of Redis.
Caffeine.newBuilder()
    .maximumSize(1_000)
    .expireAfterWrite(Duration.ofSeconds(2))    // bounded staleness
    .build(key -> redis.get(key));

Two seconds of staleness removes essentially all of the load for a key read 200,000 times a second, because the 200,000 reads become one Redis read per client instance per two seconds. This is the highest-value mitigation and the least used, because it requires accepting bounded staleness.

Redis 6's client-side caching (RESP3 tracking) does this with invalidation: the server notifies clients when a tracked key changes, so the local cache can be correct rather than merely fresh-enough.

2. Key splitting, if staleness is unacceptable:

// Write to N replicas of the key; read from a random one.
int n = 16;
String readKey  = "hot:counter:" + ThreadLocalRandom.current().nextInt(n);
String writeKey = "hot:counter:" + i;   // write to ALL n on update

Sixteen keys hash to sixteen different slots, so the load spreads across shards. The cost is that a write must update all n, which is fine for a read-heavy key and terrible for a write-heavy one.

3. A read replica per hot key, which most managed Redis offerings support by directing reads to replicas. It multiplies read capacity and adds replication lag.

Persistence, and why Redis is not a database

RDB (snapshotting):

save 900 1        # snapshot if >=1 key changed in 900s
save 300 10
save 60 10000

Mechanism:  fork(), the child writes a point-in-time snapshot
Loss on crash:  everything since the last snapshot (potentially MINUTES)
Cost:  fork() copies page tables; under heavy writes copy-on-write can
       transiently double memory
Recovery:  fast, a single sequential file read

AOF (append-only file):

appendonly yes
appendfsync everysec       # the DEFAULT

appendfsync always     fsync every command. Durable, and ~10x slower.
appendfsync everysec   fsync once a second. UP TO ONE SECOND OF LOSS.
appendfsync no         let the OS decide. Up to 30s of loss.

everysec is the default and it loses up to a second of writes on a crash. That is a deliberate, documented trade and it is fine for a cache. It is not a database guarantee, and the distance between "AOF is enabled so we are durable" and "we can lose a second of writes" is where the incidents live.

And replication is asynchronous, which is the larger issue:

1. Client writes to the primary.
2. Primary replies OK IMMEDIATELY.
3. The write propagates to replicas asynchronously.
4. Primary fails. A replica is promoted.
5. Any write acknowledged but not yet replicated is LOST.

WAIT numreplicas timeout      # block until N replicas have the write

WAIT provides a bound and not a guarantee: it tells you how many replicas have the write at that moment, and a subsequent failover can still lose it, because Redis has no consensus protocol for the data path. Redis Sentinel and Cluster handle failover, not durable consensus, which is why Jepsen's analyses have consistently found data loss under partition.

Use Redis for:      caches, sessions (if loss is tolerable), rate limiters,
                    leaderboards, pub/sub, queues where at-least-once and
                    occasional loss are acceptable
Do NOT use Redis as:  a system of record, a durable queue where loss is
                      unacceptable, or anything requiring linearizability
                      across a failover

The single-threaded consequences

Commands that block everything:
  KEYS *                 O(N) over the whole keyspace
  SMEMBERS on a big set  O(N)
  HGETALL on a big hash  O(N)
  DEL of a huge key      O(N) to free the memory
  FLUSHALL               O(N)
  Lua scripts            run to completion, atomically

# The safe alternatives:
SCAN 0 MATCH prefix:* COUNT 100        # cursor-based, non-blocking
HSCAN, SSCAN, ZSCAN                    # same for collections
UNLINK key                             # DEL, but frees memory in a background thread

UNLINK instead of DEL for large keys is a one-word change that avoids a multi-second stall, and it is the single most useful thing to know about the single-threaded model.

slowlog-log-slower-than 10000     # microseconds: log anything over 10 ms
slowlog-max-len 256

SLOWLOG GET is the first thing to check on a latency complaint, because a single slow command explains a latency spike affecting every client.

A worked example: a cache that took down checkout

An e-commerce platform. Redis as a session store and a product cache, 3 nodes in cluster mode, 64 GB each.

The incident:

11:40  a marketing email goes out; traffic rises 4x
11:42  Redis memory reaches maxmemory on shard B
11:42  writes to shard B begin failing: OOM command not allowed
11:43  sessions cannot be written -> users are logged out mid-checkout
11:44  the application retries, adding load
11:51  checkout fully unavailable
12:20  recovered by flushing the product cache manually

Root cause: maxmemory-policy noeviction, the default, on a database used as a cache.

$ redis-cli CONFIG GET maxmemory-policy
1) "maxmemory-policy"
2) "noeviction"

Nobody had set it. The cluster had run for two years without reaching maxmemory, so the policy had never mattered.

Fix 1: an eviction policy, chosen rather than defaulted.

The database held two kinds of data with different requirements:

sessions:       must NOT be evicted (a user is logged out)
product cache:  evict freely

-> separate them. Sessions in their own Redis with noeviction and enough
   memory; product cache with allkeys-lfu.

Mixing evictable and non-evictable data in one Redis is the actual design error, and volatile-lru is the tempting middle ground that does not work: it protects keys without TTLs by refusing to evict them, so a database full of them fails exactly as noeviction does.

product cache:   maxmemory-policy allkeys-lfu, maxmemory 48gb (of 64)
sessions:        maxmemory-policy noeviction,  maxmemory 12gb (of 16),
                 with alerting at 70%

Fix 2: LFU rather than LRU, decided by measurement.

Their traffic included a nightly partner crawl touching ~800,000 SKUs once each.

with allkeys-lru:   hit rate 94% -> 31% during the crawl, recovering over ~2 hours
with allkeys-lfu:   hit rate 94% -> 91% during the crawl

The crawl was evicting the hot set every night, and the morning traffic hit a cold cache. LFU fixed it because a once-accessed key has a frequency counter of 1 and is evicted first.

Fix 3: the hot key, found during the investigation.

$ redis-cli --hotkeys
Hot key found with counter: 8412903  -> 'config:feature_flags'

config:feature_flags read on EVERY request: ~180,000 reads/second
all on one shard, one thread
that shard: 96% CPU, p99 latency 41 ms (vs 0.3 ms on the others)

// Fix: a client-side cache with a 5-second TTL and pub/sub invalidation.
private final LoadingCache<String, FeatureFlags> local = Caffeine.newBuilder()
    .expireAfterWrite(Duration.ofSeconds(5))
    .build(k -> parse(redis.get(k)));

// Invalidate immediately on change, so 5s is a ceiling rather than the norm.
redis.subscribe("flags:changed", msg -> local.invalidateAll());

Redis reads for that key:   180,000/s -> ~12/s
shard B CPU:                96% -> 14%
p99 (that shard):           41 ms -> 0.4 ms

From 180,000 reads per second to 12, because 60 application instances each read it once per five seconds instead of once per request. The pub/sub invalidation is what made a five-second TTL acceptable: a flag change propagates immediately and the TTL is only a backstop.

Fix 4: a slow command found in the slowlog.

$ redis-cli SLOWLOG GET 5
1) 1) (integer) 84
   2) (integer) 1722767041
   3) (integer) 1840219            # 1.84 SECONDS
   4) 1) "KEYS"
      2) "session:*"

A KEYS session:* in an admin endpoint, blocking every client for 1.84 seconds each time someone loaded the internal dashboard.

// Replaced with SCAN, cursor-based and non-blocking.
ScanParams params = new ScanParams().match("session:*").count(100);
String cursor = "0";
do {
    ScanResult<String> r = jedis.scan(cursor, params);
    process(r.getResult());
    cursor = r.getCursor();
} while (!cursor.equals("0"));

Fix 5: persistence, reconsidered rather than tuned.

before:  appendonly yes, appendfsync everysec, on BOTH the cache and sessions

product cache:  persistence DISABLED entirely.
                Rationale: it is a cache. On restart it warms from the
                database. AOF was costing 18% of write throughput and
                a fork stall on rewrite, to protect data that is
                reconstructible by definition.

sessions:       appendonly yes, appendfsync everysec, PLUS the
                acknowledgement that up to 1 second of sessions can be
                lost on a crash, which was accepted after the alternative
                (appendfsync always) was measured at 9x the latency.

Turning persistence off on the cache was worth 18 percent of write throughput and removed the fork stalls, and it required only stating out loud that a cache is reconstructible.

Final:

                              before      after
maxmemory-policy              noeviction  allkeys-lfu (cache),
                                          noeviction (sessions, separated)
hit rate during the crawl     31%         91%
hot key reads/s               180,000     12
worst shard CPU               96%         21%
worst blocking command        1.84 s      none (SCAN)
cache write throughput        baseline    +18% (no AOF)
OOM incidents                 1 (outage)  0

The outage was caused by a default nobody had chosen, and the investigation that followed found a hot key, a blocking command and unnecessary persistence, none of which had been visible while the system was comfortably under its memory limit.

Production evidence

noeviction is the documented default, and Redis's own documentation notes it is appropriate when Redis is used as a store rather than a cache. The mismatch between the default and the most common use is the source of the failure above.

LFU was added in Redis 4.0 specifically to address the scanning-traffic problem: the release notes describe LRU's vulnerability to a scan evicting the working set, which is the crawl scenario.

UNLINK was added in Redis 4.0 for the same class of problem, freeing memory in a background thread so deleting a large key does not block. Its existence is an acknowledgement that the single-threaded model makes O(N) commands a system-wide concern.

Jepsen's analyses of Redis and Redis Cluster found data loss under partition, and Redis's own documentation states that Cluster does not guarantee strong consistency and that acknowledged writes can be lost during failover. That is documented behaviour rather than a bug, and quoting it is the fastest way to end a "can we use Redis as our database" discussion.

RESP3 client-side caching with server-assisted invalidation (Redis 6) exists because client-side caching in front of Redis is the standard hot-key mitigation and correctness required invalidation rather than short TTLs alone.

Hash tags and CROSSSLOT are documented Cluster behaviour, and the guidance to use hash tags sparingly (because they concentrate keys on one shard) appears in the cluster specification.

The debate

Which eviction policy? allkeys-lfu for a cache, in most cases, because scanning traffic is common and LRU is vulnerable to it. allkeys-lru when recency genuinely predicts reuse (sessions, recent activity). noeviction only when Redis is a store rather than a cache, and then with alerting well below maxmemory. The volatile-* family is the trap: it looks safer and it fails identically to noeviction for any key without a TTL.

Should cache and non-cache data share a Redis? No, and the worked example is why: one eviction policy cannot serve both, and any compromise fails one of them. Separate instances with separate policies is the correct design, and it also separates their failure modes so a cache filling does not log users out.

Is Redis persistence worth enabling? For a cache, usually not: it costs write throughput and fork stalls to protect data that is reconstructible by definition, and restart-warming from the source is often faster than loading a large AOF. For sessions and anything where loss is visible to users, yes, with the explicit acknowledgement that everysec loses up to a second. appendfsync always is available and roughly 10x slower, and if you need it you should ask whether Redis is the right store.

Can Redis be a database? No, and this is a position worth stating plainly. Asynchronous replication means acknowledged writes are lost on failover, WAIT bounds rather than guarantees, and Redis Cluster explicitly does not provide strong consistency. It is an excellent cache, session store, rate limiter and leaderboard, and a poor system of record, and the products that changed this (Redis Enterprise's CRDT-based active-active, RedisRaft) are separate offerings rather than the open-source default.

How do you handle a hot key? Client-side caching first, because it removes the load rather than spreading it, and a two-to-five-second TTL with pub/sub invalidation covers most correctness concerns. Key splitting when staleness is genuinely unacceptable, at the cost of fanned-out writes. Adding shards does nothing, which is the property that makes hot keys different from ordinary load.

Is single-threadedness a problem? It is a simplification that removes an entire class of concurrency bugs and makes one slow command a system-wide event. Redis 6 added threaded I/O (network read and write on multiple threads, command execution still single-threaded), which helps throughput and does not change the O(N)-command problem. The practical response is SLOWLOG, SCAN instead of KEYS, and UNLINK instead of DEL.

Follow-up Q&A

"What is the default eviction policy and why does it matter?"

noeviction, which means writes fail with an OOM error when memory is full while reads keep working. It is right when Redis is a store and wrong when it is a cache, and it is the default, so a cache that has never reached maxmemory is carrying a latent outage. In one case a 4x traffic spike filled the instance, session writes began failing, and users were logged out mid-checkout.

"LRU or LFU?"

LFU for a cache with a stable hot set and any scanning traffic, because LRU evicts the hot set during a scan: a crawled item touched once is more recently used than a hot item touched 200 ms ago. In one measurement a nightly partner crawl took the hit rate from 94 percent to 31 percent under LRU and to 91 percent under LFU. LRU is right where recency genuinely predicts reuse, such as session data.

"Why is volatile-lru a trap?"

It only evicts keys that have a TTL, so any key without one is ineligible. A database containing keys without TTLs can therefore fill completely and start failing writes, exactly as noeviction does, while appearing to have an eviction policy configured. It is the setting that looks safe and is not.

"How do you handle a hot key?"

Not by adding shards, because the key hashes to one slot and does not move. Client-side caching with a short TTL is the highest-value fix: in one case a feature-flag key read on every request went from 180,000 Redis reads per second to 12, because 60 application instances each read it once per five seconds instead of once per request. Pub/sub invalidation makes the TTL a backstop rather than the staleness bound. Key splitting across N suffixes is the alternative when staleness is unacceptable, at the cost of writing to all N.

"Is Redis durable?"

Not in the way the word usually means. AOF with the default appendfsync everysec loses up to a second of writes on a crash, and more importantly replication is asynchronous, so a write acknowledged by the primary and not yet replicated is lost on failover. WAIT tells you how many replicas have it at that moment and does not prevent a later failover from losing it. Redis Cluster's documentation states it does not guarantee strong consistency, and Jepsen has confirmed data loss under partition. It is a cache, not a system of record.

"What is the first thing you check on a Redis latency complaint?"

SLOWLOG GET. Redis executes commands on one thread, so a single slow command blocks every client, and a latency spike affecting everything simultaneously is almost always one O(N) command. In one case it was KEYS session:* at 1.84 seconds, run whenever someone opened an internal dashboard. The fixes are SCAN instead of KEYS and UNLINK instead of DEL for large keys.

Common misconceptions

"Redis evicts by default." The default is noeviction: writes fail and reads succeed. A cache that has never hit maxmemory has never exercised the policy.

"volatile-lru is the safe choice." It only evicts keys with a TTL, so a database full of keys without TTLs fails identically to noeviction.

"Adding shards fixes a hot key." The key hashes to one slot and stays there. Only client-side caching, key splitting or read replicas change the load on that shard.

"AOF makes Redis durable." The default fsync policy loses up to a second, and asynchronous replication loses acknowledged writes on failover regardless of the fsync setting.

"Redis is fast so command choice does not matter." It is single-threaded, so an O(N) command blocks every client. KEYS * on a large keyspace is a multi-second stall for the whole system.

Interview delivery note

Say this verbatim: "The default maxmemory-policy is noeviction, which means writes fail when memory is full. That is right for a store and wrong for a cache, and since a cache that has never hit its limit never exercises the policy, it is a latent outage. And I would not mix evictable and non-evictable data in one instance, because no single policy serves both." The default, why it is wrong, and the design consequence.

The senior-versus-staff separator is LFU versus LRU with the scanning argument. A senior engineer sets an eviction policy. A staff engineer knows that LRU is vulnerable to scanning traffic, that a crawl touching a million objects once each will evict the hot set because each crawled item is more recently used than a hot one, and can give the measurement: 94 percent hit rate to 31 under LRU and to 91 under LFU on the same nightly crawl.

The second signal is treating a hot key as unscalable rather than as load. Saying "adding shards does nothing because the key hashes to one slot, so the options are client-side caching, key splitting or replicas" shows you understand why it is a different problem, and pub/sub invalidation making a five-second TTL acceptable is the detail that makes the fix deployable.

Further reading

  • Redis documentation on eviction policies, including the note that noeviction suits store-like usage.
  • Redis 4.0 release notes on LFU and UNLINK, for the problems each was added to solve.
  • Jepsen's Redis and Redis Raft analyses, read alongside Redis's own statement that Cluster does not provide strong consistency.
  • Redis 6 client-side caching (RESP3 tracking) documentation, for server-assisted invalidation as the correct hot-key mitigation.