Redis: caching & feature lookups

We met Redis as Celery's broker, but that's a fraction of what it does. Redis is an in-memory key-value store, absurdly fast (sub-millisecond), and the Swiss Army knife of production systems. For ML it solves three recurring problems: caching predictions, serving features fast, and rate limiting. This chapter shows each.

Setup: pip install redis and run a Redis server. Follow-along, needs a running Redis.

Why in-memory matters

Redis keeps data in RAM, so reads and writes take microseconds, orders of magnitude faster than a database or recomputing a model prediction. The trade-off: RAM is smaller and (by default) volatile. So you use Redis for hot, ephemeral data, things you can afford to lose and want now.

Use 1: cache predictions

If the same text gets scored repeatedly (popular queries, repeated documents), don't re-run the model, cache the result. The pattern is cache-aside: check the cache; on a miss, compute and store it.

import json
import redis

r = redis.Redis(host="localhost", port=6379, db=0)

def cached_score(text, model, ttl=3600):
    key = f"pred:{text}"
    hit = r.get(key)
    if hit is not None:                       # cache HIT — skip the model entirely
        return json.loads(hit)
    score = float(model.predict_proba([text])[0])   # cache MISS — compute it
    r.set(key, json.dumps(score), ex=ttl)     # store with a 1-hour expiry
    return score

The ex=ttl gives the key a time-to-live: it auto-expires after an hour, so the cache never grows unbounded and stale predictions self-clean. Expected behavior:

first call ("great product")   -> MISS, runs model, stores -> 0.9768   (~5 ms)
second call ("great product")  -> HIT,  returns from Redis  -> 0.9768   (~0.2 ms)

The second call is ~25× faster because it never touched the model. For an expensive model (an LLM call costing money and seconds), caching is the single highest-impact optimization you can make.

Don't be confused: caching vs. the result backend. In Chapter 6 Redis stored task results so a producer could fetch them. Here it caches model outputs to avoid recomputation. Same Redis, different purpose: it's a general key-value store, and these are just two of its uses.

Use 2: a feature store / fast feature lookups

Online models need features now: when a request arrives for user:123, you must fetch that user's features in single-digit milliseconds. A database might be too slow; Redis is built for it. Redis hashes store an object's fields under one key:

# precompute features offline, write them to Redis
r.hset("user:123", mapping={"avg_order": "84.50", "n_sessions": "12", "tier": "gold"})

# at request time, fetch instantly
features = r.hgetall("user:123")
# -> {b'avg_order': b'84.50', b'n_sessions': b'12', b'tier': b'gold'}

This is the core of an online feature store (the foundations book's MLOps chapter): batch jobs compute features and push them to Redis; the serving model reads them with hgetall in under a millisecond. Tools like Feast use exactly this pattern (often with Redis as the online store), and they add the crucial guarantee that training and serving compute features identically, killing training/serving skew.

Use 3: rate limiting

Protect your API from abuse by counting requests per user per time window. Redis's atomic INCR plus a TTL makes this a few lines:

def allow_request(user_id, limit=100, window=60):
    key = f"rate:{user_id}"
    count = r.incr(key)             # atomic increment (creates the key at 1)
    if count == 1:
        r.expire(key, window)       # first hit sets the 60-second window
    return count <= limit           # allow until the limit is hit

requests 1..100  -> allow_request -> True
request 101      -> allow_request -> False   (blocked until the window resets)

Because INCR is atomic, this is correct even with thousands of concurrent requests, no race conditions. The key auto-expires, resetting the window.

The data structures you'll actually use

Production notes

• TTLs everywhere: cache entries should expire, or Redis fills up. Set an eviction policy (allkeys-lru) so it drops the least-used keys under pressure.
• Persistence: Redis can persist to disk (RDB/AOF), but treat it as a cache you can rebuild, not your source of truth.
• It's a network hop: fast, but not free; batch reads (MGET, pipelines) when you can.
• Cache invalidation: "one of the two hard problems." When the model changes, old cached predictions are wrong, version your cache keys (pred:v2:...) so a new model can't serve stale answers.

The takeaway

Redis is the in-memory workhorse: cache predictions (cache-aside with a TTL) to skip expensive recomputation, serve features in sub-millisecond lookups (the online feature-store pattern), and rate-limit with atomic counters. Always set TTLs, version your cache keys, and treat it as rebuildable. We can now serve fast and scale work, but which data trained the model we're serving? Time to version it. 👉