ML system design

The senior-level interview isn't "implement quicksort", it's "design the recommendation system for a video app" or "build fraud detection for payments." These open-ended ML system design questions test whether you can turn a vague business goal into a working, scalable ML system. There's a repeatable framework, and this chapter gives it to you, then walks one example end to end.

Why this question exists

It reveals everything a coding puzzle can't: do you ask about requirements before coding? Do you know how to frame an ML problem? Can you reason about data, features, models, serving, scale, and failure? Structure beats brilliance here, a calm, organized walk through the framework impresses far more than jumping to "I'd use a transformer."

The framework (memorize the seven steps)

1. Clarify     →  2. Frame as ML  →  3. Data & labels  →  4. Features
      →  5. Model  →  6. Evaluation (offline + online)  →  7. Serve & scale

1. Clarify requirements (never skip this)

Ask before you design. Interviewers plant ambiguity to see if you probe:

  • Goal & scale: what business metric? How many users/items/requests per second?
  • Latency: real-time (<100ms) or batch overnight?
  • Constraints: cold start? Privacy? Interpretability required? Existing systems?

Don't be confused: jumping to the model is the #1 failure. Spending two minutes on requirements is the single highest-signal thing you can do. Candidates who start with "I'd train a neural net" before knowing the scale or latency budget signal inexperience. Clarify first, always.

2. Frame it as an ML problem

Translate the business goal into a precise ML task:

  • What is being predicted (the label/target)?
  • Is it classification, regression, ranking, retrieval, generation?
  • What's the simplest version that delivers value? (Start there; iterate.)

3. Data & labels

  • Where does training data come from? What's the label, and how do you get it?
  • Implicit vs. explicit feedback: a click is implicit and abundant but noisy; a rating is explicit but rare.
  • Watch for leakage (Chapter 16) and label delay (you learn if a loan defaulted only months later).

4. Features

  • What signals predict the target? (User features, item features, context, interactions, history.)
  • Feature freshness: real-time (last click) vs. batch (30-day average), and the feature store to serve them consistently (Chapter 26).

5. Model

  • Start with a baseline (popularity, logistic regression): you must beat it.
  • Then the appropriate workhorse: gradient boosting for tabular, two-tower/embeddings for retrieval, a neural ranker for ordering.
  • Justify the choice by the requirements (latency, interpretability, data size), not by what's trendy.

6. Evaluation: offline and online

  • Offline: the right metric for the task (Chapter 10): AUC/F1 for classification, NDCG/Recall@k for ranking, RMSE for regression.
  • Online: the A/B test (Chapter 24) on the real business metric. Name your guardrail metrics.
  • State plainly that offline and online can disagree, and online wins.

7. Serve, scale & maintain

  • Online vs. batch serving (Chapter 26); the latency budget decides.
  • Scale: caching, approximate nearest-neighbor retrieval (HNSW/IVF-PQ), precomputation, the candidate-generation → ranking two-stage pattern.
  • Monitoring & retraining: drift detection, the retraining trigger, rollback.

A worked example: "Design a news feed ranker"

Watch the framework turn a one-line prompt into a system.

1. Clarify. "Rank articles for each user's home feed. ~10M users, ~100K fresh articles/day, must serve in <150ms, optimize for long-term engagement (not just clicks, avoid clickbait)."

2. Frame. For each (user, article) pair, predict P(meaningful engagement). Then sort each user's candidates by that score. It's ranking, fed by retrieval.

3. Data & labels. Implicit feedback from logs: impressions + clicks + dwell time. Label = "engaged" if click and dwell > N seconds (discourages clickbait). Beware position bias (top items get more clicks regardless of quality) and the feedback loop (we only see labels for what we showed).

4. Features. User: history, topic affinities, demographics, time of day. Article: topic, freshness, source quality, popularity. Interaction: embedding similarity between user and article (Chapter 14). Context: device, session.

5. Model, the two-stage pattern (the standard answer for feeds/recommenders):

10M articles ─► CANDIDATE GENERATION ─► ~500 candidates ─► RANKING ─► top 20
                 (cheap: embedding ANN          (expensive: gradient-boosted
                  retrieval, HNSW/IVF-PQ)         or neural ranker on rich features)

Stage 1 is fast and approximate (narrow millions to hundreds with vector search). Stage 2 is slow and accurate (score the few hundred precisely). This split is how you hit <150ms at scale, and naming it signals real seniority.

6. Evaluation. Offline: NDCG@20, Recall@k on held-out engagement (split by time, not randomly). Online: A/B test on long-term engagement and retention; guardrails on diversity, latency, and complaint rate.

7. Serve & scale. Precompute article embeddings hourly; cache user embeddings; ANN index for stage 1; the ranker behind a low-latency endpoint with a feature store. Monitor for drift; retrain daily on fresh engagement. Cold start: fall back to popularity/ trending for new users and content-based features for new articles.

(The sister Recommendation Systems book builds exactly this, including the production capstone.)

The common system-design prompts

Prepare a sketch for each: they recur constantly:

  • Recommendation / feed ranking (the two-stage pattern above)
  • Search / retrieval (embeddings + ANN + re-ranking)
  • Fraud / anomaly detection (imbalanced data, precision/recall trade-off, real-time)
  • Ad click-through-rate prediction (massive scale, calibration matters)
  • Content moderation / classification (precision/recall costs, human-in-the-loop)
  • An LLM-powered feature (RAG, latency, cost, hallucination guardrails)

The takeaway

ML system design rewards structure: clarify requirements, frame the ML problem, reason through data/labels → features → model → offline+online evaluation → serving & scale. Always start with a baseline and the simplest thing that works; know the two-stage candidate-generation → ranking pattern cold; remember the A/B test is the final judge; and talk about drift, monitoring, and cold start unprompted. Now the final chapter: the concept questions, coding drills, and prep plan to bring it all home. 👉