Search, retrieval and ranking

If your background is search, recommendations and retrieval, this is the chapter where you control the room. Most interviewers cannot go deep here, which is an advantage only if you have rehearsed the depth. The two places candidates lose it: hand-waving the evaluation story, and treating vector search as a drop-in replacement for lexical search rather than one arm of a hybrid.

The unprompted signals that land well: naming position bias and how you correct for it, naming popularity feedback loops in a recommender, and naming the recall cliff that filtered ANN search walks off.

What this chapter covers

• [done] The inverted index, mechanically
• [done] BM25 from first principles
• [done] Analyzers per language
• [done] Lucene segments: refresh, flush, merge
• [done] HNSW internals
• [done] IVF-PQ vs HNSW vs DiskANN vs ScaNN
• [done] NDCG, MRR and the offline-online gap
• [done] The multi-stage ranking funnel
• [done] Two-tower retrieval and feature parity
• [done] Multilingual index topology
• [done] Popularity bias and feedback loops
• [done] Judgment collection and a training-data pipeline
• [done] Cold start, exploration and bandits
• [done] Embedding freshness and index rebuild without downtime
• [done] Billion-scale sharding and routing
• [done] Learning to rank
• [done] Filtered vector search and the recall cliff
• [done] Interleaving and position bias

Source: §6, §44.2, §47.