Glossary: words an interviewer assumes you know

A fast lookup for every term this book defined, plus a few you'll meet in the wild. One line each, with a pointer to the chapter that explains it properly. If you can give the one-liner and sketch the code, you're fluent.

The learning loop

  • Model: a function with adjustable numbers (parameters) inside. Ch 1
  • Parameter / weight: a number the model learns via training. Ch 1
  • Hyperparameter: a knob you set and hold fixed (learning rate, #layers). Ch 1
  • Bias (the term +b): the constant added in w·x + b; shifts the output. Ch 6
  • Training / fitting: adjusting parameters to minimize the loss. Ch 1
  • Inference / prediction: running a trained model on new input (no learning). Ch 1
  • Loss / cost / objective: the single number measuring current wrongness; training minimizes it. Ch 7
  • Metric: what you report (accuracy, F1); needn't be differentiable. Ch 10
  • Supervised / unsupervised / reinforcement: learn from labels / from structure / from rewards. Ch 1
  • Epoch / iteration / batch: full data pass / one update / examples per update. Ch 16

Data & features

  • Tensor: an n-dimensional array; rank = number of dimensions. Ch 2
  • Shape: the size along each dimension; (n_samples, n_features) by convention. Ch 2
  • Broadcasting: auto-stretching size-1 dimensions so arrays of different shapes combine without loops. Ch 2
  • Feature: one numeric property of a sample; the columns of X. Ch 3
  • Feature extraction / engineering / selection: create features / hand-craft new ones / pick a subset. Ch 3
  • One-hot encoding: a 0/1 column per category, no false ordering. Ch 3
  • TF-IDF: text features weighting rare-but-distinctive words up, common words down. Ch 3
  • Standardization (z-score): per-feature mean 0, std 1. Ch 3
  • Normalization: to unit length (L2) or to [0,1] (min-max) ask which. Ch 3
  • Embedding: a learned dense vector where geometric closeness = semantic similarity. Ch 14
  • Representation learning: the model invents features instead of you crafting them. Ch 14
  • Data leakage: test information sneaking into training; inflates offline scores. Ch 16

Similarity & math

  • Dot product: Σ aᵢbᵢ; similarity that grows with alignment and magnitude. Ch 4
  • Cosine similarity: dot of normalized vectors; direction only, [-1, 1]. Ch 4
  • Euclidean / Manhattan distance: straight-line (L2) / city-block (L1) distance. Ch 4
  • RBF / Gaussian kernel: exp(-γ‖a-b‖²); turns distance into tunable similarity. Ch 4
  • Kernel (ML): a similarity function that's secretly a dot product in a richer space; the kernel trick uses that space without building it. Ch 5
  • Kernel (other 4): conv filter / GPU function / OS core / stats window. Ch 5
  • Norm: a vector's length; ‖v‖₂ = √Σvᵢ². Ch 4
  • Logit: a raw pre-sigmoid/softmax score in (-∞, ∞). Ch 6

Models & training

  • Linear regression: predict a number with w·x + b. Ch 6
  • Logistic regression: predict a probability via sigmoid(w·x + b); a classifier. Ch 6
  • Sigmoid / softmax: squash to one probability / to competing probs summing to 1. Ch 6
  • Cross-entropy (= log loss = NLL): the classification loss; punishes confident mistakes. Ch 7
  • MSE / MAE / Huber: regression losses: squared / absolute / hybrid. Ch 7
  • Gradient: the vector of partial derivatives; points uphill in loss. Ch 8
  • Gradient descent: θ -= lr · gradient; the optimization engine. Ch 8
  • Learning rate: step size; too big diverges, too small crawls. Ch 8
  • SGD / mini-batch / momentum / Adam: the optimizer family; Adam is the default. Ch 8
  • Backpropagation: the chain rule applied efficiently to get all gradients. Ch 8
  • Autograd: software that does backprop automatically. Ch 12

Generalization

  • Overfitting / underfitting: memorizing noise / too simple to capture the pattern. Ch 9
  • Bias-variance tradeoff: error from wrong assumptions vs. from sensitivity to the sample. Ch 9
  • Train / validation / test: learn params / tune hyperparams / report once. Ch 9
  • Cross-validation: rotate held-out folds to squeeze small datasets. Ch 9
  • Regularization (L1 / L2): penalize weight size; L1 → sparse, L2 → smooth shrinkage (weight decay). Ch 9
  • Dropout / early stopping / data augmentation: neural-net anti-overfitting tools. Ch 9

Metrics

  • Accuracy: fraction correct; misleading on imbalanced data. Ch 10
  • Confusion matrix: TP / FP / FN / TN; the source of all classification metrics. Ch 10
  • Precision / recall: trust of your alarms / coverage of true positives. Ch 10
  • F1: harmonic mean of precision and recall; high only if both are. Ch 10
  • ROC-AUC: threshold-free ranking quality; 0.5 = random, 1.0 = perfect. Ch 10
  • RMSE / MAE / R²: regression error (squared / absolute) / variance explained. Ch 10
  • Recall@k / NDCG: ranking metrics: fraction of the relevant items that made the top k / graded relevance of the top k, discounted by position. Ch 28

Neural networks & architectures

  • Neuron / layer / MLP: activation(w·x+b) / many neurons / stacked layers. Ch 11
  • Activation function: the non-linearity that makes depth meaningful (ReLU, GELU, tanh). Ch 11
  • Vanishing/exploding gradient: gradients shrinking to 0 / blowing up through deep nets. Ch 11
  • Weight initialization: small scaled random starts (Xavier/He) so training begins well. Ch 11
  • Inductive bias: a built-in assumption about the data an architecture encodes. Ch 13
  • CNN / convolution / filter: local, position-independent pattern detector for grids. Ch 13
  • RNN / LSTM / GRU: sequence models carrying a hidden state; gates for long memory. Ch 13
  • Transformer / self-attention / Q,K,V: softmax(QKᵀ/√d)·V; everything attends to everything. Ch 13
  • Diffusion / GAN / autoencoder / GNN / MoE: generative & specialized architectures. Ch 13
  • Two-tower model: two embedding networks (one per side, e.g. user and item) trained so matching pairs land close; the standard retrieval/candidate-generation architecture. Ch 28

LLMs & the modern stack

  • Token / tokenization: the model's subword unit; text → integer IDs. Ch 15
  • Context window: max tokens the model attends to at once; its working memory. Ch 15
  • Temperature / top-k / top-p: sampling knobs trading focus for creativity. Ch 15
  • Pretraining / fine-tuning / RLHF: broad learning / specialization / alignment. Ch 15
  • RAG: retrieve relevant docs into the prompt; adds knowledge. Ch 15
  • Hallucination: confident falsehood; the model optimizes plausibility, not truth. Ch 15
  • Quantization: fewer bits per weight to shrink/speed models. Ch 15
  • Agent / tool use: an LLM that calls functions in a loop to act. Ch 15

Tools & engineering

  • NumPy: Python's array math library; the foundation of everything here. Ch 0
  • PyTorch: tensors + autograd + nn.Module + optimizers; the deep-learning framework. Ch 12
  • nn.Module / DataLoader: your model class / the batching+shuffling iterator. Ch 12, Ch 16
  • GPU / CUDA / .to(device): parallel hardware / its programming model / moving tensors to it. Ch 16
  • model.train() / .eval() / no_grad(): dropout&bn on / off / skip the grad graph. Ch 12
  • Mixed precision / gradient accumulation / checkpointing: speed, memory, and crash-safety tricks. Ch 16
  • Seed / reproducibility: fix randomness so runs and comparisons are trustworthy. Ch 16
  • MLflow / Weights & Biases: experiment tracking for honest comparisons. Ch 16

Numerical literacy

  • Floating point: approximate arithmetic; compare with isclose, never ==. Ch 17
  • Log-sum-exp / stable softmax: subtract the max before exp to avoid overflow. Ch 17
  • NaN / inf: poison values; usual cause is a too-high learning rate or log(0). Ch 17

Classical ML algorithms

  • k-NN: classify by majority vote of the k nearest training points; no training. Ch 20
  • Naive Bayes: Bayes' rule assuming feature independence; fast text baseline. Ch 20
  • Decision tree: recursive if/else splits chosen to reduce impurity (Gini/ entropy). Ch 20
  • Random forest / bagging: many independent trees averaged; reduces variance. Ch 20
  • Gradient boosting (XGBoost/LightGBM): sequential error-correcting trees; reduces bias; king of tabular ML. Ch 20
  • SVM / support vectors / margin: max-margin classifier; kernels for non-linear boundaries. Ch 20
  • k-means: partition data into k round clusters (assign/move). Ch 21
  • DBSCAN: density-based clustering; finds arbitrary shapes & outliers, no k. Ch 21
  • PCA: project onto top variance directions (via SVD) to reduce dimensions. Ch 21
  • t-SNE / UMAP: non-linear methods for visualizing high-dim data (not as features). Ch 21

Math & statistics

  • Expectation / variance: long-run mean / spread of a random variable. Ch 22
  • Central Limit Theorem: averages become Normal; uncertainty shrinks like $1/\sqrt n$. Ch 22
  • Bayes' theorem: update belief with evidence: posterior ∝ likelihood × prior. Ch 22
  • MLE / MAP: parameters that maximize data likelihood / posterior; the origin of losses & L2. Ch 22
  • p-value: P(data this extreme | null true); not P(null), not effect size. Ch 22
  • eigenvector / eigenvalue: a direction a matrix only scales, and by how much. Ch 23
  • SVD: factor any matrix; gives the best low-rank approximation (PCA, LoRA). Ch 23
  • rank: number of independent directions in a matrix; low rank = compressible. Ch 23
  • A/B test / power / guardrail: randomized online experiment; chance of catching a real effect; metrics that must not regress. Ch 24

Tools, cloud & the modern landscape

  • Pandas / SQL / scikit-learn: wrangling (groupby) / querying data / the fit/predict API. Ch 25
  • Hyperparameter search (grid/random/Optuna): tune knobs by cross-validated search. Ch 25
  • MLOps / drift / feature store: productionizing models / input or concept shift / consistent train-serve features. Ch 26
  • Online vs. batch serving: low-latency one-at-a-time vs. high-throughput scheduled. Ch 26
  • LoRA / PEFT / quantization / distillation: cheap fine-tuning / fewer-bit weights / small student model. Ch 27
  • Vector database: store + nearest-neighbor search over embeddings; the memory of RAG. Ch 27
  • Agent / tool use / RLHF / DPO / MoE / diffusion: LLM that acts / alignment / scaled experts / denoising generation. Ch 27
  • Two-stage retrieval (candidate generation → ranking): the standard scalable recommender/feed pattern. Ch 28

How to use this glossary

In an interview or a design review, you don't need the derivation of every term, you need the one-liner plus the intuition, and the confidence that you could code it. That's exactly what this book built. If any line above still feels hollow, follow its link, re-read the worked example, and run the code. Then you'll own the word, not just recognize it.

That's the whole foundation. Go build something. 👉