AI, GenAI and LLM systems

The content of a 2026 loop shifted hard. Classical ML theory is now perhaps a quarter of the technical rounds; the rest is retrieval architecture, evaluation, agents, serving economics and the quiet ways these systems fail. Definitions are free. Judgment is what is being bought.

Four things a modern loop probes that a 2022 loop never did: system design on top of a model you do not control, judgment about retrieval and evaluation, whether you can build and debug the integration by hand, and production sense about cost, latency and silent failure.

What this chapter covers

• [done] Prefill vs decode
• [done] Diagnosing confidently wrong RAG
• [todo] The transformer, walked through with tensor shapes
• [todo] RoPE and ALiBi explained, not named
• [todo] Tokenisation, BPE, and why non-Latin scripts cost two to four times the tokens
• [done] KV cache, GQA, paging and continuous batching
• [todo] FlashAttention, speculative decoding, quantisation, parallelism
• [todo] The fine-tuning ladder with a worked LoRA example
• [todo] Sampling strategies with demonstrated outputs
• [todo] Structured output: JSON mode, constrained decoding, grammars
• [done] Chunking, and why fixed-size is usually wrong
• [done] Hybrid retrieval and reciprocal rank fusion
• [todo] Cross-encoder and LLM reranking, and the latency budget
• [done] Evaluating RAG
• [done] Evaluating an agent
• [todo] Regression gates in CI
• [done] LLM cost engineering
• [done] Prompt injection and the lethal trifecta
• [todo] Classical ML you still need: calibration, leakage, drift

Source: §15, §24, §41, §44.