Glossary: the feed-evaluation vocabulary

Every term of art from the evaluation arc (27, 28, 29) and the research arc (30, 31, 32, 33, 34, 35, 36), defined once, grouped by what it belongs to.

The unit of display

Card. One item as the user sees it: headline, thumbnail image, source, and metadata. The atom of the feed. Cards, not articles, are what duplicate each other: two cards can share an article's story with different headlines and photos.

Slate. The ranked set of k cards returned for one request; the unit the rubric grades. The word is borrowed from the phrase "a slate of candidates" (a group put forward together) and is the standard term in the recommendation literature for items presented as a set. It earns its own name because a slate has properties no single card has: duplicates, diversity, and ordering are relations between cards, so grading cards one at a time can never see them.

Feed. The ongoing stream of slates a user receives across visits. A slate is one page of the feed.

Catalog. Every article currently recommendable.

Candidate. An article that survived retrieval and may enter the slate. Candidate generation is stage 1 of the two-stage design (Chapter 9).

Bench. The ranked candidates that did not make the slate; the repair step backfills from it.

Impression. A card actually shown to a user. Click: an impression the user opened. Seen set: the articles a user has already been shown, excluded from future slates.

Repair. Applying the rubric's actions to a slate before serving: drop the losing card of every same_story pair, backfill from the bench, re-score.

The user model

Embedding. A vector representing a text (or image) such that similar content lands nearby. Embedder version: the identity tag of the model that produced a vector; vectors from different embedders live in different spaces and must never be compared.

Cosine similarity. The similarity between two vectors, measured by the angle between them; 1.0 means identical direction.

Taste vector / user profile. One vector summarizing what a user reads: the weighted average of the embeddings of their clicked articles.

EMA (exponentially weighted moving average). The weighting scheme for that average: each click's weight decays exponentially with age, so recent clicks dominate. The book also calls this the time-decayed profile (Chapter 5).

Half-life. The age at which a click's weight has fallen to half. The single knob controlling how fast the profile follows the user.

Cold start. A user (or item) with no history; handled by fallbacks (Chapter 10).

Filter bubble / over-specialization. The failure mode where nearest neighbors of one profile all look alike; surfaces as a diversity FAIL.

Retrieval and dedupe machinery

kNN / ANN. (Approximate) nearest-neighbor search: find the k vectors closest to a query vector. HNSW is the graph-based ANN index (companion book).

Story / story_id. One real-world event, however many articles cover it. Assigned at ingest by clustering; the dedupe key.

Collapse. The OpenSearch query feature that returns at most one result per distinct field value; collapsing on story_id is what removes duplicates from results.

Near-duplicate cases A through E. The five ways two cards collide: A same headline and image (exact duplicate); B same story reworded by a second outlet; C different headline over the same photo (syndication); D near-identical wording about a different event (template lookalike); E same story with no shared words (paraphrase). A is decided by equality, B and D need a judge, C needs an image hash, E needs embeddings or a judge.

Flagger / router. The cheap deterministic layer that marks pairs as possible duplicates and routes them onward. Tuned for recall; precision is the judge's job.

Residue. Whatever a cheap layer could not decide; the only thing the next, more expensive layer sees.

Jaccard similarity. Overlap between two token sets: intersection over union.

Perceptual hash (dHash, pHash). A small fingerprint of an image that barely moves under recompression, brightness shifts, or light crops, unlike a cryptographic hash. Hamming distance: the number of differing bits between two such fingerprints.

SimHash. A 64-bit fingerprint of a token set; similar sets land a few bits apart. LSH (locality-sensitive hashing): bucketing fingerprints by bands so near-duplicates collide in some bucket, making catalog-scale dedupe cheaper than comparing all pairs.

The judge

LLM judge. A language model asked one narrow, schema-forced question per call (is this pair the same story?). Verdict: its label plus a one-line reason. Vision judge: the same pattern with an image in the input (headline-image coherence).

Structured output. Forcing the model's response to match a schema (messages.parse with a Literal label set), so verdicts are machine- actionable and there is no prose to parse.

Prompt registry. The module holding every prompt with a semantic version and a content hash; the hash is stamped into every verdict so behavior changes are traceable.

Verdict cache. Stored judge decisions keyed by story pair; the reason the same question is never paid for twice.

Fail-open. The decided-in-advance behavior when a soft dependency is down: keep both cards, serve the feed anyway.

Order (position) bias. A judge's tendency to favor one input position in pairwise comparisons; neutralized by canonicalizing input order or judging both orders.

Majority vote / flip-flop. Re-judging only the pairs whose verdicts vary across runs and taking the majority; the cheap version of self-consistency.

The evaluation

Metric. A number computed against held-out labels, grading the model on average (recall@k, NDCG; Chapter 3). Rubric: a versioned list of named criteria with thresholds, grading one artifact with no labels needed. Metrics pick the model; the rubric judges what it produced.

Criterion. One row of the rubric: a question, a measurable signal, a threshold, and an on-fail action. Scorecard: the criteria evaluated for one slate. Artifact: the machine-readable JSON version of the scorecard, stamped with the rubric version.

Golden set. Hand-labeled examples (here: headline pairs) that every layer is validated against; the framework's ground truth.

Inter-annotator agreement. How much two humans agree labeling the same data; the ceiling any judge can be expected to reach.

Cohen's kappa. Agreement corrected for chance: $\kappa = (p_o - p_e)/(1 - p_e)$. The acceptance metric for judges, because plain accuracy rewards doing nothing on imbalanced data.

Precision / recall. Of the pairs flagged, how many were real (precision); of the real ones, how many were flagged (recall).

Blind-spot register. The rubric's written list of what it cannot see (for this book: case E without embeddings, image coherence without a vision judge), so a green scorecard is never mistaken for omniscience.

Evaluation mode / enforcement mode. The two lives of one rubric: scoring slates nobody sees to gate a deploy (evaluation), and the same criteria compiled into the serving pipeline (enforcement). Only enforcement runs on live traffic, and it makes zero LLM calls.

Drift. Scores changing over time with no code change, because the catalog, users, or upstream models moved; caught by charting sampled scorecards.

Shadow mode / A/B guardrails. Running the new system on real traffic without showing its output (shadow), then showing it to a small arm while watching health metrics (guardrails), before a full rollout.

The research arc

Research. The systematic reduction of uncertainty before committing resources; for a product team it splits into market, domain, technical/feasibility, and evaluative kinds, each with its own sources and standard of proof (Chapter 30).

Spec-driven development (SDD). Writing and reviewing documents (brief, PRD, architecture, stories) before generating code, so an AI agent amplifies a reviewed intent instead of an ambiguity.

PRD (product requirements document). The planning document that states what a system must do (functional requirements) and why; in a healthy pipeline every claim it makes traces back to research evidence.

BMAD. Build More Architect Dreams in the current documentation (the repository also retains the older "Breakthrough Method for Agile AI-Driven Development" wording): an open-source method (v6 as of mid-2026) organizing AI-assisted work into Analysis, Planning, Solutioning, and Implementation phases, staffed by named persona agents and installed as skills (Chapter 31).

Deep Recon. BMAD's research skill: typed research packs, three modes (draft a prompt for an external engine, process a finished report, or run a native parallel web fan-out), enforced citations, and a staleness map (Chapter 32).

Research firewall. Deep Recon's rule that project context shapes what to ask, never what is true: research subagents see only their brief, so they cannot flatter your architecture.

Staleness map. A per-claim record of how fast evidence ages and when to re-check it; the difference between refreshing a report and rerunning the world.

Run folder. Deep Recon's per-run workspace (brief.md, imports/, digests/, research.md, .memlog.md): the report and its ledger are files, so a run that dies mid-flight resumes from disk.

Memlog. The append-only .memlog.md ledger: one line per decision, source batch, claim, or assumption, written through a shared script; claim lines carry ref=[n] status=… class=… pub=… so tooling can count them.

Plan gate. Run mode's one mandatory checkpoint: it shows the decision, the pruned dimensions, the decomposition topology, and the knobs, and nothing crawls until you approve.

Two-source class. A category of claim (a market size, a version number, a regulatory assertion) that a single publisher cannot settle; verification requires an independent second source, never a syndication.

BMAD skill. A host-discoverable package entered through SKILL.md that loads a persona, runs a workflow, or performs a standalone task/tool. The AI host interprets it; BMAD does not run a separate hidden model or daemon.

Artifact contract. A persisted, reviewable file whose structure is the handoff between contexts or roles: for example, research.md from Deep Recon to the PM, or a story file from the PM to the developer.

Externalized state. Decisions, progress, inputs, and outputs written to project files so a later fresh chat can resume without depending on conversation memory.

Project context. Stable implementation rules and conventions in project-context.md, loaded by downstream workflows; distinct from fast-changing research claims and one-off experiment results.

Sparse override. A team or personal TOML customization under _bmad/custom/ containing only changed fields, so new shipped defaults remain visible after an update (Chapter 31).

Knowledge graph. Facts stored as typed nodes and edges (source makes claim, claim supports question, source cites source) so that coverage, contradiction, and priority become graph queries instead of rereads (Chapter 34).

Multi-armed bandit. The explore/exploit problem of allocating a scarce resource across options with unknown payoffs; Thompson sampling solves it by sampling from a posterior per arm and acting on the best sample.

EVPI (expected value of perfect information). What a decision would gain, in expectation, from a perfect answer; an experiment that costs more than the EVPI of its question is not worth running.

MDP / POMDP (Markov decision process). The formal frame of states, actions, transitions, and rewards; the partially observable variant makes information-gathering itself part of an optimal policy, which is the serious version of "research as an MDP."

Profiling (GDPR). Automated processing of personal data to analyse or predict a person's preferences, interests, or behaviour (Art. 4(4)); an EMA taste vector is a working implementation of the definition (Chapter 36).

Non-profiling option. A feed variant not based on profiling, required of very large platforms by DSA Art. 38 and offered as a user switch under China's recommendation provisions; structurally, our trending fallback promoted to a feature.

Press publishers' right. The EU neighboring right (DSM Directive Art. 15) over online reuse of press publications by aggregators, with a "very short extracts" carve-out that snippet cards sit directly on.

Prompt injection. Untrusted content that alters an LLM's behavior (OWASP LLM01); for this book, a crafted headline attacking the dedupe judge through the text it reads.

Shilling attack. Fake users and interactions injected to promote or bury items in a recommender; the reason click feedback cannot be treated as ground truth without defenses.

The papers behind these terms are collected in the references. 👉