Gists

A synced, readable copy of my gists from gist.github.com/bl9. Each page is the gist content pasted verbatim; the table links to the local page and to the original gist.

14 gists, newest first. Last synced 2026-07-25.

🔒 = secret gist (unlisted on GitHub).

Two additional gists (codes, gistfile1.txt) hold private recovery phrases and are intentionally kept out of this published section.

Source: bmad-rcon-howto · bmad-rcon-howto.md · updated 2026-07-25 · 🔒 secret gist

Synced verbatim from gist.github.com/bl9.

Part 1.5 — Concept-by-concept how-to

Companion to bmad-deep-recon-foundations.md (what the terms mean) and bmad-deep-recon-guide.md (task-driven walkthroughs).

What this document is for. The foundations doc explains what a dimension, a topology, or a mode is. This one shows, for each concept, exactly what you type, exactly what comes back, and exactly what you do next. One concept at a time, in the order you'll meet them.

Section 20 does the same thing without BMAD at all — the same techniques applied in a plain Claude conversation, since dimensions, plan gates, source discipline and staleness are portable ideas, not BMAD features.

Fidelity note. Session excerpts are illustrative reconstructions. The interaction shape — what you're asked, what you can say back, what lands on disk — reflects the documented design. Exact wording lives in the skill's reference files and will differ. TOML key paths should be confirmed against your installed customize.toml.

Contents

• 0. Setup
• 1. Mode: Draft
• 2. Mode: Process
• 3. Mode: Run
• 4. Type and pack
• 5. Decision shape: explore vs select
• 6. Dimension
• 7. Topology
• 8. The plan gate, line by line
• 9. Effort presets and budgets
• 10. Rounds and leads
• 11. The firewall
• 12. Verification and red team
• 13. Claims and statuses
• 14. Briefs, digests, memlog
• 15. Staleness, refresh, deepen
• 16. The run folder and slug
• 17. Headless and scripting
• 18. Configuration in practice
• 19. Recovering when things go wrong
• 20. All of this in plain Claude, without BMAD
• 21. One-page cheat card

0. Setup

npx bmad-method install     # pick your IDE; deep-recon ships in core

Verify and look around:

/bmad-customize bmad-deep-recon

Read the options; change nothing yet. You're building a map of what's adjustable.

1. Mode: Draft

What Draft does: composes a research prompt carrying the pack's craft — pruned dimensions, freshness bars, source policy, a hard citation demand — tuned to the external tool you name. You paste it into that tool. No searching happens in your session.

Use it when: you already pay for ChatGPT/Gemini/Perplexity deep research, and the question is a broad public sweep.

What you type

> /bmad-deep-recon

draft a deep research prompt for Gemini: choosing a machine translation
vendor for EN↔FR, EN↔ES, EN↔JA, at roughly 40M characters/month.
Hard requirements: EU and Canada data residency, glossary and
do-not-translate support, and a DPA we can actually sign.
Type: competitive, select shape.

What comes back (shape)

DRAFTED PROMPT — for Gemini Deep Research
─────────────────────────────────────────
Research the commercial machine-translation vendor landscape for a buyer
with the following profile: [your volume, pairs, residency needs].

Cover these dimensions, in priority order:
  1. Pricing structure at 40M chars/month — not list price, the actual
     shape (per-character, committed-use tiers, minimums)
  2. Per-pair quality evidence for EN↔FR, EN↔ES, EN↔JA specifically;
     distinguish vendor-reported from independent
  3. Data residency options for EU and Canada; processing locations
  4. Glossary / do-not-translate / custom terminology feature parity
  5. Contractual posture: DPA availability, sub-processors, training-on-
     customer-data defaults, exit and portability terms

Source requirements:
  - Every claim must carry publisher, publication date, access date
  - Pricing and feature claims older than 6 months: flag as possibly stale
  - Mark vendor self-reported quality claims explicitly as self-reported
  - Prefer primary sources (vendor docs, pricing pages, trust centres)
    over review-aggregator sites

Output: a comparison with an explicit "could not determine" section.
─────────────────────────────────────────
Paste this into Gemini. When you have the report back, run:
   process <path to the report>

What you do

Check three things before pasting. This is the whole skill of Draft mode:

Edit the prompt freely. It's a starting point, not an output.

Why it works this way

The value isn't that Deep Recon can write a prompt — you could. It's that the pack's craft gets embedded automatically: the dimension priorities, the source policy, the self-reported-claim flagging, the "could not determine" demand. Those are the things people forget, and they're the difference between a usable report and a marketing summary.

2. Mode: Process

What Process does: takes a finished report from anywhere, files the original untouched, extracts claims into digests behind the firewall, gap-checks against the pack, and distills the standard cited research.md.

Use it when: you have a document — from Draft mode, from an analyst, from a colleague.

What you type

> there's a research report at ~/Downloads/mt-vendors.pdf, process it

What comes back

PROCESSING — competitive pack

  imports/mt-vendors.pdf              preserved, unmodified
  digests/d01-pricing-structures.md   14 claims extracted
  digests/d02-quality-evidence.md      9 claims  (6 flagged self-reported)
  digests/d03-residency.md             7 claims
  digests/d04-terminology-features.md  5 claims

GAP CHECK against the competitive pack
──────────────────────────────────────
COVERED
  pricing, language coverage, published quality claims, API surface

GAPS — not covered by the imported material
  ✗ sub-processor lists and processing-location commitments
  ✗ training-on-customer-data defaults and opt-out terms
  ✗ exit terms and data portability
  ✗ per-pair quality for EN↔JA specifically (only aggregate given)

research.md written — 35 claims, 22 sources, 4 gaps flagged

What you do

Read the gap list, not the report. That's where the value is.

Notice the pattern in that example: every gap is either contractual or specific to you. Hosted research tools crawl broad public commercial surfaces well and are systematically poor at narrow contractual questions and at your particular parameters. That's the division of labour to internalize — and it tells you what to do next:

> run a focused pass on the gaps only — sub-processors, training-on-data
  terms, exit/portability, and EN↔JA quality specifically.
  Effort standard, validation high.

This lands in the same run folder, because the slug is deterministic across the draft → process → refresh lifecycle. One decision, one artifact.

Why it works this way

Preserving the original untouched is provenance: you can always go back to what the tool actually said, rather than trusting the extraction. Extracting behind the firewall means your project context can't shape which claims get pulled out.

3. Mode: Run

What Run does: native research in your session — plan gate, parallel fan-out, verification at landing, cited synthesis.

Use it when: you want it now, or you need tools only your session has (internal MCP sources, your own files, an authenticated API).

What you type

> /bmad-deep-recon

I need to choose a reranking approach for our recommendation retrieval
stage. Decision horizon ~12 months. Candidates to include but not to
confirm: cross-encoder rerank, learned sparse rerank, MMR-only diversity,
no rerank at all.

Constraints: 200ms p99 for the whole retrieval stage, of which rerank can
have at most 40ms. Multilingual content. We self-host; no hosted APIs.

Type: technical, select shape. Effort: standard. Validation: high.

What comes back

The plan gate — covered in detail in §8. You approve or amend, and then:

ROUND 1 — breadth-first, 3 assistants
  [dim 1] latency/throughput ................ 8 sources → digest landed
  [dim 2] quality delta evidence ............ 8 sources → digest landed
  [dim 3] multilingual behaviour ............ 8 sources → digest landed

  verifying load-bearing claims as they land...
  § "Latency and throughput" written

LEADS FROM ROUND 1
  - Two sources disagree on cross-encoder latency by ~3x; the difference
    appears to be batch size and hardware, not the model
  - Learned-sparse rerank quality claims are almost entirely from one
    research group

ROUND 2 — depth on the two leads
  ...

What you do

Watch for two things and speak up:

• Aggregator pileup. If several digests trace to the same summary site, that's one publisher wearing several hats: "dimension 2 is leaning on aggregators — chase the primary papers."
• Leads not being chased. Disagreement is the most valuable signal in the run. If round 2 isn't going after it, say so.

You can interject at any point; you don't have to wait for a checkpoint.

4. Type and pack

How to set it. Three ways, in increasing explicitness:

> research the MT vendor landscape                    # type inferred
> competitive research on the MT vendor landscape     # named in prose
> Type: competitive.                                  # stated as a parameter

How to see what a pack changes

Ask before approving:

> what does the competitive pack prioritise, and what are its freshness
  windows?

You'll get the card: dimension priorities, source craft, freshness per claim class. Reading this once for each type you use is 5 minutes well spent — it's how you learn what to add at the plan gate, because you'll see what the pack considers out of scope.

Choosing when it's ambiguous

When two fit, run two. A vendor decision often needs competitive (what they offer) and domain (what the regulations require of anyone offering it). Merging them gives you a report that's shallow on both, because they want different sources.

5. Decision shape: explore vs select

Two independent axes: type = the subject matter, shape = the form the conclusion takes.

explore — you don't yet know the option space

> technical research: how are teams handling multilingual embedding
  drift as models get re-versioned? Explore shape.

Output: a structured understanding. Approaches, trade-offs, who does what, where the disagreements are. No recommendation, because you haven't defined what you're choosing between.

select — you're choosing, and you know the candidates

> technical research, select shape: choose between cross-encoder rerank,
  learned sparse rerank, and MMR-only, under a 40ms budget.

Output: everything explore gives you, plus a weighted matrix where the weights are your stated constraints.

The mistake, and how to avoid it

Running select too early narrows prematurely. If you name three candidates before you know whether there's a fourth category you've never heard of, the matrix will confidently rank three options out of a space you haven't mapped.

The sequence when you're genuinely unsure:

1. > ... explore shape          # map the space, cheap preset
2. read the report; now you know the real option space
3. > ... select shape, candidates: [what you learned]

Two cheap runs beat one expensive run that answered the wrong question.

6. Dimension

This is the concept with the highest leverage, so here it is as a full interaction.

Seeing them

Dimensions appear at the plan gate. You don't create them from nothing — the pack proposes, you edit.

DIMENSIONS (5)
  1. Latency and throughput under our budget
  2. Quality delta versus no-rerank baseline
  3. Multilingual behaviour and per-language variance
  4. Serving cost and hardware requirements
  5. Implementation and operational complexity

The four operations

DELETE — the one you'll use most.

> Drop dimension 5. We have the eng capacity; complexity isn't a
  differentiator for this decision.

Why: each dimension costs a full fan-out. At deep, cutting one saves roughly 15% of the run. Dimensions that don't bear on the decision aren't neutral — they dilute.

ADD — the one that determines whether the report is useful.

> Add a dimension: behaviour under filtered ANN search. Every benchmark
  I've seen measures unfiltered top-k, but our production path always
  applies filters, and I suspect that's where the approaches diverge.

Why: the pack is generic by design. Only you know your constraints. In that example the addition is the difference between a report that answers your question and one that answers a similar-looking question.

SPLIT — when one dimension is secretly two.

> Split dimension 2. "Quality delta" for us is two separate questions
  with different evidence bases: published benchmark deltas, and
  reported production A/B results. Keep them separate — the benchmark
  literature and the engineering-blog literature don't overlap.

Why: merged dimensions get one brief, one set of sources, and a shallow answer to both halves.

MERGE — when two would hit identical sources.

> Merge 1 and 4. Latency and hardware cost come from the same
  benchmarks; running them separately duplicates the search.

How to tell whether your dimension set is good

Three questions, in order:

1. Does it span the decision? If you imagine the report coming back complete, is there any way you'd still be stuck? That's a missing dimension.
2. Is anything here not load-bearing? If a dimension's answer wouldn't change your choice, delete it.
3. Would two of these send an assistant to the same sources? Merge them.

The test that catches most errors

For each dimension, finish this sentence:

"If this comes back saying ___, I choose differently."

If you can't complete it, the dimension isn't decision-relevant. Delete it or reframe it.

7. Topology

Topology is proposed for you. You mostly review it — but knowing how to override is what lets you fix a badly-shaped run before it costs you 40 minutes.

Seeing it

TOPOLOGY   breadth-first (dimensions are largely independent)

The three, and what each looks like in practice

Breadth-first — the default, and right most of the time.

ROUND 1 — breadth-first, 3 assistants
  [dim 1] ──→ assistant A ──→ digest
  [dim 2] ──→ assistant B ──→ digest      all in parallel
  [dim 3] ──→ assistant C ──→ digest

Use when the dimensions are genuinely independent. You get even coverage across everything.

Depth-first — one question, multiple angles, iterating.

> Use depth-first. This is really one question — whether the reported
  cross-encoder latency numbers hold on our hardware class — and the
  disagreement between sources IS the problem.

ROUND 1 — depth-first, 3 assistants, one question
  assistant A: vendor and library documentation
  assistant B: independent benchmarks and reproductions
  assistant C: production engineering reports
        ↓
  synthesise, identify what's actually in dispute
        ↓
ROUND 2 — chase the disputed mechanism

Use when you have one contested question and the contradiction matters more than breadth.

Straightforward — one assistant, small budget.

> This is a lookup. Straightforward topology, quick preset.
  Just: what's the current license on [model], and has it changed
  in the last 12 months?

Use for facts. Ten agents on an easy question just burns tokens — this is the most common waste in the whole system.

How to tell the proposal is wrong

Why knowing the algorithms helps

BFS and DFS are graph-traversal algorithms (see the foundations doc, §4.6). Their known weaknesses transfer directly:

• BFS holds the whole frontier in memory → breadth-first research burns more tokens per round. If you're context-constrained, fewer dimensions beats a lower preset.
• DFS can go far down a wrong corridor → depth-first research on a question that turns out to be a side issue wastes the entire run. Only go depth-first when you're confident the question is central.

8. The plan gate, line by line

The single hard stop. Here's how to read every line of it.

DECISION                                          ← ① is this actually your decision?
  Select a reranking approach for recommendation
  retrieval; 12-month horizon, 40ms budget.

DIMENSIONS (5)                                    ← ② prune, add, split, merge
  1. Latency and throughput
  2. Quality delta vs. no-rerank baseline
  3. Multilingual behaviour
  4. Serving cost and hardware
  5. Implementation complexity

TOPOLOGY   breadth-first                          ← ③ right shape?
EFFORT     standard — 3 assistants / 8 sources
           per round / 2 rounds                   ← ④ matches reversibility?
VALIDATION high; red_team off                     ← ⑤ do you have a prior?
FRESHNESS  benchmarks 12mo · libraries 6mo        ← ⑥ fast-moving field?
ESTIMATE   ~22 minutes                            ← ⑦ matches your patience?

Approve, or tell me what to change.

The seven checks

A realistic reply

Approve with changes:
- Drop 5; complexity isn't a differentiator here
- Split 2 into published-benchmark evidence vs. production A/B reports
- Add: behaviour under filtered ANN search, not just unfiltered top-k
- Keep breadth-first and standard
- red_team on — I'm going in expecting cross-encoder to win, so argue
  against it

Why you get exactly one gate

Constant confirmation trains you to click through. One gate that genuinely matters gets read. Everything after it is light checkpoints. Your attention here is the single highest-value input you give the system — a well-cited report answering a badly-framed question is the most expensive failure mode available.

9. Effort presets and budgets

Setting it

> ... effort: deep
> ... quick preset, this is just a sanity check

The presets

Precedence — memorize this

Your request beats a pinned setting beats the preset. Whatever you say in the moment wins. So you can pin standard in TOML and still say "deep for this one" without editing config.

Overriding one axis only

You don't have to take a preset whole:

> standard preset, but 3 rounds instead of 2 — I expect the first round
  to surface contradictions and I want them chased properly

Changing effort mid-run

> stop after this round and write up what you have — I have less time
  than I thought

The stop-and-write valve exists precisely for this. Files-first means everything already landed is kept.

How to choose

Scale to reversal cost, not to how interesting the question is:

10. Rounds and leads

A round is one full cycle of parallel searching, followed by reassessment. Leads are what round 1 produces that shapes round 2: contradictions, unexpected connections, gaps.

Watching leads form

LEADS FROM ROUND 1
  - Two sources disagree on cross-encoder latency by ~3x
  - Learned-sparse quality claims trace to one research group
  - No source addresses filtered-search behaviour at all

Steering round 2 — this is where you add most value mid-run

> For round 2: the 3x latency disagreement is the most important thing
  here. Chase the mechanism — I suspect it's batch size and hardware
  class rather than the model. Deprioritise the sparse-rerank lead;
  single-group evidence is enough for me to discount it.

Contradictions are the highest-value lead type. When two credible sources disagree, one of three things is true, and all three are worth knowing:

1. The claims are about different conditions (usually — and the conditions are the real finding)
2. One source is wrong (which tells you about that source)
3. The field genuinely disagrees (which means your decision carries more risk than the report's tone suggests)

When rounds stop

A dimension stops early when its questions are answered or a full round surfaces nothing new. You can force it:

> dimension 3 is done — stop there and move on

11. The firewall

The rule: your project context may shape what gets asked. It may never shape what counts as evidence. Subagents receive only their brief.

Working with it correctly

✅ Our retrieval stage has a 200ms p99 budget and we self-host.
   Given those constraints, what reranking approaches are viable?

Your context became a constraint on the question. Legitimate — that's the context of discovery.

Breaking it by hand — what not to do

❌ Here's our current architecture doc [paste]. Research whether our
   reranking approach is the right one.

You've supplied the conclusion as context. You'll get a well-cited report explaining that your approach is sound. It will be persuasive and worthless.

The tell: if you're pasting a design in and asking "is this right?", stop. Reverse the order:

✅ 1. What do the constraints and evidence say the right approach is?
   2. [read the report]
   3. Now compare your design yourself.

That ordering is the discipline. It costs you nothing and it's the difference between research and justification.

When you genuinely need internal context

Sometimes the research genuinely requires internal sources — your own metrics, an internal wiki, a private API. The firewall doesn't forbid this; it's about where the context enters:

> Use our internal metrics MCP for dimension 1 only — I need our actual
  p99 distribution, not published benchmarks. Keep dimensions 2-4
  external-only.

Scoping internal sources to a specific dimension keeps the rest of the run clean. What you're avoiding is internal context bleeding into dimensions where it would bias the external evidence gathering.

Checking it held

> did any dimension's findings rest on context I supplied rather than
  retrieved sources?

A clean run answers "no." If the answer is anything else, that's a finding about the report's reliability.

12. Verification and red team

Setting the level

> ... validation: high

Turning on red team, and why the phrasing matters

> red_team on. I'm going in believing cross-encoder rerank is the right
  answer — argue the strongest case against it.

Naming your prior explicitly gives the adversarial pass a target. "Red team this" is weaker than "here is what I believe; attack it."

The rule for when: turn it on exactly where you already believe the answer. Nobody walks into a security or privacy review neutral — you walk in believing you're fine. That's the condition it exists for. It's off by default because it costs tokens, not because it's optional for high-stakes work.

Challenging a specific claim mid-run

> claim [7] is doing a lot of work in this recommendation and it's from
  a vendor blog. Verify it independently or downgrade it.

What verification is not

The system checking a claim is the same system that produced it. That's a coherence check, not independent corroboration. Useful, weaker than the word suggests. Treat high validation as "the obvious errors were caught," not "this is confirmed."

13. Claims and statuses

Every claim carries a status: unverified → verified → disputed → overturned.

Reading them

[4] Cross-encoder rerank adds 35-60ms at batch size 32 on A10-class
    hardware.  [verified · 2 independent sources · pub 2026-02, 2026-04
    · accessed 2026-07-25]

[7] Learned-sparse rerank matches cross-encoder quality at 40% the
    latency.  [disputed · vendor-reported · single research group ·
    pub 2025-11 · accessed 2026-07-25]

What to do with each status

Why overturned claims aren't deleted

Because erasing the old belief destroys the audit trail that makes the new one trustworthy. If you can rewrite history, nobody can tell "this was always true" from "someone changed it." Same reason version control doesn't delete old commits.

Challenging the tally

> how many claims in the recommendation section are still unverified?

That count comes from the deterministic script, not the model's estimate — it's a real count.

14. Briefs, digests, memlog

Inspecting a brief — what one assistant was actually told

cat _bmad/planning/rerank-selection-2026-07/briefs/latency-throughput.md

Why look: if a dimension came back thin, the brief usually shows why — it was too broad, or too narrow, or it didn't mention your hardware class. That diagnoses whether to re-run the dimension or reframe it.

Inspecting a digest — what one assistant found

cat _bmad/planning/rerank-selection-2026-07/digests/d01-latency.md

Why look: trace one claim from research.md back through its digest to its source. Do this once, early. After you've seen how a claim in the report relates to what was actually retrieved, you'll never over-trust a polished report again.

Reading the memlog

tail -40 _bmad/planning/rerank-selection-2026-07/memlog

Append-only, so it shows the sequence — including claims whose status changed. ref= and status= entries are what the deterministic script tallies. Last status wins.

Why files-first exists

Everything lands on disk when it exists, not at the end. Consequences you'll actually feel:

• A run that dies at minute 30 resumes from disk
• The report builds in front of you instead of behind a spinner
• You can inspect intermediate work and intervene

15. Staleness, refresh, deepen

The staleness map

At the bottom of every report:

STALENESS MAP
  Fast-moving (re-check ~3 months)
    [4] latency figures — hardware and runtime releases move these
    [11] library versions and API surface
  Medium (~12 months)
    [7] published quality benchmarks
  Slow (~24 months)
    [2] license terms

Turn this into calendar entries the same day. This is the step everyone skips, and skipping it is what turns a living asset into a snapshot.

Refresh — time has passed

> refresh the rerank research

CONFIRMED (9)
CHANGED (2)
  [4] latency improved ~25% after a runtime release
  [11] library API changed; migration guide published
OVERTURNED (1)
  [7] the quality parity claim was retracted by the authors

⚠  Claim [7] is referenced by:
       docs/decisions/0021-reranking.md

That warning is why the lifecycle exists. What you do: open the ADR, note the change, decide whether it changes anything. Often it doesn't. The value is that the question got asked.

Deepen — a dimension was under-answered

> deepen the filtered-ANN-behaviour dimension

Drills one dimension without re-running the rest. Minutes, not tens of minutes.

Choosing between them

16. The run folder and slug

Finding your runs

ls _bmad/planning/
# rerank-selection-2026-07/
# mt-vendor-selection-2026-07/
# embedding-model-selection-2026-06/

The slug is deterministic — generated by the script, not the model — so draft, process, and refresh of the same decision all land in the same folder.

Anatomy

rerank-selection-2026-07/
├── imports/     # originals, untouched — provenance
├── digests/     # extracted claims — the working layer
├── briefs/      # what each assistant was told — file-based, never shell
├── memlog       # append-only sequence — the truth
└── research.md  # the canonical cited report — the artifact

Where to put it in your repo

docs/decisions/
├── 0021-reranking.md          # ADR: what we chose, and why
└── 0021-research.md           # the evidence, with provenance

The ADR tells you what in six months. The research file tells you whether it still holds.

17. Headless and scripting

output_format = auto renders HTML for interactive runs and plain markdown for headless or skill-invoked ones — no configuration needed.

A quarterly refresh job

# .github/workflows/research-refresh.yml
on:
  schedule: [{ cron: "0 6 1 */3 *" }]
jobs:
  refresh:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: |
          bmad-deep-recon refresh \
            --headless --output-format md \
            --workspace docs/decisions/research/
      - run: ./scripts/alert-on-overturned.sh

The alerting rule that keeps it alive

Alert on overturned, and on changed where the claim is referenced by a decision record. Ignore confirmed.

Get this wrong and you've built a quarterly notification everyone mutes by the second quarter. Get it right and you have decision-rot detection that costs nothing and requires nobody to remember.

18. Configuration in practice

The three layers

Merge rules: scalars override; tables deep-merge; arrays of tables keyed by code/id replace matches and append new ones; other arrays append. No removal mechanism — you override, you don't delete.

A sensible day-one personal config

# _bmad/custom/bmad-deep-recon.user.toml

[workflow]
output_format = "both"
validation    = "normal"
# red_team stays off globally — turn it on per-run, deliberately

A domain source policy — the highest-value override

[[workflow.source_policies]]
code = "regulatory"
preferred = ["eur-lex.europa.eu", "edpb.europa.eu", "ico.org.uk",
             "priv.gc.ca", "cai.gouv.qc.ca"]
banned_kinds = ["law-firm marketing", "compliance-vendor content marketing"]

[[workflow.source_policies]]
code = "security"
preferred = ["nist.gov", "owasp.org", "attack.mitre.org", "nvd.nist.gov"]
banned_kinds = ["security-vendor content marketing"]

Write a policy for a domain you know well. You can already tell good sources from bad in your own field — that expertise is exactly what a source policy encodes, and it's the single most effective configuration you can make.

Confirm key paths against your installed customize.toml; the shape and intent are what matter here.

19. Recovering when things go wrong

That last row is the important one. Re-framing until you get the answer you wanted is the one failure mode the whole system cannot protect you from, because you're driving.

20. All of this in plain Claude, without BMAD

Most of what makes Deep Recon good is discipline, not tooling. You can run a decent approximation in any Claude conversation. Here's the whole method.

20.1 The five moves

Move 1 — State the decision, not the topic.

I'm choosing between cross-encoder rerank, learned sparse rerank, and
MMR-only for a recommendation retrieval stage. Constraints: 40ms budget
for rerank, multilingual content, self-hosted only, 12-month horizon.

Before you research anything: propose 4-6 independent dimensions of
investigation, and tell me which you'd prioritise. Don't search yet.

That last line manufactures your own plan gate.

Move 2 — Prune the plan before any searching.

Drop the "implementation complexity" dimension — not a differentiator
for us. Add: behaviour under filtered ANN search specifically, since
benchmarks measure unfiltered top-k and our path always filters.
Now go.

Move 3 — Demand the source discipline explicitly.

For every claim: publisher, publication date, and note whether it's
vendor self-reported or independent. Flag anything published more than
12 months ago as potentially stale rather than stating it as current.
If several claims trace back to one original source, say so — I want to
know when three citations are really one.

Move 4 — Ask for the negative space. The single highest-value instruction, and the one people never give:

End with two sections:
- What you could not determine, and why
- Which of these claims will go stale fastest, and roughly when I should
  re-check them

Move 5 — Red-team it in a separate turn.

I'm inclined toward cross-encoder. Make the strongest case that I'm
wrong, using the evidence you gathered.

Separate turn matters — asking for a recommendation and its refutation in one breath produces a hedge instead of an argument.

20.2 Keeping the firewall

The discipline is entirely on you here, because there's no architectural barrier:

❌ Here's our architecture [paste]. Is our approach right?
✅ What do the constraints and evidence say? [read] ... then compare yourself.

If you must supply internal context, scope it and say why:

Use this only as a constraint on the question, not as evidence:
our p99 budget is 200ms and we self-host. Don't let it shape which
sources you weight.

Weaker than a real firewall — it's an instruction, not a wall — but stating it makes you notice when you're about to break it, which is most of the benefit.

20.3 Approximating the artifact

Ask for the output as a file with the metadata that makes it durable:

Write this to a markdown file with frontmatter: date, the decision it
supports, and a source list with publication and access dates.

Then commit it next to the ADR, exactly as you would with BMAD output. The lifecycle is the part you can fully replicate without any tooling — a dated file in version control and a calendar reminder from the staleness section gets you most of the refresh mechanism.

20.4 Approximating refresh

Three months later, open the file:

Here's a research file from three months ago. Re-check only the claims
it flags as fast-moving. Tell me: confirmed, changed, or overturned —
and flag anything overturned that the linked decision depends on.

20.5 What you can't replicate

Being honest about the gap:

The honest summary: roughly 70% of the value is discipline you can adopt today, in any conversation, with no installation. The tooling buys you enforcement, parallelism, and lifecycle — real things, but they're the multiplier on the discipline rather than a substitute for it.

If you take one thing from this document into tomorrow's work, take Move 1 and Move 4: state the decision before the topic, and always ask what couldn't be determined.

21. One-page cheat card

Framing

I'm choosing between X, Y, Z. Constraints: A, B, C. Horizon: N months.
Include those candidates but don't treat the list as the frame.
Type: <market|domain|technical|competitive|user-voice|academic-lit>
Shape: <explore|select>   Effort: <quick|standard|deep>
Validation: <normal|high|max>   red_team: <on|off>
Freshness: claims older than N months are history, not fact.

At the plan gate

Approve with changes:
- Drop dimension N — doesn't bear on the decision
- Add: <the thing only you know matters>
- Split N into <A> and <B> — different evidence bases
- Topology: <breadth-first|depth-first|straightforward>
- red_team on — I'm going in believing <X>

Mid-run

Dimension N is leaning on aggregators — chase the primaries.
Round 2: prioritise the <contradiction>; deprioritise <weak lead>.
Claim [n] is load-bearing and single-sourced — verify or downgrade.
Stop after this round and write up what you have.

Reading the report — in this order

1. Could not determine       4. Staleness map → calendar
2. Selection matrix weights  5. The recommendation, last
3. Self-reported flags

Lifecycle

refresh the <topic> research      # time passed
deepen the <dimension> dimension  # under-answered
/bmad-deep-recon                  # genuinely new question

The sentence that gates everything

"I am choosing between ___, ___ and ___, under constraints ___, ___ and ___, and I'll live with it for ___ months."

Can't write it? You're not ready to research — you're ready to brainstorm.

Companions: bmad-deep-recon-foundations.md (what the terms mean) · bmad-deep-recon-guide.md (task-driven walkthroughs, applied domains, epistemology)

Source: llm · llm.md · updated 2026-07-25 · 🔒 secret gist

Synced verbatim from gist.github.com/bl9.

How an LLM Actually Works

A mental model for someone who already knows transformers on paper but can't reconcile "matrix multiplications over vectors" with "it understood what I meant."

Contents

Part I — The object

• 0. The one thing to unlearn first
• 1. Concrete anchor: a real model's dimensions
• 2. Tokenization
• 3. The residual stream
• 4. Attention, dissected properly
• 5. The MLP — where knowledge is stored
• 6. Superposition
• 7. Circuits — worked examples
• 8. Depth = composition, and where intent lives
• 9. From vector to token: the output side

Part II — Putting it together

• 10. Full worked trace
• 11. Why prediction produces intent modeling
• 12. In-context learning
• 13. Chain of thought = serial compute

Part III — How it got that way

• 14. Training: SGD as a compiler
  • 14.1 The reframe
  • 14.2 What one SGD step actually does
  • 14.3 Why one update improves a million unrelated cases
  • 14.4 Why it cannot memorize
  • 14.5 The numbers
  • 14.6 The loss curve has structure
  • 14.7 Where modern gains come from
  • 14.8 The post-training stages
  • 14.9 The disanalogy: no reject state

Part IV — Applied

• 15. Worked scenario: a landing page
• 16. Worked scenario: a compiler pass
• 17. Failure modes
• 18. What is honestly still unknown

Part V — Reference

• 19. Every concept, defined and modeled
  • 19.1 The substrate
  • 19.2 Architecture
  • 19.3 Training
  • 19.4 Inference
  • 19.5 Interpretability
  • 19.6 Practical and operational
  • 19.7 Behavior and failure
• Appendix A: the compressed mental model
• Appendix B: the analogy that actually holds
• Further reading

0. The one thing to unlearn first

If your intuition for "text → vector" comes from embedding models (sentence-transformers, bi-encoders, HNSW indexes), that intuition is actively blocking you.

An embedding is a photograph. A transformer forward pass is a film — and the frames modify each other.

That is the whole reason one can only do retrieval while the other can write a compiler pass.

1. Concrete anchor: a real model's dimensions

Vague talk about "dimensions" is why this feels like magic. Here are actual numbers, using a Llama-3-70B-class model:

d_model (residual stream width)   8,192
layers                               80
attention heads per layer            64
head dimension                      128
MLP hidden width                 28,672
vocabulary                      128,256
context window                  128,000 tokens
total parameters             70,000,000,000

Derived facts worth internalizing:

• Attention head computations per token: 80 × 64 = 5,120. Not one similarity lookup — five thousand of them, each with its own learned notion of what to look for and what to bring back.
• Parameter split: roughly 1/3 attention, 2/3 MLP. Most of the model is memory, not routing.
• FLOPs per generated token: ~2 × 70B = 140 billion. For a 500-token answer, ~70 trillion floating point operations. It is not a lookup. It is a lot of computation.
• 8,192 numbers per position. In fp16 that's 16 KB. That 16 KB is the model's entire working belief about that word in that context, at that depth.

2. Tokenization — the layer everyone skips, and shouldn't

Text is split into subword tokens by BPE. Rough rule: 1 token ≈ 3.5–4 characters of English.

"the tests are failing"     → ["the", " tests", " are", " failing"]        4 tokens
"unfortunately"             → ["unfort", "unately"]                        2 tokens
"strawberry"                → ["str", "aw", "berry"]                       3 tokens
"getUserByID"               → ["get", "User", "By", "ID"]                  4 tokens
"128256"                    → ["128", "256"]                               2 tokens
"色即是空"                   → ["色", "即", "是", "空"] or worse            4+ tokens

Two consequences that explain real behavior:

1. The model cannot see letters. "strawberry" arrives as three opaque chunks. Asking it to count the r's is asking someone to count letters in a word they only ever heard spoken. This single fact explains a whole genre of "how is it so dumb about this" failures.
2. Non-English and code are tokenized less efficiently. The same sentence in Thai or Tamil may cost 3–5× the tokens of English. Since compute is per-token, this is also a fairness and cost issue in multilingual retrieval systems — the tokenizer is a quiet source of bias before any model math happens.

Each token ID indexes a row of the embedding matrix: 128,256 × 8,192. That row is the starting value of the residual stream at that position. It encodes essentially nothing contextual yet — just "this token, generically."

Position is added separately (modern models use RoPE, which rotates query and key vectors by an angle proportional to position, so attention naturally sees relative distance rather than absolute index).

3. The residual stream — the single most important object

Picture a spreadsheet:

• Columns = token positions in your prompt
• Rows = layers, 80 of them, bottom to top
• Each cell = an 8,192-dimensional vector

                 "the"     "tests"    "are"    "failing"
             ┌──────────┬──────────┬──────────┬──────────┐
Layer 80     │ v80,1    │ v80,2    │ v80,3    │ v80,4    │ ← this one becomes the prediction
             ├──────────┼──────────┼──────────┼──────────┤
    ...      │   ...    │   ...    │   ...    │   ...    │
             ├──────────┼──────────┼──────────┼──────────┤
Layer 2      │ v2,1     │ v2,2     │ v2,3     │ v2,4     │
             ├──────────┼──────────┼──────────┼──────────┤
Layer 1      │ v1,1     │ v1,2     │ v1,3     │ v1,4     │
             ├──────────┼──────────┼──────────┼──────────┤
Embedding    │ e(the)   │ e(tests) │ e(are)   │ e(fail…) │
             └──────────┴──────────┴──────────┴──────────┘

Critically, the update rule is additive:

x ← x + attention_output(x)
x ← x + mlp_output(x)

Not x ← f(x). The stream is a shared workspace that every component reads from and writes into. Nothing is overwritten; contributions accumulate.

This is why the mental model of "a bus" or "a whiteboard" is better than "a pipeline." Layer 3 can write a fact that nothing touches until layer 61 reads it. Components communicate across depth by leaving things in the stream.

Two directions of information flow:

• Vertical (within a column): MLPs. Enrich this position's representation with knowledge.
• Horizontal (between columns): Attention. This is the only mechanism by which positions see each other. Remove attention and you have 4 independent MLPs that have never heard of one another.

4. Attention, dissected properly

The textbook formula is softmax(QKᵀ/√d)V. The useful decomposition is different — split it into two independent learned circuits:

The QK circuit — "where do I look?"

Each position produces a query ("what am I looking for right now?") and a key ("what am I, advertised to others?"). Dot product, scaled, softmaxed over all earlier positions → attention weights.

This is exactly the retrieval you know from vector search, with three differences that change everything:

1. The corpus is your own prompt, not a static index.
2. The query is recomputed from the current stream state at every layer — so a position's question at layer 60 is informed by what it learned at layers 1–59.
3. It runs 5,120 times per token.

The OV circuit — "what do I bring back?"

Separately, each position produces a value vector, and the head has an output matrix. What gets copied back into the stream is attention_weights × values × W_O.

The decoupling is the point. What matched and what gets copied are different learned functions. A head can attend to the word "France" but write a country-code direction rather than the France representation. Retrieval systems don't do this; they return the document they matched.

Multi-head = parallel specialists

64 heads per layer, each with a 128-dim subspace. They specialize sharply:

That last row is worth pausing on. Some heads exist to say no. The network implements something like negation and inhibition using attention.

Causal masking

Position i can only attend to positions ≤ i. This is what makes training efficient — one forward pass over a 4,000-token document yields 4,000 training signals at once, each a legitimate "predict the next token" problem. It's also why the whole thing generates left to right.

KV cache — why the first token is slow and the rest are fast

• Prefill: your whole prompt is processed in one parallel pass. All keys and values for all positions are computed and cached. This is compute-bound and is the delay before the first token appears.
• Decode: each new token attends against the cache. Only one new column is computed per step. This is memory-bandwidth-bound.

Cache size scales linearly with context length. At 128K context this is tens of GB. This is the real reason long context is expensive — not the attention math, the memory.

5. The MLP — where knowledge is stored

Two-thirds of the parameters live here, and the structure is a key-value memory (Geva et al., 2021):

h = GELU(W_in · x)        # 8,192 → 28,672 : 28,672 pattern detectors
out = W_out · h           # 28,672 → 8,192 : each fires, writes its associated vector back

• Each of the 28,672 rows of W_in is a key: a direction in stream-space it detects.
• Each corresponding column of W_out is a value: what to add to the stream when that key fires.

So: "if the stream currently contains [Paris-ness] + [capital-of-relation-active], add [France-direction]."

Multiply by 80 layers: 2.3 million key-value slots, chained so later ones read the results of earlier ones. That's the fact store — and because it's composed across depth, it does multi-hop lookups, not just single retrievals.

6. Superposition — the answer to "how can dimensions do this"

The intuitive objection: 8,192 dimensions can't hold millions of concepts.

The intuition is wrong, and here's the geometry.

In 8,192 dimensions you can only fit 8,192 perfectly orthogonal vectors. But if you relax to "almost orthogonal" — say, any pair within 85–95° — you can fit an exponential number. This is the Johnson–Lindenstrauss lemma. Concretely, in d dimensions you can pack roughly exp(εd) near-orthogonal directions.

At d = 8,192 that number is astronomically larger than the millions of concepts a model needs.

The tradeoff: features interfere slightly. Every feature bleeds a little noise into every other. This is tolerable if only a handful of features are active at any moment — which is exactly the case, since any given token in any given context is only a few things at once. This is sparse coding, and the network learns it because it's the optimal use of limited width.

This has been verified empirically, not just theorized. Anthropic's sparse autoencoder work on Claude 3 Sonnet extracted tens of millions of interpretable directions from the residual stream: a Golden Gate Bridge feature, a "code with a security vulnerability" feature, a "sycophantic praise" feature, an "inner conflict" feature. Amplify one and the model's behavior changes accordingly and predictably.

So the correct reformulation:

A residual stream vector is not a point in an 8,192-concept space. It is a sparse sum of a few dozen active features drawn from a dictionary of ~10⁷ learned directions.

That reframing is what makes "8,192 numbers" stop feeling too small.

7. Circuits — worked examples of the machinery

These aren't hypotheses. They've been reverse-engineered and causally verified by ablation.

7a. Induction heads — the origin of in-context learning

Setup. A two-head circuit spanning two layers:

1. A previous-token head in layer L: at each position, copies the identity of the token before it into the stream. So position i now carries "I am preceded by X."
2. An induction head in layer L+1: its query is "find positions preceded by the token I currently am." It matches, then copies that position's own token to the output.

Net effect: given [A][B] … [A], predict [B].

Worked example — why the model can track your made-up variable names:

const zqBuffer = new ArrayBuffer(1024);
...200 lines...
const view = new Uint8Array(zq▮

zqBuffer appears nowhere in training data. There is no stored fact. Yet the completion is Buffer, near-certainly.

Trace:

• Position of the final zq emits a query: "where else have I seen zq?"
• Duplicate-token / previous-token machinery has tagged the earlier occurrence.
• The induction head attends to the token after the earlier zq — which is Buffer.
• OV circuit copies Buffer into the output stream.
• Unembedding turns it into a very high logit.

Why this matters far beyond variable names: induction heads generalize from exact-match copying to fuzzy, abstract pattern completion. The same circuit family is what lets few-shot prompting work. When you write:

sea otter -> loutre de mer
peppermint -> menthe poivrée
plush girafe ->

induction-style heads recognize the X -> Y structure, identify the transformation as translation, and apply it. No weights changed. The "learning" happened entirely as data movement inside one forward pass.

There is a striking empirical detail: during pretraining, induction heads form abruptly at a specific point, and the loss curve visibly bends at exactly that moment. In-context learning is not a smooth emergent haze — you can watch the mechanism switch on.

7b. The IOI circuit — indirect object identification

Prompt: "When Mary and John went to the store, John gave a drink to ___"

Correct answer: Mary. Both names appear; the model must pick the one that isn't the repeated subject. The full circuit was reverse-engineered in GPT-2 small (Wang et al., 2022) — 26 heads in three groups:

So the algorithm is literally: find all names → detect which one is duplicated → inhibit it → copy the survivor.

This is a genuine algorithm, discovered by gradient descent, implemented in attention patterns, verified by ablating individual heads and watching the answer flip to John. Nobody designed it. Nobody wrote "if duplicate then suppress." It was the lowest-loss way to model English discourse.

When you ask "how does it know what I meant" — this is the shape of the answer. Thousands of circuits like this, composed.

8. Depth = composition, and where intent physically lives

A rough map of what happens as you climb the 80 layers:

The middle band is the interesting one, and there's direct evidence for it.

Task vectors / function vectors

Take a few-shot prompt for some task (say, "translate English to French"). Read the residual stream at a mid layer at the final position. Average across many examples. You get a single vector.

Now take a completely unrelated prompt, with no examples and no instructions, and add that vector into the stream at the same layer.

The model starts translating to French.

Let that land. There is a specific vector, in a specific place, that is the concept "the task right now is: translate to French," represented independently of any wording that expressed it. You can extract it, store it, transplant it.

This is the mechanistic answer to "how does it know what I meant." Your intent stops being words and becomes a vector somewhere in the middle of the network. Different phrasings of the same request converge to nearby vectors — which is exactly why "make this faster," "this is too slow," and "optimize perf here" all get you the same kind of help.

Logit lens — reading the model's mind mid-computation

You can apply the final unembedding matrix to intermediate layers to see what the model would predict if it stopped there. Typical trajectory for a factual question:

Layer 10 : generic function words ("the", "a", "of")
Layer 30 : right category, wrong specifics ("city", "place", "region")
Layer 55 : right answer starting to appear, low confidence
Layer 70 : right answer dominant
Layer 80 : sharpened, high confidence

The prediction is built up, progressively refined. It is not looked up.

9. From vector to token: the output side

At the last position, after layer 80:

1. Unembed: logits = W_U · x → 128,256 raw scores.
2. Softmax with temperature: p = softmax(logits / T). T=0 → always the argmax (deterministic). T=1 → the raw learned distribution. T>1 → flatter, more random.
3. Truncate: top-k (keep the k best) or top-p / nucleus (keep the smallest set summing to p, typically 0.9–0.95). This cuts the long tail of garbage tokens.
4. Sample one token.
5. Append it to the input and run the entire thing again.

That last step is the loop, and it's why the model can't un-say something. Each token is committed and becomes input. There's no backtracking, no edit buffer.

Dissecting an actual distribution. For the prompt "The tests are failing after I added the cache":

token          logit    prob
" layer"        18.2    0.41
" ."            17.6    0.23
" to"           16.9    0.11
" middleware"   16.1    0.05
" and"          15.8    0.04
" invalidation" 15.4    0.03
...128,250 more tokens, together ~0.13

Two things to notice:

• The model is never certain. Even "obvious" continuations carry substantial mass elsewhere. Confidence is a distribution, not a boolean.
• The whole answer is present in embryo. The distribution already encodes "this is a technical debugging context, in English, mid-sentence, about caching." Every subsequent token narrows further.

10. Full worked trace

Prompt: "the tests are failing after I added the cache layer"

Note what is not in that sentence: no question mark, no request, no "please help," no specification of what output you want. Yet you'd get back a ranked list of likely causes with debugging steps. Let's trace how.

Prefill — layers 1 through 80, at the final position

Where did the "request" come from?

Nowhere in the text. It was inferred, and here's the mechanism, not the hand-wave:

During pretraining, the model saw millions of instances of this exact discourse pattern — a Stack Overflow post, a Slack thread, a GitHub issue, an IRC log — where someone states a symptom and the very next thing in the document is a diagnostic response. To minimize prediction loss on those documents, the model had to learn: "symptom stated by developer" → "diagnosis follows."

Post-training then sharpened which of the many possible continuations it produces (a helpful diagnosis rather than, say, a snarky reply or another user's unrelated question), but the underlying inference was already there in the base model.

Your intent was reconstructed from statistical regularities in how humans structure conversations. Not from a rule. Not from an intent classifier. From compression.

Decode — dissecting the actual answer

Suppose the response begins:

"A few likely culprits, roughly in order of how often they bite:

1. Cache state leaking between tests — the cache isn't cleared in teardown, so test B sees what test A wrote."

Dissect where each part came from:

Nothing here was retrieved. There is no stored answer to "tests failing after cache layer." Every token was computed. The reason it's right is that the machinery for reasoning about cache-and-test interactions is genuinely encoded in the weights, in the form of features and circuits that generalize.

11. Why next-token prediction produces intent modeling

This is the philosophical crux, and it's actually rigorous.

The setup. Minimize −log P(next token | all previous tokens) over ~15 trillion tokens of human-written text.

The critical observation: that text was not generated randomly. It was generated by humans with goals, knowledge, moods, expertise levels, and plans. The text is a downstream shadow of those hidden variables.

So to predict it well, you must model the hidden variables. There is no shortcut:

• To predict the next line of a Stack Overflow answer, you must model what the asker needs.
• To predict the next token of a legal brief, you must model the argument being constructed.
• To predict the closing brace of a C function, you must model the scope structure.
• To predict the punchline, you must model what the audience finds surprising.
• To predict the next move in a chess transcript, you must model the position.

And memorization is unavailable. 70B parameters versus 15T training tokens is roughly 200 tokens compressed per parameter. A lookup table is not on the table. The only representation that fits is general machinery.

This gives the chain:

accurate prediction → requires compression → requires generalization →
requires modeling the generating process → and the generating process is
a human with an intent

Intent modeling is not a feature that was added. It is the lowest-loss solution to the objective, and gradient descent found it because it was cheapest.

The same argument explains capabilities that look unrelated to text: arithmetic, code execution traces, spatial reasoning, theory of mind. Each is a compression win on some slice of the corpus.

12. In-context learning — learning without learning

Weights are frozen at inference. Nothing is stored. Yet:

Prompt:
  gxlfj -> 5
  hqm   -> 3
  aabbccdd -> 8
  zzz ->

Answer: 3. The task ("count characters") was never named, never trained on with these strings, and no gradient was computed.

What actually happens: induction-style heads recognize the input -> output structure, mid-layer circuits form a task representation from the demonstrations, and later layers apply it to the new input. The forward pass implements a learning algorithm.

There's evidence that in some settings this is literally gradient descent implemented inside the attention layers — a linear-regression task in-context produces internal updates that approximate one or more gradient steps on that task. The transformer learned to run an optimizer as a subroutine, because doing so lowers prediction loss on documents containing patterns.

Practical corollary: your prompt isn't a query. It's a program that reconfigures the model's effective computation. That's why prompt structure matters so much — you're not phrasing a search, you're building a temporary specialist.

13. Chain of thought = renting more serial compute

Fixed architecture means fixed serial depth: 80 layers. Some problems require more sequential steps than that, no matter how wide the model is.

The escape hatch is the output itself. Every emitted token is fed back as input, so:

serial computation available = 80 layers × number of tokens generated

Writing intermediate steps turns the context window into external working memory and buys unbounded serial depth.

Direct:  "What is 47 × 83?"  →  "3,901"        ← one 80-layer pass. Often wrong.

CoT:     "47 × 83
          = 47 × 80 + 47 × 3
          = 3,760 + 141
          = 3,901"                              ← ~40 tokens = ~3,200 layers of serial work

Each line is computed, then read back as ground truth for the next line. This is why "think step by step" works, and why reasoning-tuned models generate long internal traces. They're not being verbose — they're allocating compute.

Important caveat: the written trace is not guaranteed to be the actual causal path. Models can reach an answer internally and then generate a plausible-looking justification. Chain of thought is a compute mechanism and a post-hoc narrative, and telling those apart is an open research problem.

14. Training: SGD as a compiler

14.1 The reframe: the training loop is the compiler

If you know compilers, you already have the right structure in your head — it just sits one level up from where you're looking.

gcc pipeline
  C source  ──►  gcc  ──────────►  binary  ──►  CPU
  human-written  passes by hand     ~5 MB       can reject

LLM pipeline
  15T tokens ──►  SGD  ──────────►  weights ──►  forward pass
  human-written  passes by search   ~140 GB     never rejects

Same shape. Two differences carry all the weight:

1. Nobody wrote the passes. In gcc, a human wrote constant folding, a human wrote loop unrolling, and each is provably correct. In an LLM, a stochastic search over a 70-billion-dimensional program space found the passes. The 80 layers genuinely contain bracket matching, scope tracking, type propagation, dead-branch suppression — but they were discovered, not authored, and nobody has fully read them back.
2. The runtime has no reject state. More on this in §14.9; it's the single most clarifying fact about why the thing "accepts everything."

The closest thing in your world is superoptimization — searching the space of instruction sequences for one better than a human would write. SGD is superoptimization with a different objective, a vastly larger search space, and no proof of anything.

14.2 What one SGD step actually does

Forget "training" as a vague process. A single step is small and entirely mechanical. Say the batch contains this snippet scraped from GitHub:

def process(items):
    result = []
    for item in items:
        result.append(transform(item))
    return ▮

Step 1 — forward pass. The model produces a distribution over all 128,256 tokens at that position. Suppose it assigns p(" result") = 0.31.

Step 2 — loss. Cross-entropy is just the negative log of the probability assigned to the actual next token:

loss = −ln(0.31) = 1.17 nats

If it had been confident:  −ln(0.95) = 0.05
If it had been clueless:   −ln(1/128256) = 11.76

The loss is literally "how surprised were you." Nothing more.

Step 3 — backward pass. Backpropagation computes ∂loss/∂w for every one of the 70 billion weights — one number per weight saying "nudge me this direction to have been less surprised." This is the chain rule applied to a computation graph, executed in reverse.

Step 4 — update.

w ← w − learning_rate × ĝ

With a learning rate around 1e-4 (decaying on a cosine schedule), each weight moves by roughly one part in ten thousand. In practice the optimizer is AdamW, which keeps running averages of the gradient and its variance so each weight gets its own effective step size.

That's the entire step. In isolation it is nearly worthless — the model became microscopically less surprised by one snippet. Everything comes from doing this ~10⁶ times over ~10¹³ tokens.

14.3 Why one update improves a million unrelated cases

This is the crux, and it has no compiler analogue.

To predict result correctly, the network had to run a real inference chain:

• result was bound at the top of this scope
• it's an accumulator — mutated in a loop, never otherwise read
• Python functions conventionally return their accumulator
• we are at return, in this function's scope, not the enclosing one

The gradient does not reward "the string result." It rewards whichever circuits performed that inference: the variable-binding tracker, the scope-boundary detector, the accumulator-pattern recognizer.

And those circuits are shared. The identical variable-binding machinery fires for:

• Rust let bindings
• SQL aliases in a CTE
• pronoun resolution in English prose
• "the aforementioned party" in a contract
• a defined term in a mathematical proof

So one Python snippet measurably improves the model's handling of legal documents. Not by analogy — literally, the same weights, updated once, better everywhere.

This is the generalization mechanism, and it explains the otherwise baffling fact that 15 trillion tokens of miscellaneous internet text produces something that can write a compiler pass. There's a well-documented empirical consequence: adding code to the training mixture improves reasoning on non-code benchmarks, because code is unusually dense in explicit logical structure and, crucially, was checked by a compiler before being committed.

14.4 Why it cannot memorize, even if it wanted to

15,000,000,000,000  training tokens
    70,000,000,000  parameters
──────────────────────────────────
              ~200  tokens compressed per parameter

At 2 bytes per parameter in fp16, that's roughly 0.01 bytes of storage per training token. Lossless memorization is off the table by three orders of magnitude.

SGD is therefore under brutal compression pressure, and the resolution is the one you'd reach yourself if you were hand-writing a codec: store the rules, not the instances.

• A generative rule for "how Python scoping works" costs a few thousand parameters and covers billions of tokens.
• Memorizing those same tokens costs billions of parameters and generalizes to nothing.

Gradient descent finds the first solution because the second doesn't fit. Generalization is not an achievement the model unlocks. It is the only thing that fits in the space available.

This is also the cleanest answer to "how does it know almost everything." It doesn't store everything. It stores the generative structure of everything, and re-derives specifics at inference time — which is exactly why it's fluent on common things and confidently wrong on rare ones.

14.5 The numbers, for calibration

That last row deserves a pause. There is no drilling, no repetition, no curriculum of the same material. The model learns Python scoping from seeing millions of different correct examples one time each — which is precisely the setup that forces it to learn the rule instead of the instances. Repetition would encourage memorization; the single pass forbids it.

The 6 × params × tokens figure decomposes as 2 for the forward pass and 4 for the backward pass (backward costs roughly twice forward). Handy for estimating any training run.

14.6 The loss curve has structure

Training loss is not a smooth glide. Capabilities arrive as phase transitions:

loss
 │╲
 │ ╲___                    learning token frequencies, basic syntax
 │     ╲
 │      ╲__               ← visible bend: induction heads form
 │         ╲___             in-context learning switches ON here
 │             ╲__        ← arithmetic circuits consolidate
 │                ╲____
 └──────────────────────► tokens seen

The induction-head bend is the famous one. Before it, few-shot prompting essentially doesn't work. After it, it does. You can watch a mechanism switch on by looking at a loss curve — the capability is not a smooth emergent haze, it's a circuit forming.

This is the same shape as grokking: a network can sit at memorization-level performance for a long time and then abruptly generalize, when the general circuit finally beats the memorized lookup on the loss. You are watching a search find a better program and swap it in.

14.7 Where the modern gains actually come from

Architecture is roughly frozen. A 2019 transformer with modern normalization and RoPE is close to a 2026 one. Two things moved instead:

1. Data curation — the real secret sauce, and the least published part.

• Aggressive deduplication (near-duplicate web pages are actively harmful — they induce memorization)
• Quality classifiers trained to recognize "textbook-like" writing
• Mixture ratios: how much code vs. web vs. books vs. math vs. multilingual, tuned empirically
• Annealing: saving the highest-quality data for the last few percent of training, where the learning rate is tiny and the effect is disproportionate. The model's final impressions are its most durable ones.
• Synthetic data: using a strong model to generate clean training material for the next one

2. RLVR — reinforcement learning from verifiable rewards. This is the stage that made code good, and it's the one a compiler person will find most familiar:

sample N candidate solutions
  ↓
run the test suite / check the proof / execute the program
  ↓
reward the trajectories that passed
  ↓
gradient-ascend toward those

Unlike RLHF ("did a human prefer this?"), the reward here is ground truth. You can optimize against it hard without immediately Goodharting, because a test suite has no taste to exploit. This is why RLVR adds genuine capability while RLHF mostly adds manners.

It also explains the capability profile you observe in practice: models are disproportionately strong in domains with cheap automated verification (code, math, formal logic) and weaker where correctness is a matter of judgment.

14.8 The post-training stages in detail

The mental model for the middle three: the base model can simulate an enormous range of authors. Post-training doesn't teach it anything new; it selects which author it becomes when addressed as an assistant, and makes that selection reliable.

The cost asymmetry is stark — pretraining is 98%+ of the compute, post-training is a rounding error. Yet post-training is what makes the artifact usable. It is a thin, cheap layer of steering over an enormously expensive engine.

The known downside: RLHF optimizes a proxy for human approval, and approval is not truth. This is the mechanistic origin of sycophancy — reversing a correct position under pushback is what the reward signal rewarded.

14.9 The disanalogy that resolves "it accepts everything"

You're conflating two properties. Separate them and most of the mystery dissolves.

Property 1: "It accepts anything." This is architectural, and not impressive.

Look at where a compiler can fail versus where a transformer can:

There is no code path in a transformer for "I don't understand this input." No grammar, no parser, no error state, no partial-failure mode. Every stage is a total function over its input domain. Feed it line noise and it will confidently predict the next token of line noise.

That's not intelligence — it's the absence of a rejection mechanism. And it's the direct architectural cause of hallucination: a system that cannot fail to parse also cannot fail to answer. When the model doesn't know something, there is no representational slot for "unknown" and no branch to take. The output head produces a peaked distribution regardless, because that's the only thing it can do.

Everything that looks like the model saying "I'm not sure" is learned behavior from post-training, generated as ordinary tokens — not a structural check. It's a string, not a status code.

Property 2: "It knows almost everything." This is capacity plus compression — and it's less true than it feels.

It knows the high-frequency structure of nearly everything, and degrades smoothly into the tail:

frequency in training data
       high ──────────────────────────────► low
       │                                     │
   reliable                            plausible
   and correct                         and wrong
       │                                     │
  "how does a         "LLVM             "the signature of
   dominator           dominator          getAnalysisUsage
   tree work"          tree API"          in LLVM 17.0.3"

And the confidence is flat across that entire axis, because nothing in the architecture measures how well-supported a fact was. The useful heuristic that falls out: trust its structure, verify its specifics. Algorithms and architecture generalize from circuits. API signatures, version numbers, and citations are memorized facts, and memorized facts decay with training frequency and go stale with time.

15. Worked scenario: "build me a landing page for a dog-walking service"

Why does this work, when the exact page was never in training data?

The abstractions that exist in the weights:

• landing-page schema: hero, value proposition, social proof, pricing, CTA, footer
• local-service-business schema: service area, trust signals, booking mechanism
• HTML/CSS syntax circuits: matched tags, valid property names, correct nesting
• Design conventions: contrast, hierarchy, whitespace, mobile-first

What the forward pass does:

1. L25–45: composes landing-page ∩ local-service ∩ pet-industry into a joint plan. Pet industry contributes warm colors, photography-forward, informal-but-trustworthy tone.
2. L45–65: instantiates concrete sections in order.
3. Decode: emits HTML. Here the syntactic circuits do heavy lifting — an open <div> creates a strong attention signal that must eventually be discharged by </div>. This is bracket-matching implemented as attention, and it's why generated code is structurally valid even when semantically wrong.
4. Each token conditions the rest. Having written a hero section with a specific headline, the CTA later in the page will echo its phrasing — because that's what coherent documents do, and coherence is what was optimized.

Where it breaks and why: it will confidently produce a plausible-looking page that may use a CSS property that doesn't exist, or a framework API from a version that changed. It's generating from a distribution over plausible code, and plausible ≠ correct. It has no compiler.

16. Worked scenario: a compiler pass

Prompt: "Write an LLVM pass that eliminates redundant loads."

Knowledge assembled from the MLPs: SSA form, dominance relations, alias analysis, LoadInst and StoreInst APIs, the runOnFunction signature, memory dependence.

Reasoning assembled from circuits: the actual algorithm — walk the dominator tree, track available loads, kill on any aliasing store, replace dominated redundant loads with the earlier value.

Why it can do this: the LLVM codebase, its documentation, dozens of textbooks, and thousands of blog posts are in the training data. But more importantly, the abstractions generalize: "dominance," "available expressions," "kill sets" are structures the model represents, not strings it memorized. Ask for the same pass targeting a different IR and it adapts, because the structure transfers.

Where it breaks: it will get the algorithm right and the API version wrong. Reasoning circuits generalize; specific API signatures are memorized facts, and memorized facts decay with training-data frequency and go stale with time. The intuition to carry: trust its structure, verify its specifics.

17. Failure modes, and what each one reveals about the architecture

Failures are the best evidence for the mechanism, because they map onto architectural constraints exactly.

Notice that none of these are random. Every one falls out of a structural property. That's a good sign your mental model is correct — a correct model predicts the failures.

18. What is honestly still unknown

Intellectual honesty matters here, because a lot of writing on this topic overclaims.

Established: the architecture; superposition; specific circuits (induction, IOI, and dozens more); feature extraction via sparse autoencoders; task vectors; scaling laws.

Not established:

• No end-to-end account of any complex behavior. Nobody can trace why a specific 500-token compiler answer came out correct. We have footholds, not a map.
• Why in-context learning is as good as it is. Mechanisms are known in pieces; the full story isn't.
• Whether chain of thought is faithful to the actual computation.
• What the mid-layer "planning" representations really are — evidence exists that models plan several tokens ahead (e.g. picking a rhyme word before writing the line that leads to it), but the general structure is unclear.
• Whether anything like understanding is present in a sense beyond functional. This is partly empirical and partly a question about what the word means, and pretending it's settled in either direction is unjustified.

19. Every concept, defined and modeled

Each entry: what it literally is, then the mental model that makes it make sense. Where a compiler analogy helps, it's there.

19.1 The substrate: numbers and shapes

19.2 Architecture

19.3 Training

19.4 Inference

19.5 Interpretability

19.6 Practical and operational

19.7 Behavior and failure

Appendix A: the compressed mental model

1. Your text becomes tokens; tokens become vectors.
2. Those vectors are a shared workspace, one column per position, and every layer adds to rather than replaces them.
3. Attention moves information sideways — learned retrieval, recomputed 5,120 times per token, where "what to match" and "what to copy" are separate learned functions.
4. MLPs add knowledge vertically — ~2.3 million learned key-value slots.
5. Each vector holds a sparse combination of millions of learned features, packed via near-orthogonality. Width was never the constraint.
6. Depth is composition. By mid-network, your intent exists as an extractable vector, independent of your wording.
7. The final vector projects to a distribution over 128K tokens; one is sampled; the loop runs again.
8. It knows what you meant because predicting text written by intentional agents requires modeling intent, and compression made that the cheapest available strategy.
9. Generating tokens buys serial compute; that's what chain of thought is for.
10. It fails exactly where the architecture says it should. That's the strongest evidence the model above is right.

Appendix B: the analogy that actually holds

Not a search engine. Not a database. Not a person.

A very large, very fast, learned interpreter — running a program written by gradient descent, on data supplied by your prompt. The weights are the program. Your prompt is the input and a runtime reconfiguration of which parts of the program execute. The output is what that program computes.

The reason it feels like understanding is that the program it learned is, in substantial part, a model of the people who wrote its training data — including their intentions.

Further reading (primary sources)

• A Mathematical Framework for Transformer Circuits — Elhage et al., 2021
• In-context Learning and Induction Heads — Olsson et al., 2022
• Toy Models of Superposition — Elhage et al., 2022
• Interpretability in the Wild (the IOI circuit) — Wang et al., 2022
• Transformer Feed-Forward Layers Are Key-Value Memories — Geva et al., 2021
• Scaling Monosemanticity — Anthropic, 2024
• Function Vectors in Large Language Models — Todd et al., 2023
• Training Compute-Optimal Language Models (Chinchilla) — Hoffmann et al., 2022

Source: BMAD Recon · bmad-recon.md · updated 2026-07-25 · 🔒 secret gist

Synced verbatim from gist.github.com/bl9.

BMAD Deep Recon — Complete Guide

Covers PR #2611 (merged 23 Jul 2026) and docs/explanation/deep-recon.md. Written for someone with zero BMAD background, then applied to two concrete features: user recommendation and multilingual translation.

Reading order. bmad-deep-recon-foundations.md — what every term means and where its name came from. Then bmad-deep-recon-howto.md — concept-by-concept, exactly what you type and what comes back (plus how to apply the same techniques in plain Claude, without BMAD). Then this document — applied domains, epistemology, and task-driven walkthroughs.

Contents

• 0. The short, honest answer to "how useful is this?"
• 1. Background: BMAD concepts you need first
• 2. What PR #2611 actually changed
• 3. The three modes
• 4. Research types (the "packs")
• 5. How a native Run thinks
• 6. Why the reports hold up
• 7. The run folder, and research as a living asset
• 8. Where it sits in the BMAD lifecycle
• 9. Configuration
• 10. Research checkpoints across a feature's life
• 11. The full use-case catalog
• 12. Applied: a user recommendation feature
• 13. Deep dive: privacy and regulatory research
• 14. Deep dive: security and adversarial research
• 15. Deep dive: algorithm and technique-trend research
• 16. Applied: a multilingual translation feature
• 17. Deep dive: licensing, procurement and vendor risk
• 18. Best practices
• 19. Anti-patterns
• 20. Command cheat sheet
• 21. Transferable design lessons
• 22. The epistemology underneath all of this
• 23. Worked walkthroughs for software engineers
  • Walkthrough A — Native Run: choosing an embedding model
  • Walkthrough B — Draft → Process: the MT vendor landscape
  • Walkthrough C — The compliance gate, done without fooling yourself
  • Walkthrough D — Threat-modeling a ranking surface
  • Walkthrough E — Refresh, three months later
  • Walkthrough F — Deepen one dimension
  • Walkthrough G — Headless refresh in CI
  • Walkthrough H — The anti-walkthrough
  • 23.9 Team integration patterns
  • 23.10 Calibration — what to spend where
• 24. Sources

0. The short, honest answer to "how useful is this?"

Deep Recon is decision infrastructure, not feature-building infrastructure. It pays off only when the following are all true:

• You face a choice with real switching cost (a model, a vendor, an architecture, a metric definition, a locale strategy).
• The answer is not in your repo — it lives in vendor docs, benchmark papers, pricing pages, forum threads, standards documents.
• You will otherwise decide from the model's memory, which is stale and uncited.

It is not useful for:

• Writing the ranking service itself (that's bmad-dev / story workflows).
• Anything answerable by reading your own codebase (that's bmad-document-project).
• Facts you already know cold.

Practical read for your two examples:

Rule of thumb: if you can write the decision as "X vs Y vs Z, and I'll be stuck with it for 12 months," Deep Recon is worth 20–40 minutes. If you can write it as "how do I implement…", it isn't.

1. Background: BMAD concepts you need first

If you already know BMAD v6/v7, skip to §2.

1.1 What BMAD is

BMAD-METHOD ("Breakthrough Method for Agile AI-Driven Development") is an open-source framework that packages agentic software development into skills — structured markdown instruction sets an AI coding agent (Claude Code, Cursor, Copilot, etc.) loads on demand. It imposes a lifecycle: analysis → planning → architecture → implementation.

1.2 The vocabulary

1.3 Why "research" is a skill at all

Two failure modes drove it:

1. Hallucinated grounding. Ask a model about market size or vendor pricing and it answers from training data — confidently, uncitably, and often 18 months stale.
2. Uncaptured reasoning. Even when the research is good, it evaporates when the chat ends. Downstream planning has to re-derive it.

Deep Recon addresses both: no conclusion may rest on training data alone, and every engagement writes a durable cited research.md.

2. What PR #2611 actually changed

2.1 Consolidation

Three legacy skills — bmad-market-research, bmad-domain-research, bmad-technical-research — totalling ~5,136 lines of near-duplicate step files, were replaced by one skill, bmad-deep-recon, at roughly 650 lines. The three old names survive as v6 shims that forward with the corresponding type pre-set.

2.2 Relocation to core

The skill was moved from src/bmm-skills/1-analysis/ to src/core-skills/. Rationale: research is not code-project-specific (same precedent as brainstorming), so non-software and core-only installs get it too. On core-only installs, {planning_artifacts} falls back to {output_folder}. It also ships as a standalone marketplace plugin.

2.3 The v2 rework (the important part)

The first version was field-tested and found slow, token-heavy, and locally biased, with end-of-run verification degrading quality and subagent digests stranded in contexts that then died. The rework:

• Replaced the "acquisition mode" / engine-delegate registry with three modes: Draft, Process, Run.
• Added the research firewall.
• Went files-first — digests land on disk on arrival.
• Moved verification to landing time, not an end pass.
• Rescaled presets faster (standard is now 3 subagents / 8 sources / depth 2).
• Cut SKILL.md from ~3,989 to ~2,091 tokens by carving procedures into references/.

2.4 Determinism kit

scripts/recon_kit.py (stdlib-only, PEP 723, unit-tested) owns work the model shouldn't hand-derive:

• Citation marker ↔ appendix cross-check
• Memlog claim tally (ref= / status= convention, last status wins)
• Staleness re-check dates from per-class freshness windows
• Deterministic run-folder slugs across the draft→process→refresh lifecycle
• Escaped/validated source-appendix HTML

This is the transferable lesson: anything mechanical and verifiable should be a script, not a prompt. Models are bad at counting and date math and excellent at judgment.

2.5 Security findings addressed in review

Worth knowing because the same mistakes appear in your own agent tooling:

3. The three modes

Choosing

The trade, stated plainly: hosted deep-research products crawl wider per dollar because you've already paid for the subscription; Run costs your tokens and minutes but stays in context and can use every tool your harness has. On a bare "research X" ask, Deep Recon states this trade once and remembers your preference for the session.

4. Research types (the "packs")

A type selects a pack: a ~25-line card of prioritized dimensions, source craft, and freshness rules. Deep Recon infers the type from your ask, or you name it.

Custom types can be added via override TOML through bmad-customize.

Decision shape — a second, independent axis

• explore (default) — build understanding
• select — layers a weighted-matrix method over any type for a structured choose-between

Any type can end in a selection matrix. technical + select is the combination you'll use most for engineering decisions.

5. How a native Run thinks

Plan gate (decision, dimensions, topology, effort, time estimate)
        │ you approve
        ▼
   Topology choice
   ├── breadth-first ....... assistants split independent sub-questions
   ├── depth-first ......... assistants take different angles on one question
   └── straightforward ..... one assistant, small budget
        │
        ▼
Digests written to disk as each assistant returns
        │
        ▼
Verify load-bearing claims as material lands
        │
        ▼
Write the dimension's section (report grows in front of you)
        │
        ├── leads + budget remain? → follow the leads (next round)
        └── coverage or exhaustion → next dimension
        │
        ▼
Synthesis: cross-dimension insights, recommendations, staleness map
        │
        ▼
Mechanical citation check → research.md (+ optional HTML briefing)

5.1 The plan gate

The single hard stop. It shows the decision, the dimensions pruned to it, the topology, the knobs in force, and a realistic time estimate. Approve it and the run proceeds with light checkpoints rather than constant interrogation.

This is where you spend your attention. A bad plan gate produces a beautiful, well-cited, useless report.

5.2 Topology

Fan-out is a deliberate choice, not a default. Independent sub-questions get parallel assistants. One deep question gets several perspectives on the same material. A simple lookup gets one assistant with a handful of calls — ten agents on an easy question just burns tokens.

5.3 Effort presets

Precedence: request > pinned knob > preset. Anything you say in the request overrides.

Rounds follow leads — contradictions and unexpected connections from round one become round two's assignments. A dimension stops early when its questions are answered or a full round surfaces nothing new.

5.4 Other run craft

• Per-assistant tool-call budgets — prevents one assistant from eating the whole run.
• Short-query craft with an evaluate-before-next-call loop — read the result before firing the next search.
• Shared source-quality card — answer engines (Perplexity et al.) count as one publisher; chase their citations to the primaries.
• Stop-and-write valve — commit what you have rather than spiraling.

6. Why the reports hold up

6.1 Two standing rules, inherited verbatim by every subagent

1. No conclusions from training data. Model memory proposes questions and search strategy; every claim in the report traces to a source retrieved or imported during this engagement.
2. The research firewall. Your project files and briefs shape what gets asked, never what gets found. Assistants receive only their assignment. persistent_facts defaults to empty. A run therefore cannot come back quietly confirming whatever your local context already believed.

The firewall is the single most important idea in the whole design. Confirmation bias is the default failure mode of agentic research: give an agent your architecture doc and ask it to research alternatives, and it will find that your architecture is excellent.

6.2 Citation discipline

Every claim carries a publisher, a publication date, and an access date. Inline [n] markers resolve to a source appendix. A mechanical cross-check at finalize confirms markers and appendix agree — script, not vibes.

6.3 Verification levels

Verification runs as material lands, never as an end-of-run rewrite pass (the v1 lesson — a late pass degraded quality). The red-team pass is the single adversarial mechanism. An outcome adjusts a claim's status; it never licenses rewriting findings.

6.4 Freshness as part of truth

Each pack sets freshness windows per claim class. A market size from three years ago is reported as history, not as fact. This matters enormously in fast-moving areas — an MT quality benchmark from 2023 describes a world that no longer exists.

6.5 Files-first

Digests, extractions, and report sections hit the run folder the moment they exist. The conversation is a control channel, not storage. A run that dies mid-flight resumes from disk with nothing lost, and the report builds in front of you instead of behind a spinner.

7. The run folder, and research as a living asset

Each engagement gets one folder under your planning artifacts:

<planning_artifacts>/<run-slug>/
├── imports/      # originals, untouched
├── digests/      # extracted claims, behind the firewall
├── briefs/       # per-dimension assignments (file-based, never shell-interpolated)
├── memlog        # append-only truth
└── research.md   # the canonical cited report

research.md always exists as the machine-readable report. The HTML briefing is its regenerable face — this is the v7 artifact protocol: memlog = truth, markdown = distillation under contract, HTML = face. output_format = auto | html | md | both; auto renders HTML for interactive runs and markdown only for headless or skill-invoked runs.

The report ends with a staleness map naming which claims age fastest and when to re-check. That map powers the lifecycle:

• Refresh — re-verifies only stale claims, appends a delta report (confirmed / changed / overturned), and warns you when an overturned claim feeds a downstream artifact.
• Deepen — drills into one dimension without re-running the rest.

The overturned-claim warning is the sleeper feature. If your PRD's rationale rests on a pricing claim that just changed, you find out.

8. Where it sits in the BMAD lifecycle

bmad-deep-recon  →  research.md (+ metadata frontmatter)
        │
        ├──→ bmad-product-brief   (reads the summary; never reprocesses the original)
        ├──→ bmad-prd
        └──→ architecture / ADRs

Downstream skills consume the summary and metadata directly. That's the payoff of the fixed output shape — a PRD doesn't care whether the research came from Draft+Process or a native Run.

Entry points: /bmad-deep-recon, or through the Analyst agent (Mary), whose menu gained TS (select shape), CR (competitive), and UV (user-voice) alongside the existing research entry.

9. Configuration

9.1 Resolution layers

Per-skill config resolves through a layered merge:

1. _bmad/custom/{skill-name}.user.toml — personal, gitignored — wins
2. _bmad/custom/{skill-name}.toml — team/org, committed
3. The skill's own customize.toml — defaults

Merge rules: scalars override; tables deep-merge; arrays of tables keyed by code or id replace matching entries and append new ones; all other arrays append. There is no removal mechanism — you can't delete a base item, only override it (e.g. by code with a no-op) or fork the skill.

9.2 Deep Recon's knobs

Use /bmad-customize bmad-deep-recon rather than hand-authoring TOML.

10. Research checkpoints across a feature's life

Most teams treat research as a single event at the start of a project. It isn't. A feature passes through several gates, each with a different question, a different type pack, and a different cost of getting it wrong.

The pattern worth internalizing: almost everyone does gate 6 and skips gates 3, 4, and 5 — then discovers them during security and legal review, after the design is frozen and the sprint is committed. Gates 3–5 are cheap in week one and brutal in week ten. They're also the gates where external evidence matters most and where model memory is least reliable, which is exactly the profile Deep Recon is built for.

Gate 6 — the one everybody does — is the one where research adds the least, because you can usually just try both.

11. The full use-case catalog

The categories below are where Deep Recon earns its cost on feature work. Each maps to a type pack and a signal that tells you it's worth running.

11.1 Underrated advantages of running research as a skill rather than ad-hoc chat

Beyond the findings themselves:

• It's a durable team artifact. research.md with provenance frontmatter outlives the chat, the sprint, and the engineer. New joiners read why the decision was made instead of asking.
• It's audit evidence. Publisher + publication date + access date on every claim is precisely the shape compliance functions want. In a regulated environment, "we evaluated these options against these criteria on this date, here are the sources" is materially valuable on its own.
• It settles disagreements on evidence rather than seniority. A cited weighted matrix is harder to argue with than an opinion, and easier to change when the evidence changes.
• It's negotiation leverage. A sourced pricing and feature comparison is a real input to a vendor conversation.
• It parallelizes. Draft mode fires the expensive crawl into a subscription tool while you keep working; Process folds the result back in.
• It's automatable. The headless JSON contract means a quarterly refresh can run in CI, not on someone remembering.
• It inoculates against your own agentic pipelines. If you're building AI features, "no conclusions from training data" and the firewall are the disciplines you'll want in your own retrieval systems anyway. Deep Recon is a working reference implementation.
• It forces the decision to be written down. Half the value arrives at the plan gate, before a single search runs, because you had to state what you're actually deciding and under what constraints.

12. Applied: a user recommendation feature

Much of this feature is internal work — but the external parts are not the ones most engineers expect. Privacy and security carry more research value here than model selection does.

12.1 Decisions that genuinely need external evidence

12.2 Decisions that should not go to research

• Your index mapping, shard layout, refresh interval — read your cluster.
• Which features exist in your event stream — read your schema.
• Bucketing/assignment for A/B — that's design, not evidence.
• "How do I implement MMR" — you know this; it's twelve lines.

12.3 A worked plan-gate framing

Decision: pick the embedding model for the recommendation retrieval stage, deployable on our existing OpenSearch/HNSW infrastructure, for the next ~12 months.

Constraints: must handle our language mix; ≤200 ms p99 for the retrieval stage; self-hostable preferred; index memory budget is a hard ceiling.

Candidates to include (not to confirm): [3–5 named], plus any strong option the research surfaces.

Type: technical, select shape. Effort: deep. Validation: high.

Freshness bar: benchmark claims older than 12 months are reported as history.

Note the phrasing "to include, not to confirm." Naming candidates without that framing quietly asks the run to justify your shortlist.

12.4 What good output looks like

• A weighted matrix with your constraints as the weights, not generic ones.
• Every benchmark number carrying its dataset, date, and who ran it — vendor-run numbers flagged as such.
• An explicit "what we could not determine" section (Process mode gap-checks for exactly this).
• A staleness map that says, in effect: re-check the model landscape in 4 months; the license terms in 12.

13. Deep dive: privacy and regulatory research

Not legal advice. This produces an evidence base for a conversation with privacy counsel, not a substitute for one. The value is that you arrive at that conversation with cited primary sources and specific questions instead of vague anxiety.

13.1 Why this is the strongest case for Deep Recon

Privacy research satisfies every condition that makes the skill worth its cost, simultaneously:

• Entirely external. Statutes, regulator guidance, enforcement decisions, case law. None of it is in your repo, and none of it is inferable from your architecture.
• Jurisdiction-plural. A Toronto-built product with EU users sits under PIPEDA, Quebec Law 25, GDPR, and the DSA at once — and they don't agree on consent, notice, or automated decisioning.
• Fast-moving in exactly the layer that matters. Statute text is stable; regulator guidance and enforcement posture are not, and it's the guidance that determines whether your design passes.
• Load-bearing and expensive to reverse. You can swap an embedding model next sprint. You cannot swap a lawful basis after you've collected two years of data under it.
• The worst possible case for model memory. Legal reform moves constantly and a confidently-stated stale claim here is far more damaging than a stale benchmark. The no conclusions from training data rule is doing real work.

That last point is not hypothetical. Canadian federal privacy and AI reform has been through multiple bill cycles; I would not assert its current status from memory, and neither should any model you're relying on. That uncertainty is the argument for running the research, not a reason to skip it.

13.2 Split it into two engagements, not one

The common mistake is one sprawling "GDPR for recommender systems" run. That's two different decisions with two different source policies, and merging them produces a report that's shallow on both.

Run A — domain type: what the rules actually require

Dimensions to insist on at the plan gate:

• Lawful basis for profiling-based recommendation. Consent vs. legitimate interest, and specifically the contexts where legitimate interest has been rejected by regulators. Include the balancing-test documentation expectations.
• GDPR Art. 22 threshold. When does ranking become "automated decision-making producing legal or similarly significant effects"? Ordinary content ranking usually doesn't; ranking that gates access, pricing, or opportunity may.
• Art. 15 access and Art. 20 portability. What must you be able to show a user about their own profile? This has direct schema consequences.
• Art. 17 erasure scope. Does the right reach a trained model and derived embeddings, or only the raw interaction log? Regulator positions differ and have moved.
• Purpose limitation and secondary use. Interaction data collected for one surface being reused to train a ranker for another is a live issue.
• DSA recommender-system obligations. Parameter transparency, and the requirement to offer a non-profiling option — plus the thresholds that determine whether they apply to you at all.
• EU AI Act. Recommenders are generally not Annex III high-risk, but transparency duties and GPAI obligations can still attach if you're using a foundation model in the pipeline. Get the classification question answered explicitly rather than assumed.
• Quebec Law 25 automated-decision notice. Stricter than PIPEDA on this point and routinely missed by teams who "did GDPR."
• PIPEDA and Canadian reform status. Current, not remembered.
• US state law patchwork. Where sensitive-inference and opt-out-of-profiling provisions bite.
• Children's and minors' data. Separate, stricter, and often triggers age-assurance obligations.
• Cross-border transfer posture. Applies to embedding inference calls, which teams frequently forget are transfers.

Run B — technical type: how to actually satisfy the above

• Right-to-erasure vs. HNSW reality. Vector indexes don't truly delete — you get a soft delete plus eventual segment merge. What's the real deletion latency, is it bounded, and does a bounded-but-slow deletion satisfy the requirement? This is a genuine engineering-meets-law question with published discussion.
• Are user embeddings personal data? Increasingly treated as yes. If so, every derived vector inherits the obligations.
• Embedding inversion attacks. Published work shows embeddings can leak the attributes they were built from — which converts a "we only store vectors" argument into a liability.
• Membership inference. Can an attacker determine whether a given user was in the training set?
• Machine unlearning. What's genuinely deployable versus what's a paper with a toy benchmark. Be ruthless here.
• Differential privacy / cohort aggregation. What does the privacy budget actually cost you in NDCG?
• Retention windows. How much interaction history does the model measurably need, versus how much you're keeping by default? Data minimization is easiest to argue when you've measured the marginal value of older data.
• Consent propagation. When a user withdraws consent, what happens to precomputed recommendation caches, materialized candidate sets, and warm segments?
• Pseudonymization that survives joins. Whether your scheme actually resists re-identification given the other datasets you hold.

13.3 Configuration that matters for regulatory research

The firewall discipline is at its most valuable and most tempting to break here. The instinct to say "here's how we currently handle deletion, is it compliant?" produces a motivated report. Ask "what does the law require for deletion in vector stores?" and compare afterward.

13.4 What to hand counsel

The output should let a lawyer work in an hour rather than a week:

• Requirements traced to primary sources, with dates, not to secondary summaries
• An explicit "could not determine" section — jurisdictional gaps are the most important finding
• Conflicts between regimes surfaced as conflicts, not averaged away
• A staleness map that says which obligations to re-check and when
• Your specific open questions, framed as questions

Then keep it refreshed. The "overturned claim feeds a downstream artifact" warning is at its most valuable here: if a lawful-basis assumption in your PRD stops holding, you want the delta report to tell you, not an auditor.

14. Deep dive: security and adversarial research

Security is the other category I'd rank above component selection, and it's the one most consistently skipped until a review board forces it.

14.1 Why it fits the skill

Threat knowledge is published, adversarial, and perishable — the three properties the verification and freshness machinery exist for. It's also a domain where the model's training data is systematically incomplete: novel attack classes appear faster than they get absorbed.

14.2 Threat dimensions for a recommendation feature

Most engineers threat-model the service (authn, injection, rate limits) and skip the ranking system as an attack surface. Both belong in the run.

Manipulation and integrity

• Shilling / profile-injection attacks — coordinated fake interaction patterns that promote or bury items. Published attack models and detection approaches exist; this is a solved-ish literature that most teams have never read.
• Data poisoning of the training or feedback loop, including slow-drip attacks below anomaly thresholds
• Popularity and trending manipulation, and feedback-loop amplification
• Catalog enumeration via recommendation responses

Confidentiality and inference

• Embedding inversion — recovering source attributes from a vector (overlaps §13, but it's a security control question here)
• Membership inference — determining whether a user or item was in training data
• Cross-user leakage — inferring another user's behavior from changes in shared segments, co-occurrence surfaces, or "people also viewed"
• Model extraction — reconstructing your ranking function through systematic querying
• Cache and shared-segment poisoning

Access control

• IDOR and horizontal privilege issues on personalization endpoints — mundane, and the most commonly exploited

14.3 If there's an LLM anywhere in the pipeline

Add these dimensions explicitly:

• Indirect prompt injection via user-generated content, item metadata, or retrieved documents — the dominant risk class for retrieval-backed systems
• Tool-use boundary design and least-privilege for agentic components
• Output handling — treating model output as untrusted before it reaches a shell, a template, a query, or a browser

PR #2611 is itself the canonical worked example: a research brief was being interpolated into a double-quoted shell template, so $(), backticks, or a quote break inside researched or imported content could shape a command — amplified by a permission-skipping flag. The fix was structural, not a filter: briefs are written to a file and the template substitutes a skill-generated path, so untrusted text never touches a command line. Take that pattern wholesale into your own agent tooling. Filters on untrusted text fail; changing the channel so the text never reaches an interpreter doesn't.

14.4 Supply chain

• Model provenance — where the weights came from, what's actually verifiable
• Serialization risk in model artifacts (pickle-family deserialization remains a live RCE vector)
• Dataset provenance and whether training data carries obligations
• Dependency health — release cadence, maintainer count, open-issue and security-advisory trends
• CVE landscape for your serving stack, index, and inference runtime

14.5 Configuration for security research

14.6 Turning the report into work

The output shouldn't stop at a threat list. Convert it: threat → likelihood/impact for your deployment → control → story. Threats you consciously accept get written down as accepted, with the citation attached. That record is what makes the next security review fast, and it's the thing that's always missing when a reviewer asks "did you consider X?"

15. Deep dive: algorithm and technique-trend research

This is the weakest of the categories you raised — but it's rescuable if you reframe it.

15.1 First, disambiguate "EMA"

Two unrelated research problems share the acronym:

1. Exponential decay of user interest profiles — recency-weighted aggregation of a user's interaction vectors, so last week outweighs last year. A ranking-features question.
2. EMA teacher weights in self-supervised training — momentum encoders in the BYOL/DINO/MoCo family, where a slowly-updated copy of the network supervises the fast one. A representation-learning question.

They share nothing except the smoothing formula. A run that doesn't disambiguate produces a report that's half-relevant twice over.

15.2 Why "trends in algorithms like X" is the anti-pattern

No decision means no dimension pruning, which means unbounded scope, maximum fan-out, and an expensive encyclopedia entry you'll skim once. Convert it into a decision:

15.3 The dimension you must add by hand

Negative results, failed reproductions, and industry postmortems. Name it explicitly at the plan gate.

The literature is systematically positive-biased: papers report wins, and a survey that only aggregates claimed improvements is worse than nothing — it's confidently wrong in a specific, actionable direction. Recommender systems have a well-documented reproducibility problem in particular, where reported gains over properly-tuned simple baselines often fail to replicate.

This is also where "no conclusions from training data" pays off most, because model memory of an ML technique is weighted heavily toward the abstract of the original paper — the most optimistic sentence anyone ever wrote about it.

15.4 Read benchmark claims like an adversary

Require every reported number to carry:

• The dataset and its known idiosyncrasies
• The date — a 2023 retrieval benchmark describes a world that no longer exists
• Who ran it — self-reported vendor numbers flagged as such
• Tuning-budget parity — was the baseline tuned as hard as the proposed method? Usually not.
• Whether the gain survives at your scale and data regime, which is rarely the paper's regime

15.5 Know where research stops and your eval starts

For a parameter like an EMA half-life, research gives you the prior and the conventions — plausible ranges, how others parameterize per surface, what failure looks like when decay is too aggressive. It cannot give you the answer. Your offline replay against your own interaction logs decides it, and your A/B confirms it.

Running a deep engagement to pick a number your own data can answer in an afternoon is the clearest form of research-as-procrastination. Use quick for the prior, then go measure.

16. Applied: a multilingual translation feature

This is the higher-value case, because almost every decision is external.

16.1 Decision map

16.2 Traps this pack is built to catch

• Vendor-reported quality numbers. The source-quality card treats an aggregator as one publisher; verification at high cross-checks the claim classes the recommendation rests on. Insist that self-reported quality claims are marked as such.
• Language-coverage asymmetry. "Supports 100+ languages" is a marketing claim; per-pair quality varies by an order of magnitude. Make coverage-quality a named dimension, not a checkbox.
• Stale benchmarks. Set the freshness bar aggressively — 12 months, not 36.
• Pricing shape, not just price. Per-character vs. per-token vs. per-request vs. committed-use changes the ranking entirely at your volume. State your actual volume at the plan gate.

16.3 Suggested mode

Draft → Process → focused Run on the gaps:

1. Draft a prompt for whichever deep-research product you subscribe to. Vendor landscapes are exactly what hosted crawlers are good at.
2. Run it there; bring the report back.
3. Process it — original preserved in imports/, claims extracted, gaps flagged against the competitive pack.
4. Run natively on the two or three gaps (usually: your specific language pairs, and anything requiring internal or MCP-only sources).

16.4 Then refresh

Set a calendar reminder from the staleness map. MT pricing and model versions turn over on a roughly quarterly cadence. refresh the translation vendor research gives you a delta report — confirmed / changed / overturned — with a warning if an overturned claim underpins something you already shipped a decision on.

17. Deep dive: licensing, procurement and vendor risk

The most-skipped category, and the one that kills projects latest in the cycle — after integration, when unwinding is most expensive.

17.1 Model and dataset licensing

• Open weights ≠ open source. Many popular model licenses carry field-of-use restrictions, acceptable-use policies, monthly-active-user thresholds that flip commercial terms, or naming and attribution requirements. Several forbid using outputs to train competing models — which matters if you plan to distill.
• Dataset provenance. A permissively-licensed model trained on a non-commercial dataset is a real and common trap, particularly for multilingual and speech models.
• OSS license compatibility with how you actually distribute — SaaS, on-prem, embedded, and client-side each trigger different obligations.
• Output IP and indemnification. Who owns generated translations? Does the vendor indemnify you against third-party claims, and under what conditions do they void it?

Type: domain (or technical when it's mostly library compatibility). Preferred sources: the license texts themselves, model cards, official FAQs. Banned: summaries of licenses. Read the primary text; this is a category where paraphrase loses the operative clause.

17.2 Procurement and vendor assurance

If a vendor will process your data — an MT API, a hosted embedding endpoint — these determine whether you're allowed to use them at all, and they should be researched before you build against the API, not after:

• SOC 2 Type II / ISO 27001 status and scope
• DPA availability, SCCs, and transfer mechanism
• Sub-processor list and whether it's stable and notified
• Data residency and processing-location commitments
• Training-on-customer-data policy — the default matters, and so does whether opting out changes pricing or features
• Retention and deletion commitments, with actual timelines
• Breach notification SLA
• Exit terms and data portability — the exit cost is the real measure of lock-in
• Uptime SLA, rate limits, and what happens at the ceiling

Type: competitive. This is the research whose absence produces the sentence "legal won't approve the vendor we've already integrated."

17.3 Dependency and deprecation risk

Cheap to run, disproportionately useful:

• Release cadence and whether it's slowing
• Maintainer count and bus factor
• Open-issue and security-advisory trend lines
• Corporate backing, and whether the backer has a history of sunsetting
• Migration paths that already exist for people leaving

A quick preset run answers most of this and has saved more projects than any model comparison.

18. Best practices

18.1 Framing

1. Lead with the decision, not the topic. "Research embeddings" gets you an encyclopedia entry. "Pick the embedding model for our recommendation retrieval stage under these constraints" gets you a recommendation.
2. State your constraints as constraints. Latency budget, memory ceiling, language mix, volume, license posture, deploy target. These become matrix weights.
3. Name candidates as inclusions, never as the frame. Otherwise you get a confirmation exercise.
4. Declare your freshness bar when the field moves fast. Don't accept the pack default silently.
5. Pick the decision shape deliberately. select for choose-between, explore when you don't yet know the option space. Running select too early narrows prematurely.

18.2 At the plan gate

6. Read every dimension. This is your only hard stop. Delete dimensions that don't bear on the decision — each one costs a full fan-out.
7. Sanity-check the topology. Independent sub-questions → breadth. One contested question → depth. A lookup → straightforward. If it proposes six assistants for something simple, say so.
8. Match effort to reversibility. quick for a decision you can undo next sprint; deep + validation = high for something that becomes load-bearing.
9. Check the time estimate against your patience. Approving a 40-minute run you'll abandon at minute 12 wastes everything except what's already on disk.

18.3 During and after

10. Trust the firewall; don't defeat it. Resist the urge to paste your architecture doc "for context." Project context is legitimately framing; it is inadmissible as evidence. Feeding it in is how you get a report that agrees with you.
11. Chase answer-engine citations to primaries. If the report leans on an aggregator, that's one publisher, not three.
12. Read the "could not determine" section first. Gaps are more informative than confirmations.
13. Check the staleness map and act on it. Research that's never refreshed is research that silently becomes wrong.
14. Keep research.md in version control alongside the PRD it grounds. The provenance frontmatter is what makes the decision auditable in six months.

18.4 Cost and token discipline

15. Draft → Process is the cheap path. If you already pay for a deep-research subscription, the round-trip is one paste and it crawls wider per dollar.
16. Default to standard. The presets were rescaled downward because v1 was too slow and too token-heavy. Reach for deep deliberately.
17. Use deepen instead of re-running. Drilling one dimension is far cheaper than a fresh engagement.
18. Use refresh instead of redoing. It re-verifies only what's stale.

18.5 Hygiene

19. Turn on red_team for decisions with a strong prior. It's off by default. If you walked in already believing the answer, buy the adversarial pass.
20. Redact before quoting people. The user-voice pack now requires stripping usernames, handles, emails, and identifying URLs from verbatim quotes — carry that rule into anything you paste into a PRD.
21. Never shell-interpolate researched text. The PR's critical finding. Researched and imported content is untrusted input; pass it by file path, not on a command line — especially near any --dangerously-skip-permissions flag.

18.6 Domain-specific craft

22. Run the compliance and security gates in week one, not week ten. Their findings are design constraints. Discovered late, they're rewrites.
23. Tune source policy per domain, not once. Regulatory research wants regulators and bans law-firm blogs; security research wants NIST/OWASP/CVE and bans vendor marketing; academic research wants venues and bans press releases. The default policy is a compromise; override it.
24. Set freshness per claim class, not per run. Statute text, regulator guidance, and enforcement posture age at wildly different rates — and so do attack classes, advisories, and tooling.
25. Always add a "negative results and failures" dimension to any technique or trend research. Published literature is positive-biased and won't volunteer it.
26. Split regulatory research into requirements and implementation. One run answers "what must be true"; a second answers "how do we make it true in a vector index." Merging them makes both shallow.
27. Demand a "could not determine" section on compliance work. In regulatory research the gaps are the finding — an unresolved jurisdictional question is more actionable than five confirmed ones.
28. Convert security output into accepted-or-mitigated decisions, with citations attached. A threat list is not a deliverable; a threat register with dispositions is.
29. Know where research stops. Priors and conventions come from the literature; the actual parameter comes from your offline replay. Research that could be settled by an afternoon of measurement is procrastination.

19. Anti-patterns

20. Command cheat sheet

Legacy IDs (bmad-market-research, bmad-domain-research, bmad-technical-research) still work and forward with the type pre-set.

21. Transferable design lessons

Independent of whether you adopt Deep Recon, these generalize to any agentic system you build:

1. Consolidate near-duplicate prompt trees into one skill with data-driven variation. 5,136 lines → ~650 by turning three workflows into six ~25-line policy cards.
2. Keep the entry point lean; defer detail. ~4k → ~2.1k tokens by carving procedures into just-in-time reference files. Entry-point tokens are paid on every invocation.
3. Files-first, conversation-as-control-channel. Anything valuable goes to disk on arrival. Contexts die; disks don't.
4. Verify at landing, not at the end. A late pass degraded quality in field testing.
5. Mechanical work belongs in a tested script. Counting, cross-referencing, date math, slug generation.
6. Build an explicit bias firewall. Separate "what shapes the question" from "what counts as evidence," and enforce it at the subagent boundary.
7. Choose fan-out topology deliberately. Parallelism is a cost, not a virtue.
8. Treat tool output as untrusted input. File-based handoff over shell interpolation, protocol allowlists, escaping at the boundary.
9. Give artifacts a lifecycle. Staleness maps, delta refreshes, and downstream-impact warnings turn a document into an asset.
10. Single-writer rule for parallel agents. Fan out reads; serialize writes through the lead.

22. The epistemology underneath all of this

This section assumes no philosophy background. Every term is defined in plain language before it's used.

The PR has a section literally headed "Epistemics and reliability." That's not decoration — it's the honest label for what the whole design is. Read it this way and the mechanisms stop looking like a feature list and start looking like a single coherent argument.

22.1 What "epistemology" means

Epistemology is the branch of philosophy that asks: what is knowledge, and what makes a belief justified?

The classic starting point: knowing something requires three things.

1. It's true
2. You believe it
3. You have good reason — justification — for believing it

Miss any one and you don't have knowledge. If it's true but you have no grounds, you got lucky. If you have grounds but it's false, you're mistaken.

Why this matters here: an LLM produces sentences that satisfy #1 and #2 by accident and skip #3 entirely. It asserts things without holding beliefs, and its confidence is uncorrelated with whether it has grounds. It can tell you something perfectly true while having no justification whatsoever — the sentence just came out fluent.

There's a name for the case where you have a true, believed claim that's only accidentally right: a Gettier case (after a famous three-page paper from 1963). Picture a stopped clock that happens to show the correct time when you glance at it. You now believe something true, for a reason that gives you no actual grounds. Every uncited model output is a potential stopped clock.

Deep Recon's whole purpose is to bolt a justification layer onto a system that natively has none.

22.2 The core concepts, defined

22.3 Every mechanism is an epistemological commitment

Now the mapping. Each row is a design choice in the skill and the philosophical position it encodes.

22.4 The elegant one: the research firewall

This is the deepest idea in the design, and it's a direct implementation of a distinction from philosophy of science.

Context of discovery vs. context of justification (Hans Reichenbach, 1930s). These are two separate questions about any belief:

• Discovery: how did you come up with this idea? A hunch, a dream, a colleague's offhand remark, your existing assumptions — anything goes. There's no wrong way to generate a hypothesis.
• Justification: what makes it true? Here almost nothing goes. The grounds have to be independent of how you happened to think of it.

The classic illustration: the chemist Kekulé reportedly hit on the ring structure of benzene after daydreaming about a snake biting its tail. Perfectly fine as discovery. Utterly irrelevant as justification — the structure is right because of the evidence, not because of the dream.

The firewall implements exactly this split at a process boundary:

• Your project files, architecture docs, and briefs are allowed to shape which questions get asked — that's discovery, and your priors are a legitimate source of good questions.
• They are inadmissible as evidence — that's justification, and it has to come from outside.
• Subagents receive only their assignment. persistent_facts defaults to empty.

Why this is unusually good engineering: most bias mitigation is an instruction — "be objective," "consider alternatives." Instructions fail, because a system that wants to agree with you will find a way. This is architectural. The subagent cannot be biased by context it was never given.

The failure it prevents is the single most common one in agentic research: hand an agent your design doc and ask it to research alternatives, and it comes back explaining that your design is excellent. Not through deceit — through ordinary confirmation bias, which is what you get when the thing generating hypotheses also gets to grade them.

22.5 Where the epistemology is genuinely weak

Being honest about this matters more than admiring the design.

1. The independence assumption is shaky. Fan-out to multiple assistants borrows its logic from a result about crowds: if many judges each do slightly better than chance and their errors are independent, the majority is very reliable. But these subagents share a base model, a training corpus, and query craft. They are correlated witnesses, not independent ones. Six of them do not buy six times the reliability — they buy the same blind spot, six times, expressed differently.

2. Verification isn't independent either. The system checking a claim is the same system that produced it. That's a coherence check — does this hang together? — not corroboration, which requires a genuinely separate source. Useful, but weaker than it looks.

3. The web is not a neutral evidence base. Search ranking is itself an epistemic filter, shaped by commercial incentives. What is findable is not the same as what is true, and the gap is systematic rather than random. SEO-saturated domains — compliance, security vendors, "best X for Y" — are exactly where this bites hardest, which is why the banned-source policy is doing far more work than it appears to.

4. Citation prevents fabrication, not laundering. A well-cited report built on weak sources is more dangerous than an uncited one, because it wears the costume of justification. Rigor in the form can disguise rot in the content.

5. The plan gate is the least-examined step and the most consequential. You choose which dimensions count before any evidence exists. Every downstream mechanism assumes the question was well-posed. That's where the deepest bias enters — and nothing in the system checks it. Which is precisely why it's the one place your own judgment is irreplaceable, and why "read every dimension at the plan gate" is the most important practice in §18.

The honest summary: Deep Recon doesn't give you knowledge. It gives you auditable justification — a chain you can inspect and challenge. That's a real and unusual improvement over uncited fluency, and it's less than it can look like when the output is well-formatted and thoroughly cited.

22.6 Why this matters doubly for search and recommendation work

Here's the reflexive turn: search and recommendation systems are themselves epistemic technologies.

A ranking function is an implicit theory of relevance. A theory of relevance is an implicit theory of what is worth knowing. When you build a recommender, you are building a machine that shapes what people can come to know — you're operating on the justification side of other people's beliefs, at scale.

Which means the same concepts run in both directions:

That last row is the sharpest one, and it's worth sitting with. Deep Recon's firewall says: your context shapes the questions, never the answers. A recommendation system that lets a user's history shape what it presents as true — rather than merely what it surfaces — is doing the thing the firewall exists to prevent, to someone else, without their knowledge.

The discipline Deep Recon applies to your own knowing is the same discipline you owe your users about theirs. Provenance, defeasibility, calibrated uncertainty, source independence, visible criteria. In your research process they're hygiene. In your ranking systems they're ethics.

23. Worked walkthroughs for software engineers

Fidelity note. The session excerpts below are illustrative reconstructions. They show the shape of each interaction — the decision points, what you type, where you push back, what lands on disk. Exact prompt wording and gate formatting live in the skill's reference files and will differ in detail. The workflow, artifacts, and decision points reflect the documented design; the TOML key paths should be confirmed against your installed customize.toml.

23.0 Setup, once

npx bmad-method install          # select your IDE; deep-recon ships in core

Confirm it's there, then look at what you can change before your first real run:

/bmad-customize bmad-deep-recon

Two things worth setting on day one, before you've formed habits:

# _bmad/custom/bmad-deep-recon.user.toml   (personal, gitignored)

[workflow]
output_format = "both"      # md for the repo, html for sharing with non-engineers
validation    = "normal"    # raise per-run when the decision is load-bearing

Leave red_team off globally. Turn it on per-run, deliberately, when you notice you already believe the answer.

Walkthrough A — Native Run: choosing an embedding model

Scenario. You own multilingual search and recommendation. Retrieval runs on OpenSearch with HNSW. You need to commit to an embedding model for the recommendation retrieval stage and live with it for roughly a year, because re-embedding the corpus and rebuilding the index is a multi-week project.

This is the gate everyone runs. It's genuinely the least valuable of the gates — but it's the best one to learn the mechanics on, because the feedback loop is short.

A.1 The invocation, and the fork in the road

> /bmad-deep-recon

Deep Recon opens the floor and, on a bare research ask, states the Draft-vs-Run trade once and remembers your answer for the session. For this one, Run is right: you'll want follow-up rounds on contradictions, and some sources are technical enough that you'll want to chase citations interactively.

Now the part that determines everything downstream.

❌ What most engineers type:

research the best embedding models for multilingual search

What comes back: a competent encyclopedia entry. Every dimension is in scope because nothing was pruned. Maximum fan-out, maximum tokens, and a report that ranks models against generic criteria you don't share. You'll skim it once and decide on vibes anyway.

✅ What to type instead:

I need to pick an embedding model for the retrieval stage of our
recommendation system. Decision horizon ~12 months — re-embedding the
corpus is a multi-week project, so this is expensive to reverse.

Constraints:
- Content is multilingual; the mix is roughly [your actual mix]
- Retrieval stage budget is 200ms p99, end to end
- Index memory has a hard ceiling; dimensionality is a real cost, not a
  preference
- Self-hostable strongly preferred; API-only is a significant negative
- Must be usable commercially without a revenue-threshold clause

Include these candidates — but do not treat the list as the frame, and
surface anything stronger: [3-5 named models].

Type: technical, select shape. Effort: deep. Validation: high.
Freshness: benchmark claims older than 12 months are reported as history,
not as current fact.

Four things are doing work there:

A.2 The plan gate — the only place your attention is irreplaceable

An illustrative gate for the above:

DECISION
  Select an embedding model for recommendation retrieval; 12-month horizon,
  high reversal cost.

DIMENSIONS (6)
  1. Multilingual coverage and per-language retrieval quality
  2. Dimensionality vs. index memory and HNSW build/query cost
  3. Published retrieval benchmarks (MTEB/BEIR family) with dataset caveats
  4. Inference latency and throughput; self-host vs. hosted
  5. License terms, commercial-use restrictions, model provenance
  6. Maintenance signals: release cadence, backing, deprecation risk

TOPOLOGY   breadth-first (dimensions are largely independent)
EFFORT     deep — 6 assistants / 12 sources per round / 3 rounds
VALIDATION high; red_team off
FRESHNESS  benchmarks 12mo · license 24mo · maintenance 6mo
ESTIMATE   ~35-45 minutes

Approve, or tell me what to change.

What to actually do here — this is the whole skill:

1. Delete dimensions that don't bear on the decision. Each one costs a full fan-out. If you'll self-host regardless, dimension 4 collapses to "self-host latency only" — say so. Cutting one dimension at deep saves roughly 15% of the run.
2. Add what's missing. Two things are absent above and belong: fine-tuning and adaptation cost (if you'll adapt to your domain, a model that resists adaptation is disqualified regardless of zero-shot numbers), and how each model behaves under the filtered-search patterns your recommender actually uses. Benchmarks measure unfiltered top-k; your production path is filtered.
3. Sanity-check the topology. Breadth-first is right here — these are independent questions. If it had proposed depth-first, that would be a signal the decision was framed as one contested question rather than six separable ones.
4. Check the estimate against your patience. 40 minutes is real. If you'll wander off at minute 12, drop to standard now. Files-first means you keep what landed, but you'll have a partial report you don't trust.

A realistic reply:

Approve with changes:
- Drop dimension 4's hosted-API sub-question; we self-host, full stop.
- Add: fine-tuning/adaptation cost and whether the license permits it
- Add: behavior under filtered ANN search, not just unfiltered top-k
- Keep effort at deep

A.3 During the run

You'll see digests landing and report sections committing per dimension, because everything hits disk on arrival. Two things to watch:

• Aggregator pileup. If several digests trace back to the same summary site, that's one publisher wearing several hats. Say so: "dimension 3 is leaning on aggregators — chase the primary benchmark repos."
• Round-two assignments. Contradictions from round one become round two's work. If two sources disagree on a latency figure and round two isn't chasing it, flag it. Disagreement is the highest-value lead in the run.

A.4 What lands on disk

_bmad/planning/embedding-model-selection-2026-07/
├── briefs/
│   ├── multilingual-coverage.md
│   ├── dimensionality-index-cost.md
│   └── ...
├── digests/
│   ├── d01-mteb-multilingual-subsets.md
│   ├── d02-hnsw-memory-scaling.md
│   └── ...
├── memlog                          # append-only; claim tally lives here
└── research.md                     # the canonical cited report

research.md is the artifact. Everything else is the audit trail — and the reason a run that dies at minute 30 resumes instead of restarting.

A.5 Reading the report — where to look first, in order

1. "Could not determine." Read this before the recommendation. If per-language quality for your two most important languages is in here, the selection matrix is weaker than it looks and you should say so out loud in the ADR.
2. The selection matrix. Check that the weights are your constraints. If it weighted "ease of getting started" heavily, it defaulted to generic criteria and you under-specified at the plan gate.
3. Self-reported vs. independent numbers. Every vendor-run benchmark should be flagged. If they aren't, ask.
4. The staleness map. Turn it into calendar entries the same day. This is the step everyone skips and it's what makes the artifact a living asset rather than a snapshot.
5. The recommendation, last. By now you know how much weight it can bear.

A.6 When the matrix disagrees with you

This happens, and it's the most useful moment in the process. Two legitimate responses:

• Your constraint was wrong or unstated. Fix the weights and say why in the ADR. "We overrode the matrix because index memory is a harder ceiling than the weighting reflected" is a fine sentence.
• Your prior was wrong. Rarer, more valuable.

What's not legitimate is re-running with a different framing until you get the answer you wanted. If you're tempted, that's the moment to turn red_team on instead.

A.7 Handoff

research.md carries metadata frontmatter that downstream skills consume directly:

/bmad-prd

The PRD reads the summary without reprocessing the digests. In your repo, put the report next to the ADR it grounds:

docs/decisions/0014-embedding-model.md          # the decision
docs/decisions/0014-research.md                 # the evidence

Realistic cost: ~40 minutes wall clock, most of it unattended. Compare against the multi-week cost of re-embedding after a wrong pick.

Walkthrough B — Draft → Process: the MT vendor landscape

Scenario. You need translation for the multilingual feature. Six plausible vendors, opaque and structurally different pricing, quality claims that are all self-reported.

This is the textbook Draft case: a broad public sweep over commercial sources, which is exactly what hosted deep-research products are good at, and which you're probably already paying for.

B.1 Draft

> draft a deep research prompt for Gemini: choosing a machine translation
  vendor for [your language pairs], ~[N] million characters/month, EU +
  Canada data residency requirements, glossary/do-not-translate support
  required. Competitive type, select shape.

You get back a prompt carrying the competitive pack's craft — pruned dimensions, freshness bars, source policy, and a hard citation demand, tuned to the tool you named.

Check it before you paste. Two things:

• Does it demand citations with publication dates? If the citation demand is soft, the report will be soft.
• Does it name volume and language pairs? Pricing rankings invert with volume, and "supports 100+ languages" is a marketing claim that says nothing about your specific pairs. If those aren't in the prompt, add them by hand.

B.2 Round-trip, then Process

Paste into your subscription tool, let it crawl, save the output.

> there's a research report at ~/Downloads/mt-vendors.pdf, process it

What Process does, and why each part matters:

B.3 The gap report is the point

Typical output from the gap check on a vendor sweep:

COVERED
  pricing, language coverage, published quality claims, API surface

GAPS (not covered by the imported material)
  - sub-processor lists and data residency commitments
  - training-on-customer-data defaults and opt-out terms
  - glossary / do-not-translate feature parity across vendors
  - exit terms and data portability
  - per-pair quality for [your two lowest-resource pairs]

Note what the gaps are: every single one is a procurement or per-pair specific question. Hosted research tools crawl broad commercial surfaces well and are poor at the narrow, contractual, and specific. That's the division of labour to internalize.

B.4 Targeted Run on the gaps only

> run a focused research pass on the gaps only — sub-processors,
  training-on-customer-data terms, glossary parity, exit terms.
  Type competitive, effort standard, validation high.

Small, cheap, and it lands in the same run folder — the slug is deterministic across the draft→process→refresh lifecycle, so the report is one artifact, not three.

Net cost: one paste, one file drop, one 12-minute run. Your subscription paid for the expensive crawling.

Walkthrough C — The compliance gate, done without fooling yourself

Scenario. Design freeze is in two weeks. You need to know whether the deletion path survives a right-to-erasure request against a vector index.

C.1 The framing trap, shown concretely

❌ Here's our retention and deletion design [paste]. Is it GDPR compliant?

This defeats the firewall by hand. You've supplied the conclusion as context, and you'll get a well-cited report explaining why your design is fine. It will be persuasive. It will be worthless.

✅ What does GDPR Art. 17 require when personal data has been embedded into
   a vector index that does not support hard delete? Include regulator
   guidance on derived data and on bounded-but-delayed deletion.
   Type: domain. Validation: high. red_team on.

Then compare your design against the requirements yourself, afterward. That ordering is the entire discipline.

C.2 Source policy override — do this before running

Regulatory research is the clearest case for overriding the default source policy, because the SEO layer in this space is thick and sales-motivated:

# _bmad/custom/bmad-deep-recon.user.toml

[[workflow.source_policies]]
code = "regulatory"
preferred = [
  "eur-lex.europa.eu", "edpb.europa.eu", "ico.org.uk",
  "priv.gc.ca", "cai.gouv.qc.ca",
]
banned_kinds = ["law-firm marketing", "compliance-vendor content marketing"]

[workflow.freshness.regulatory]
statute     = "36mo"
guidance    = "12mo"
enforcement = "6mo"

Confirm the exact key paths against the shipped customize.toml; the shape and the intent are what matter.

C.3 Two runs, not one

Run 1 (domain) — what the law requires. Erasure scope for derived data, whether embeddings are personal data, Art. 22 threshold for your ranking surface, Quebec Law 25 automated-decision notice, transfer posture for inference calls.

Run 2 (technical) — how to satisfy it in your stack. Real deletion latency under HNSW soft-delete plus segment merge; whether bounded-but-delayed deletion is defensible; consent propagation to precomputed candidate sets and warm caches; embedding inversion as a residual risk.

Merging these produces a report that's shallow on both, because they want different sources and different freshness windows.

C.4 What you hand counsel

Not the whole report. A one-pager built from it:

• Requirements, each traced to a primary source with a date
• Our design's position on each — your comparison, done after the fact
• "Could not determine", verbatim. In compliance work the gaps are the finding
• Three specific questions you need a lawyer to answer

An hour of counsel time instead of a week. And when the auditor asks "how did you conclude this?", the answer is a file with publishers and access dates, not a recollection.

Walkthrough D — Threat-modeling a ranking surface

Scenario. The recommendation feature ships to authenticated users next quarter. Security review is a launch gate.

D.1 The run

> technical research: attack surface of a personalized recommendation
  ranking system serving authenticated users. Cover manipulation of
  rankings, poisoning of the interaction feedback loop, inference attacks
  against user embeddings, cross-user leakage through shared segments,
  and model extraction. Include documented incidents, not just papers.
  Validation: high. red_team on.

red_team on is deliberate. You are walking in believing the system is basically safe — buy the adversarial pass.

"Include documented incidents, not just papers" is the other lever. Academic attack papers describe what's possible; incident writeups describe what's happened, which is a much better guide to what to fix first.

D.2 The transformation that makes it useful

A threat list is not a deliverable. Convert it, in the same session:

Accepted risks get written down with the citation attached. That record is what makes the next security review fast, and it's exactly what's missing when a reviewer asks "did you consider X?" and the honest answer is "probably, in a meeting."

D.3 If there's an LLM in the pipeline

Add indirect prompt injection via item metadata and user-generated content as a first-class dimension, and take the structural lesson from PR #2611 itself: they didn't filter untrusted text out of a shell command, they changed the channel so it never reached one. Briefs went to a file; the template substitutes a skill-generated path. Filters on untrusted text fail eventually. Removing the interpreter from the path doesn't.

Walkthrough E — Refresh, three months later

> refresh the embedding model research

Only stale claims get re-verified, per the staleness map. You get a delta:

CONFIRMED (11)   license terms, dimensionality/memory figures, ...

CHANGED (3)
  [4] Model B latency benchmark — improved ~20% after a runtime release
  [7] Model C pricing — restructured to committed-use tiers
  [9] Model A release cadence — slowed; last release 5 months ago

OVERTURNED (1)
  [12] Model D commercial-use terms changed; the revenue threshold that
       made it viable for us no longer applies as documented

⚠  Claim [12] is referenced by:
       docs/decisions/0014-embedding-model.md

That warning is the whole reason the lifecycle exists. A licensing assumption underpinning a shipped decision moved, and you found out from a 6-minute refresh rather than from procurement, or a lawyer, or a blog post someone forwards you.

What you do: re-open the ADR, note the change, decide whether it changes anything. Often it doesn't. The value is that the question got asked at all.

Walkthrough F — Deepen one dimension

You committed to a model. Now you need the fine-tuning story properly, and only that.

> deepen the fine-tuning and adaptation dimension

Drills one dimension without re-running the other five. Minutes, not tens of minutes. This is the command that makes research iterative — you stop trying to front-load every question into one enormous engagement and instead pull threads as they become load-bearing.

Rule of thumb: deepen when the question was in scope but under-answered. A fresh run when the question is genuinely new.

Walkthrough G — Headless refresh in CI

The headless JSON contract means the staleness map can run itself. A quarterly job, roughly:

# .github/workflows/research-refresh.yml
on:
  schedule: [{ cron: "0 6 1 */3 *" }]      # first of the month, quarterly
jobs:
  refresh:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Refresh decision research
        run: |
          bmad-deep-recon refresh \
            --headless --output-format md \
            --workspace docs/decisions/research/
      - name: Open an issue if anything was overturned
        run: ./scripts/alert-on-overturned.sh

Headless runs present canonical markdown only — output_format = auto handles that without configuration. Alert on overturned and on changed that touches a referenced claim; ignore confirmed. Otherwise you've built a quarterly notification everyone mutes by the second quarter.

This is the cheapest high-value automation in the whole system: decision rot, detected without anyone remembering to look.

Walkthrough H — The anti-walkthrough

Worth seeing the failure mode concretely, because it's what a first run usually looks like.

> research recommendation system best practices

What the plan gate proposes: eleven dimensions, breadth-first, deep, ~55 minutes. Nothing was pruned, because no decision was stated and therefore nothing could be pruned.

What you get: a well-cited, genuinely accurate survey of recommender systems. Collaborative filtering, content-based, hybrids, cold start, evaluation, MLOps. All true. All findable in any textbook. None of it weighted to a choice you face.

Cost: ~55 minutes and a large token spend, for something you skim once.

What went wrong, precisely: the plan gate is a pruning device. It prunes dimensions against the decision. No decision means no pruning function, so the run is unbounded by construction. The mechanism didn't fail — it was handed nothing to work with.

The fix is one sentence, always the same shape: "I am choosing between X, Y and Z, under constraints A, B and C, and I'll live with it for N months."

If you cannot write that sentence, you're not ready to research. You're ready to brainstorm — which is a different skill, and the right one for that moment.

23.9 Team integration patterns

Keep the research in the repo, next to the decision.

docs/decisions/
├── 0014-embedding-model.md          # ADR: what we chose and why
├── 0014-research.md                 # the evidence, with provenance
├── 0015-mt-vendor.md
└── 0015-research.md

The ADR states the decision and the reasoning. The research file carries the citations, dates, and staleness map. Six months later the ADR tells you what, and the research file tells you whether it still holds.

Review research in PRs. A research report in a pull request gets the same treatment as code: someone checks whether the plan-gate dimensions were right, whether the weights match the team's actual constraints, and whether the "could not determine" section was read rather than skipped. This catches motivated framing better than anything else, because the person reviewing didn't want the answer.

Onboarding. A new engineer reading 0014-research.md learns why the stack looks the way it does, from sources, in twenty minutes. Compare with the usual mechanism: asking whoever's been there longest and getting a partly-reconstructed memory.

Don't let it become theatre. Research that's produced, filed, and never refreshed or contradicted is a compliance ritual. The signal that it's real: someone, at some point, overrode a recommendation and wrote down why.

23.10 Calibration — what to spend where

Two patterns in that table worth naming:

• red_team is on exactly where you have a prior. Nobody walks into a privacy or security review neutral — you walk in believing you're fine. That's the condition the adversarial pass exists for.
• The cheapest runs are the most consistently underused. A six-minute dependency-health check has probably prevented more grief than any model comparison ever run.

24. Sources

• BMAD-METHOD PR #2611 — "feat(core): consolidate research trio into bmad-deep-recon", merged 23 Jul 2026 — https://github.com/bmad-code-org/BMAD-METHOD/pull/2611
• docs/explanation/deep-recon.md — https://github.com/bmad-code-org/BMAD-METHOD/blob/main/docs/explanation/deep-recon.md
• docs/how-to/customize-bmad.md (config layering and merge rules) — https://github.com/bmad-code-org/BMAD-METHOD/blob/main/docs/how-to/customize-bmad.md

Applied guidance in §12–§17, the epistemological analysis in §22, and the walkthroughs in §23 are my analysis, not sourced from the BMAD docs. The named philosophical concepts (Gettier, Popper, Reichenbach, Condorcet) are standard terms of art in epistemology and philosophy of science.

Source: bmad-rcon-foundation · bmad-rcon-foundation.md · updated 2026-07-25 · 🔒 secret gist

Synced verbatim from gist.github.com/bl9.

Part 0 — Foundations

Read this before bmad-deep-recon-guide.md. It assumes no background in agentic tooling. Every term is defined, traced to where its name came from, and shown in use.

Why the etymologies? Almost every word in this system is borrowed from somewhere else — the military, graph theory, newspaper typesetting, caching, law. That's not decoration. When you know that "topology" comes from network diagrams and "breadth-first" is a graph-traversal algorithm, you stop memorizing vocabulary and start predicting what the system will do, because you already understand the source concept. Naming is compressed explanation.

Contents

• 1. The five-minute model, before any vocabulary
• 2. Layer 1 — the AI substrate
• 3. Layer 2 — how BMAD is built
• 4. Layer 3 — Deep Recon's own vocabulary
• 5. Six mental models that carry the most weight
• 6. Where the metaphors break
• 7. Learning path: your first two weeks
• 8. Resources and references
• 9. What I'm confident about, and what I'm not

1. The five-minute model, before any vocabulary

Strip away every term and this is the whole system:

You have a decision to make. You don't trust the AI's memory, because it's stale and can't show its work. So you make it go look things up, in parallel, from sources it has to name — while preventing your own assumptions from leaking into what it finds. Everything it learns gets written to a file with dates attached, so that in six months you can tell whether it's still true.

Four moving parts:

1. A decision — stated up front, with constraints. Everything else is shaped by it.
2. A plan you approve — what will be investigated, how, and for how long.
3. Parallel lookup with a barrier — several sub-processes search, none of them sees your assumptions.
4. A dated, cited file — the durable output, with an expiry warning built in.

If you hold only that, the rest of the vocabulary is just precise names for these four things.

2. Layer 1 — the AI substrate

You need these before BMAD makes sense. Skip any you already know.

2.1 The basics

2.2 Agents and tools

Why subagents exist at all — this is worth understanding, because it explains the whole architecture. The context window is finite. If one model does twelve searches, it accumulates twelve pages of raw text and chokes. Instead: spawn six subagents, each with its own fresh window, each doing two searches and returning a half-page summary. The parent receives six half-pages instead of twelve full pages.

The cost: each subagent only knows what you told it. Coordination becomes the hard problem. Nearly every complexity in Deep Recon — briefs, digests, the single-writer rule — is machinery for managing that trade.

3. Layer 2 — how BMAD is built

3.1 The naming collision, flagged early

When BMAD docs say "the Analyst agent," they mean a persona file, not an autonomous process. Both senses appear in the same repo. Read from context.

3.2 Structural vocabulary

Why "append-only" matters — if you can rewrite history, the record can't be trusted, because you can't distinguish "this was always true" from "someone changed it." Append-only logs are the backbone of databases, version control, and blockchains for exactly this reason. Here it means: a claim's status can be added to (unverified → verified → overturned), and the earlier states remain visible.

4. Layer 3 — Deep Recon's own vocabulary

4.1 "Recon"

Military reconnaissance: scouting ahead of the main force to learn the terrain and the enemy's position before committing troops. From French reconnaître, "to recognize" — literally to know a place again, by having been there.

Why it's the right metaphor:

• It precedes commitment. You scout before you move, not during.
• It's cheap relative to what it protects. A patrol costs far less than a failed assault.
• It's information-gathering, not action. Recon doesn't take the hill. It tells you whether the hill is takeable.
• "Deep" recon is a real military term: patrolling far beyond the forward line, at higher risk, for strategic rather than tactical information.

Where the metaphor is useful to you: it correctly implies that research is preparatory and bounded. Research that never terminates in a decision has failed at being recon.

4.2 "Run" — the term you asked about

What it is: one complete, bounded execution of a research engagement. It has a beginning, an end, a configuration, an identifier, a folder on disk, and outputs. When you say "the run," you mean that whole bounded episode and everything it produced.

Where the name comes from — three overlapping ancestors:

1. Manufacturing. A production run: one batch produced under one set of settings. Change the settings, it's a different run. This gives you the sense of bounded, configured, and reproducible-ish.
2. Early computing. In batch-processing days, you submitted a job and the operator ran it. One run = one pass of one job through the machine, producing one printout. This gives you one execution, one output.
3. Machine learning experiment tracking — almost certainly the immediate ancestor. In tools like MLflow and Weights & Biases, a run is one execution of an experiment: it gets a run ID, a config, logged metrics, and an artifact directory. That's exactly Deep Recon's structure — a slug, a config, a memlog, an artifact folder.

Why the word does real work here. Compare the alternatives:

• "Session" would imply it's tied to your conversation. It isn't — it survives on disk.
• "Query" would imply one question, one answer. It's dozens.
• "Report" names only the output, not the process.
• "Run" names the bounded episode plus its record. That's the concept you need, because you refresh a run, deepen a run, resume a run.

How you use it — the three senses, which trip up beginners:

Practical consequences of a run being a bounded, identified thing:

• It has an identity. The folder slug is deterministic, so a draft, its later processing, and a refresh three months on all land in the same run folder. One decision, one run, one artifact — not three loose files.
• It's resumable. Because everything hits disk as it arrives, a run that dies at minute 30 picks up from disk rather than starting over.
• It's refreshable. refresh re-executes only the stale parts of an existing run. You can only do that to something with an identity.
• It's configurable as a unit. Effort, validation, and freshness are properties of the run, set once at the plan gate.

4.3 "Mode" — Draft, Process, Run

Mode = a way of operating; from Latin modus, "measure, manner." Same sense as a camera's portrait/night mode: the same device, different operating logic.

The three are named for what you do, and they differ in where the searching happens:

4.4 "Type" and "pack"

Type = the category of research: market, domain, technical, competitive, user-voice, academic-lit.

Pack = the ~25-line configuration card that a type selects. From the gaming/software sense of a pack: a bundle of assets shipped as a unit (texture pack, expansion pack).

A pack contains: prioritized dimensions, source craft, freshness rules per claim class. This is the "data-driven variation" that let three near-identical workflows collapse into one skill — the procedure is shared, only the card differs.

Decision shape is a second, independent axis: explore (build understanding) or select (choose between options, using a weighted matrix). Shape because it describes the form the conclusion takes. Any type can combine with either.

4.5 "Dimension" — the second term you asked about

What it is: one independent axis of inquiry within your research question. Your question gets decomposed into dimensions; each one is investigated separately, often in parallel; the results are recombined.

Where the name comes from: geometry and measurement. A dimension is an independent direction of variation — you can move along one without moving along the others. Length, width, height. Data analysis borrowed it (a "dimension" in a data cube: time, region, product), and the sense carried is the same: a separable axis.

Why the word is well chosen — it encodes three properties at once:

1. Independence. Dimensions should be as orthogonal as possible, because that's what makes parallel investigation valid. If two dimensions overlap heavily, you're paying twice for one answer.
2. Coverage. Together they should span the decision. In geometry, dimensions that don't span the space leave regions you can't reach; a dimension set that doesn't span the decision leaves blind spots. That's what the "could not determine" section reports.
3. Separability. Each can be handed to a different subagent with a self-contained brief.

How you use it — this is the highest-leverage skill in the whole system. At the plan gate you're shown a proposed dimension list. Your job:

Concrete example from the guide: for an embedding-model choice, the proposed dimensions covered benchmarks, cost, license and maintenance — but omitted behavior under filtered ANN search. Benchmarks measure unfiltered top-k retrieval; your recommender queries with filters. That missing dimension is the difference between a report that answers your question and one that answers a similar-looking question.

4.6 "Topology" — and where breadth-first/depth-first come from

What it is: the shape of how subagents are arranged against the dimensions — how work is split and connected.

Where the name comes from: mathematics. Topology studies properties of shape that survive stretching and bending — what's connected to what, regardless of distance. Network topology (star, ring, bus, mesh) is the borrowing you've probably met: the diagram of what connects to what, ignoring cable length.

Here it means the same: the connection pattern of the work graph. Not how long each part takes — the shape.

The three topologies:

Where "breadth-first" and "depth-first" actually come from — and this is the good part. They are graph-traversal algorithms from computer science.

Imagine exploring a maze:

• Depth-first search (DFS): pick a corridor, follow it as far as it goes, hit a dead end, back up, try the next. You go deep before you go wide. Traces back to Trémaux's 19th-century maze algorithm; formalized in mid-20th-century CS.
• Breadth-first search (BFS): step one pace down every corridor, then two paces down every corridor, and so on. You go wide before you go deep. Developed in the 1950s (Moore, for maze routing; Lee, for circuit routing).

The classic property: BFS finds the shortest path and explores evenly; DFS goes further faster on one path but can get lost down a long wrong corridor.

The metaphor transfers exactly:

• Breadth-first research: cover all six dimensions to a moderate depth, evenly.
• Depth-first research: drive hard down one question, following where it leads, accepting you'll learn less about everything else.

Knowing the algorithms means you can predict the failure modes. BFS's weakness is memory — it holds the whole frontier at once, which is exactly why breadth-first research burns more tokens per round. DFS's weakness is going deep down a corridor that doesn't matter — which is exactly what happens when you depth-first a question that turns out to be a side issue.

Fan-out — a related term. From digital electronics: the number of gate inputs a single output can drive before the signal degrades. Distributed systems borrowed it for one request spawning many downstream calls. Here: one plan spawning N assistants. The electronics sense carries a useful warning — fan-out has a limit past which quality degrades. More parallelism is not free, and it is not automatically better.

4.7 "Plan gate"

What it is: the single mandatory stop where the system shows you what it intends to do, and waits.

Where the name comes from: gate in the sense of a controlled passage — nothing proceeds until it opens. Two direct ancestors:

1. Stage-Gate in product development (formalized by Robert G. Cooper in the 1980s): projects move through stages, and between them sit gates where a decision-maker says go, kill, hold, or recycle. The point of a gate is that it's the only place a decision is made, so the decision gets real attention.
2. Quality gates in CI/CD: the build doesn't advance unless it passes.

Why there is exactly one. Constant interrogation trains you to click through. One well-placed gate that actually matters gets read. After it, the run proceeds with light checkpoints. This is a deliberate attention-budget decision, and it means your attention at the gate is the single highest-value input you provide to the entire system.

What it shows: the decision, the dimensions, the topology, the knobs, and a time estimate. What you do: prune, add, adjust, check that the estimate matches your actual patience.

4.8 Rounds, leads, budgets, presets

Precedence — worth memorizing: your request beats a pinned setting beats the preset. Whatever you say in the moment wins.

4.9 Briefs, digests, claims

Why briefs are written to files and not passed as text on a command line — the security fix in PR #2611, and worth understanding as a beginner because it generalizes. If untrusted text is pasted into a shell command, characters like $(...) or backticks are interpreted as commands rather than read as text. Researched content is untrusted by definition — you fetched it from the internet. The fix wasn't to filter dangerous characters (filters always leak); it was to change the channel so the text never reaches an interpreter. It goes to a file; the command references the file's path.

That pattern — remove the interpreter from the path rather than sanitizing the input — is one of the most transferable ideas in the whole PR.

4.10 "Firewall"

What it is: the rule that your project context may shape which questions get asked, but never what counts as evidence. Subagents receive only their brief.

Where the name comes from: building construction. A firewall is a physical wall built to a fire-resistance standard, dividing a structure into compartments so a fire in one can't spread to the next. Cars borrowed it (the barrier between engine and cabin), then networking borrowed it in the late 1980s.

The property that carries through every sense: a barrier at a boundary that stops one specific thing from propagating, by design and by construction rather than by vigilance.

Why it matters more than it sounds. The failure it prevents is subtle: hand an AI your architecture document and ask it to research alternatives, and it comes back explaining that your architecture is excellent. Not through deceit — through ordinary confirmation bias. The thing generating hypotheses is also grading them.

Most attempts to fix this are instructions — "be objective," "consider alternatives." Instructions fail, because a system inclined to agree with you will find a way. The firewall is architectural: the subagent cannot be biased by context it was never given. That's the difference between a policy and a wall.

The closest analogy from research methodology is the double-blind trial: the experimenter doesn't know which subject got the drug, so their expectations cannot contaminate the measurement. Not "must not" — cannot. Same idea, same reason it works.

4.11 Staleness, freshness, refresh, deepen

The distinction that matters: refresh when time has passed. deepen when a dimension was under-answered. A brand-new question is neither — that's a fresh run.

4.12 Provenance, citations, verification

Why provenance is the right word, not "sources." A source list says where things came from. Provenance adds when it was published, when you accessed it, and who published it — which is what lets someone else evaluate a claim they didn't verify themselves. That's the difference between a bibliography and an audit trail.

4.13 The remaining machinery

5. Six mental models that carry the most weight

If you remember nothing else:

1. Recon is scouting before you commit troops. Cheap relative to what it protects, and it must terminate in a decision or it wasn't recon.

2. The run folder is a lab notebook. Dated, append-only, showing failed attempts alongside successes. The value isn't the conclusion; it's that someone else can check your work — including future you.

3. The firewall is a double-blind trial. The researcher can't be influenced by expectations they were never given. Architectural, not aspirational.

4. Dimensions are axes that must span the space. Independent enough to parallelize, complete enough to cover the decision. Gaps in the span are exactly what "could not determine" reports.

5. Staleness is cache TTL for beliefs. Still readable, no longer trustworthy. Different claim classes have different TTLs, and that's a feature.

6. The plan gate is a stage gate. There is exactly one, deliberately, so that it gets read. Your attention there is the highest-value input you give the system.

6. Where the metaphors break

Understanding a borrowed term includes knowing where the borrowing stops. Four that mislead beginners:

That last one is the most important caveat in this entire document. Deep Recon does not give you knowledge. It gives you auditable justification — a chain you can inspect and challenge. That's a real improvement over uncited fluency, and it is less than a well-formatted, thoroughly-cited report can look like.

7. Learning path: your first two weeks

Week 1 — mechanics

Day 1 — Install and read the shape.

npx bmad-method install

Then /bmad-customize bmad-deep-recon and just read the options. Don't change anything. You're building a map of what's adjustable.

Day 2 — The calibration run. This is the single most useful beginner exercise, and almost nobody does it:

Run a quick engagement on a question you already know the answer to cold — something in your own stack you could write a memo about from memory.

Because you know the content, you're free to evaluate the machinery. Where did it go for sources? Did it find the thing you know is the definitive reference? Where was it confidently thin? Where did it flag uncertainty you'd have glossed over? Twenty minutes here teaches more about calibration than five real runs.

Day 3 — Deliberately break it. Run the anti-pattern: research recommendation system best practices. Watch the plan gate propose eleven dimensions and fifty-five minutes. Kill it at the gate. You'll have learned viscerally why a decision statement is load-bearing, for the price of two minutes.

Day 4 — A real cheap run. Pick a genuine but low-stakes decision — a library choice, a dependency health check. quick, standard validation. Take the output to a colleague and defend it.

Day 5 — Read the artifacts. Open the run folder. Read a brief, a digest, the memlog, then research.md. Trace one claim from the report back through its digest to its source. Once you've done this once, you'll never over-trust a report again.

Week 2 — judgment

Day 6–7 — Draft → Process. Draft a prompt, run it in whatever deep-research tool you have, process it back. The gap report is the lesson: see for yourself what hosted tools systematically miss.

Day 8 — Configure. Write your first user.toml. Set a source policy for one domain you know well — you can tell good sources from bad in your own field, which is exactly what you need to write a policy.

Day 9 — A load-bearing run, with red_team on. Notice how the adversarial pass changes the tone.

Day 10 — Refresh something. Even a two-week-old run. Watch the delta mechanism work.

The habit that makes it stick

Every time you're about to make a decision you'll live with for months, write this sentence first:

"I am choosing between ___, ___ and ___, under constraints ___, ___ and ___, and I'll live with it for ___ months."

If you can write it, you're ready to research. If you can't, you're not ready to research — you're ready to brainstorm, which is a different skill and the right one for that moment.

8. Resources and references

Verify before citing. These are from my own knowledge, not fetched and checked in this session. Titles and authors I'm confident about; URLs and current versions you should confirm.

8.1 BMAD itself — start here

8.2 Agentic systems generally

8.3 The computer science behind the terms

8.4 Epistemology — for §22 of the main guide

All free online:

8.5 Research methodology — best practices worth stealing

8.6 Domain references for the applied sections

Retrieval and recommendation

• MTEB (Muennighoff et al.) and BEIR (Thakur et al.) — the benchmark suites, and more importantly their stated limitations
• Malkov & Yashunin, HNSW (2016/2018) — the original paper for the index you're running
• Shilling / profile-injection attack literature in the recsys venues (RecSys, SIGIR)

Security and adversarial ML

• OWASP Top 10 for LLM Applications — the standard reference for prompt injection and agentic risk
• MITRE ATLAS — adversarial threat landscape for AI systems, structured like ATT&CK
• NIST AI Risk Management Framework
• Shokri et al. (2017) on membership inference; Morris et al. on text-embedding inversion

Privacy and regulation — always primary sources, never summaries

• GDPR text on EUR-Lex; EDPB guidelines and opinions
• ICO (UK) — unusually clear practical guidance, useful even outside UK scope
• OPC Canada (priv.gc.ca) and CAI Québec (cai.gouv.qc.ca) for PIPEDA and Law 25
• The EU AI Act and DSA texts on EUR-Lex

Decision records

• Michael Nygard, "Documenting Architecture Decisions" (2011) — the origin of the ADR format, and the natural home for research output. Short and practical

9. What I'm confident about, and what I'm not

Worth stating plainly, since this document makes a lot of etymological claims.

Well-documented and standard:

• BFS/DFS as graph-traversal algorithms; the maze intuition
• Fan-out from digital electronics
• Firewall from building construction → automotive → networking
• Red team from Cold War military war-gaming
• Slug from newspaper typesetting
• Frontmatter from book publishing
• Provenance from art history and archaeology
• Stage-Gate from Robert G. Cooper's product-development work
• Shim from mechanical engineering
• Digest from Latin digerere and Justinian's Digest
• Reconnaissance from French reconnaître
• TOML named for Tom Preston-Werner
• The philosophical works, authors and dates in §8.4

My inference, reasonable but not documented as BMAD's stated rationale:

• That "run" specifically inherits from ML experiment tracking (MLflow, W&B) rather than the general computing sense. The structural match — run ID, config, artifact directory — is strong enough that I'd bet on it, but the docs don't say so.
• That "dimension" carries the spanning/orthogonality intuition deliberately rather than just meaning "aspect."
• The specific mapping of each mechanism to a named epistemological position in §22 of the main guide. The PR does head a section "Epistemics and reliability," so the intent is explicit; the particular philosophical labels are mine.

Things I have not verified in this session:

• Every URL in §8. I'm confident about titles and authors; check the links.
• Exact TOML key paths — confirm against your installed customize.toml.
• Exact prompt and plan-gate wording, which lives in the skill's reference files.

Where this document and the actual skill disagree, the skill wins. Metaphors are for building intuition, not for settling facts.

Next: bmad-deep-recon-howto.md — every concept here shown as an executable session. Then bmad-deep-recon-guide.md — the full applied guide.

Source: bmad_Research · bmad_research.md · updated 2026-07-23 · 🔒 secret gist

Synced verbatim from gist.github.com/bl9.

BMAD Research Capabilities — The Complete Reference

Scope & recency. BMAD-METHOD V6 (BMM module), current as of July 2026. The V6 line shipped steadily through spring 2026 (v6.3 → v6.8.x by late May). BMAD moves fast; command names and folder layout shift between point releases, so treat exact command strings as version-dependent and confirm against your install with bmad-help. Where a detail could not be verified against current published docs, it is flagged inline rather than guessed.

Table of Contents

1. At a Glance
2. Where Research Sits in the Lifecycle
3. Choosing the Right Research Type
4. The Three Research Workflows in Detail
5. The Sub-Agent Architecture
6. Research Depth and Scale-Adaptive Intelligence
7. End-to-End Walkthrough
8. Scenario Library
9. Multiple and Parallel Agents (Party Mode)
10. Where the Facts Come From (Provenance and Retrieval)
11. Cost, Context, and Running Off-IDE
12. Brownfield Research and Established Projects
13. Worked Example: A User Recommendation System
14. Markov Decision Processes and Stochastic Dependency Graphs
15. Observations and Findings
16. Practical Tips and Best Practices
17. Quick Reference
18. Glossary
19. Appendix A: Copy-Paste Research Prompts
20. Appendix B: Custom Party Definitions
21. Sources and Recency

1. At a Glance

Research in BMAD is Phase 1 (Analysis) work. It is optional — the required backbone is Planning → Solutioning → Implementation — but it exists to stop you building architecture on unvalidated assumptions. There are three distinct research workflows, all driven by the Analyst agent (persona "Mary"):

Each stands alone — run one, two, or all three. Output is a structured findings document that feeds the PRD, product brief, or architecture workflows.

Command-naming caveat. Current official docs use the short forms above. Some builds and the internal registry use a module-qualified form (bmad-bmm-market-research, etc.). Both refer to the same workflow under bmad/bmm/workflows/1-analysis/research/. If one form isn't recognized, try the other or ask bmad-help.

2. Where Research Sits in the Lifecycle

Phase 1: ANALYSIS (optional)        Phase 2: PLANNING (required)
─────────────────────────           ───────────────────────────
• brainstorming                      • PRD  (bmad-create-prd)
• RESEARCH  ← you are here           • UX design (optional)
    - market
    - domain          ─── findings feed ──▶  the PRD workflow ingests
    - technical                              briefs, PRFAQ, research
• product-brief                              findings, and brainstorm
• prfaq                                       reports as input

The key design idea is that artifacts flow forward. Research findings are not a throwaway chat — they land as documents in your output folder, and the PRD workflow explicitly accepts them as input, synthesizing whatever Phase 1 produced into structured requirements. Running research before planning compounds in value rather than being busywork.

BMM follows a waterfall-with-gates model: each phase produces documents that become context for the next, and mandatory checkpoints (gates) sit before implementation to keep requirements, architecture, and work breakdown aligned.

Default output locations (from config.yaml, set at install time):

• output_folder → _bmad-output (base for everything)
• planning_artifacts → {output_folder}/planning-artifacts
• project_knowledge → docs (long-lived research / reference docs live here)

So durable research can be steered into docs/, while cycle-specific research lands under planning-artifacts/.

3. Choosing the Right Research Type

The three types answer genuinely different questions. Conflating them is the most common mistake — "research my recommender" is really three separate investigations.

                    What are you actually unsure about?
                                  │
        ┌─────────────────────────┼─────────────────────────┐
        ▼                         ▼                          ▼
  "Does anyone want       "I don't understand        "Can we build it,
   this / who else         this problem space          and how?"
   is doing it?"           / its rules?"
        │                         │                          │
        ▼                         ▼                          ▼
  bmad-market-research     bmad-domain-research      bmad-technical-research
   (MR) — external,         (DR) — vocabulary,        (TR) — feasibility,
   competitive, sizing      constraints, SME depth    architecture, libraries

Rules of thumb:

• Internal feature for a known audience? Skip market research. Run technical, maybe a light domain pass.
• Entering an unfamiliar regulated domain? Domain research first — it changes how you'll judge everything downstream.
• Choosing between concrete technical approaches? Technical research, one focused pass per decision (see §13).
• Brownfield / existing codebase? Your real entry point is often bmad-document-project, not greenfield research (see §12).
• Not sure? Run bmad-help and describe the situation; it recommends a starting point based on what you've already produced.

4. The Three Research Workflows in Detail

Market Research (bmad-market-research, MR). Examines competitors, trends, and user sentiment; produces TAM/SAM/SOM sizing, competitive positioning, and persona/journey notes. This is the most mature of the three because it fans out to five specialized sub-agents (§5). Use it to validate that a concept is worth resources before committing.

Domain Research (bmad-domain-research, DR). Builds subject-matter expertise: the terminology practitioners use, industry-specific constraints, regulatory/ethical boundaries, and how "good" is judged in this space. Use it when entering an unfamiliar domain — getting this wrong produces a technically clean system that solves the wrong problem.

Technical Research (bmad-technical-research, TR). Evaluates feasibility, architecture options, and implementation approaches. This is the workhorse for engineering decisions. Note it is currently single-agent (§5) — its output quality depends heavily on how sharply you scope the prompt and what constraints you feed it.

All three are optional Phase-1 workflows run by the Analyst, and all three write findings documents that the PRD workflow can ingest. Run any subset; each stands alone.

Adjacent capability worth knowing — bmad-investigate. V6.8 added a forensic-investigation workflow in the implementation loop that produces evidence-graded findings for debugging and complex system analysis. It isn't one of the three Phase-1 research workflows, but for "why is production behaving like this" questions it's the research-shaped tool you actually want, and it's more grounding-disciplined than a generic research pass.

5. The Sub-Agent Architecture

The research workflow isn't a single monolithic prompt. Market research dispatches to five specialized sub-agents that can work in parallel:

The known gap

As of the V6 line, technical research has NO dedicated sub-agents (tracked upstream as a feature request). When you run technical/architecture research, the main Analyst agent executes queries sequentially by itself. Consequences:

• Slower — no parallel fan-out the way market research gets.
• Shallower specialization — no dedicated "architecture evaluator" or "library comparator."
• You carry more of the load — quality depends on how sharply you scope the prompt and how much source material / constraints you supply up front.

The single most useful thing to internalize before relying on bmad-technical-research for anything load-bearing: it is a well-structured single-agent research pass, not a multi-agent swarm. Treat it accordingly, and use Party Mode (§9) when you need genuine multi-perspective technical research.

6. Research Depth and Scale-Adaptive Intelligence

Step 5 of the workflow asks you to set research depth — a knob controlling how much source-gathering and cross-checking happens versus a fast synthesis pass. Deeper settings mean more web search, more sources, longer runs, and higher token cost.

Honest, still-open gap. The exact depth option values (e.g. quick/standard/deep vs. numeric levels) are defined in the workflow's step files, not the public docs, and could not be verified at time of writing. Don't hard-code a value from memory; when the workflow prompts, read the options it offers, or ask bmad-help what your installed version exposes.

Scale-Adaptive Intelligence is the related, documented system: bmad-help analyzes project-complexity indicators (team size, architectural complexity, compliance requirements) and recommends appropriate phase entry points and planning depth. It's how BMAD scales from a five-minute bug fix (Quick Flow) to enterprise-platform planning without forcing heavy process on small work. The three planning tracks — Quick Flow, BMad Method, Enterprise — are the concrete expression of this and each defines which workflows are required, optional, or conditional.

7. End-to-End Walkthrough

The workflow follows five interactive steps. Here's a full run for a technical decision.

Step 0 — Fresh chat. Always start research in a new chat. Stale context from an unrelated workflow degrades quality and costs more tokens.

Step 1 — Load the Analyst.

bmad-agent-analyst

(You can also invoke the workflow skill directly and let the tool route to the Analyst; loading the agent first gives you the conversational menu with the MR/DR/TR triggers.)

Step 2 — Run the workflow.

bmad-technical-research

Step 3 — Choose research type. Since you invoked the technical variant directly, this is already TR.

Step 4 — Provide context. This is where the run is won or lost (especially given no technical sub-agents). Give a tight problem statement, the constraints, and what a good answer looks like. See Appendix A for ready-to-use prompt shapes.

Step 5 — Set research depth. Pick depth by stakes. A "which library" spike can be shallow; an architecture bet that shapes service topology warrants the deepest setting.

What comes out. A structured findings document in your output folder — typically: options considered, comparison across your constraints, tradeoffs, a recommendation, and (for technical research) a sketch of a validation/PoC path. Because it's a file, it becomes an input you hand straight to bmad-create-architecture later.

What happens next automatically. The workflow ends by invoking bmad-help, which inspects project state and suggests the next step (usually product-brief or PRD). If you treat BMAD as a personal tool, ignore the suggestion and keep the findings doc.

8. Scenario Library

Scenario A — Technical research (feasibility spike)

Situation: Deciding whether pre-execution validation of Athena queries should be a hand-rolled parser, a sqlglot-based validator, or Athena's own dry-run / EXPLAIN path. Expected findings: a tradeoff matrix (safety coverage vs. cost vs. language fit), a recommended defense-in-depth layering, and which failure modes each layer catches. Observation: with no technical sub-agent, expect to iterate — feed back the option you're leaning toward and ask it to pressure-test that specifically.

Scenario B — Domain research (unfamiliar space)

Situation: Scoping a recommendation feature inside a legal-tech product; you want the domain landscape before writing a PRD. Expected findings: a subject-matter primer — vocabulary, the constraints that make the domain different (authority, jurisdiction, recency of law), and the evaluation lens you'd otherwise miss. Why domain, not technical: you're building framing, not choosing a library.

Scenario C — Market research (validation before build)

Situation: Deciding whether an internal demo is worth productizing. Expected findings: this is where the five sub-agents earn their keep — a competitor map, sizing, trend forecast, and persona notes assembled in parallel, then synthesized.

Full prompt text for each is in Appendix A.

9. Multiple and Parallel Agents (Party Mode)

There are two distinct forms of "multi-agent" in BMAD, and conflating them is the most common mistake:

1. Built-in sub-agent fan-out — automatic, inside a single workflow. Market research already does this (§5). You don't control it directly.
2. Party Mode — you convene multiple agent personas in one room to attack a question from different angles. This is the deliberate, steerable tool.

Reality check on "parallel." BMAD agents are registered as Skills, which run synchronously in the main chat thread. Even Party Mode is a simulated round-table, not OS-level concurrency — there's an open request to wrap agents as native IDE subagents for true async execution, but that isn't the default. Read "parallel" as "independent reasoning per persona," not "N agents at the same wall-clock instant." The exception is agent-team mode (Claude Code only), which stands personas up as a persistent team.

9.1 The four Party Mode modes (who does the thinking)

Invoke with bmad-party-mode; pick a mode with --mode:

Fallback chain: agent-team → subagent → session when a harness can't do the rest. On a non-Claude-Code tool, asking for agent-team quietly degrades.

The core principle: one model voicing five personas will quietly converge — they share a mind, so they tend to agree. Spawning real agents (subagent/agent-team) keeps reasoning genuinely separate, which is the entire point of a review panel or focus group. If you use multi-agent for diverse input and run it in session, you paid for theater.

9.2 ⚠️ Critical best practice: grounding vs. fabrication

A known V6.3 regression worth internalizing before trusting subagent mode on anything codebase-grounded: the default subagent spawn prompt ended with an instruction telling each persona not to use tools — "just respond with your perspective." On a codebase-grounded question, spawned agents returned zero tool calls and fabricated a meaningful fraction of their claims (roughly a third in one reported run on a ~1000-line file), because they reasoned from the orchestrator's lossy summary instead of reading the source. In the same tests, --solo/session and tool-enabled variants stayed grounded.

Rules that follow:

• For grounded work (real code, real data), verify the spawned personas can actually read the sources. If the spawn prompt forbids tools, the panel is opining on a summary, not the artifact.
• Party Mode shines for judgment questions, not fact-retrieval. "Which reranking approach fits our constraints?" is perfect. "What does line 340 do?" is exactly where the summarization gap bites.
• When in doubt, feed the source into the room yourself (paste the diff, metrics, findings docs) rather than trusting each subagent to fetch it.
• Confirm on your version — this specific prompt has been iterated upstream, so newer builds may already ground subagents.

9.3 The shipped review panels

BMAD ships two custom parties as working templates (defined but inactive until summoned — they cost nothing sitting in the pool):

Code Review Crew — five lenses that attack a change and argue about what matters: Vex (security/exploit path), Grumbal (adversary, assumes it's broken), Boundary (edge cases, nulls, races, timezones), Yui (craft, simplicity, naming), Dana (pragmatist, ranks real vs. nit). Run with subagent so each reviews before they clash.

Anti-Consensus Club — for decisions and fuzzy questions where one assistant agrees too fast or debates past the useful point: Wildcard (alternative framings), Level (claim/confidence checker), Killjoy (stops repetition and fake disagreement), Splinter (challenges easy consensus). Run with --party anti-consensus-club --mode subagent.

9.4 Working around the technical-research sub-agent gap

Because bmad-technical-research is single-agent (§5), Party Mode is the sanctioned way to get multi-perspective technical research. Two patterns:

Pattern A — Fan out, then convene. Run several separate technical-research chats, one per candidate approach (fresh chat each), producing one findings doc apiece. Then open a Party Mode room in subagent mode, feed in all the findings docs, and let a purpose-built panel reconcile them. Depth (focused single-agent research per option) and independent cross-examination, without the missing built-in sub-agents.

Pattern B — Build a custom research panel via bmad-party-mode, create a new party (writes to your overrides through bmad-customize). Ready-to-use definitions are in Appendix B.

9.5 Steering, memory, and keepsakes

• Steer actively: bring a voice in, tell one persona to "take that apart," switch rooms mid-session, or summon a named custom member.
• 2–3 active voices per round, not all at once — more turns to noise.
• Non-interactive: --non-interactive "…" runs one pass to a close and releases spawned agents (good for scripted use).
• Keepsake: on wrap-up the orchestrator offers a self-contained HTML document laid out by persona — a genuinely useful artifact to attach to a ticket.
• Memory: saved parties keep lightweight memory of past sessions (dynamics, open threads) unless you turn it off via bmad-customize. Good for a standing panel; off for one-offs you don't want coloring later runs.

10. Where the Facts Come From (Provenance and Retrieval)

The question a technical reader asks first and most docs skip: when a research workflow states a "fact," where did it come from?

BMAD ships no crawler, search index, or retrieval engine. The research workflows are orchestration — structured prompts, step files, personas, output templates. The actual source-gathering rides on the host coding agent's tool access. In Claude Code that means the model's web search, any MCP connectors you've enabled, and file/repo reads. In a locked-down harness with no web tool, the same workflow falls back to the model's parametric knowledge — cutoff-bound, potentially stale or confabulated.

Consequences:

• Research currency is bounded by your host's tools. Market/domain research is only as fresh as the web access behind it.
• Technical research in a brownfield repo sources mostly from your own code — so the host needs file-read access, and (per §9.2) spawned subagents need tool access or they hallucinate.
• Provenance hygiene is on you. Ask the workflow to cite what it actually fetched; treat uncited claims as model recall, not verified fact. Matters most for bmad-technical-research, where a confidently-wrong claim about a library costs you a bad architecture bet.
• Working memory: a memlog primitive (_bmad/scripts/memlog.py) carries state across steps within a run, so a multi-step pass doesn't drag the whole context forward.

The honest framing: BMAD makes research structured and repeatable; it does not make it grounded — grounding is a property of the tools you point it at.

11. Cost, Context, and Running Off-IDE

Token cost is real. Independent testing has put a full planning cycle in the rough ballpark of ~$200 in tokens (an order-of-magnitude signal, not an official figure). Two knobs multiply it:

• Depth (§6) — deeper = more sources, more cross-checking, more tokens.
• Party Mode spawning (§9.1) — subagent/agent-team spin up a separate agent per persona per round, so an N-persona panel is roughly N× the per-round cost of session. Great for independence, expensive by design.

Context management keeps this affordable. BMAD's step-file architecture loads workflow steps just-in-time rather than holding the whole workflow in context, and each SKILL.md uses YAML frontmatter to persist runtime state across turns. The fresh-chat-per-workflow discipline (§7, §9.5) isn't just a quality rule — it's a cost rule; stale accumulated context is both worse and pricier. memlog carries the few things worth remembering so a fresh chat isn't a cold start.

Web Bundles (v1.0, May 2026) are the token-conscious escape hatch. BMAD can package planning/research skills — market research, brainstorming, product briefs, PRFAQ, PRD, UX specs — as Google Gemini Gems and ChatGPT Custom GPTs. Run the research-and-planning-heavy front of the process on a flat-rate web subscription instead of metered IDE tokens, then bring artifacts back into the IDE for solutioning and implementation. For token-conscious use this is the single biggest lever: offload the exploratory, high-token research phase to a fixed-cost surface.

Customization. Research workflows are customizable like every BMad skill via bmad-customize (writing to your overrides). Pin a default depth, set house rules the workflow holds for the whole run, add persistent facts so you don't re-explain your stack every time. The clean, version-safe way to shape behavior without forking the workflow.

12. Brownfield Research and Established Projects

Most scenarios frame research as pre-build validation, but in an existing codebase your real research entry points differ.

bmad-document-project (DP, Analyst) reverse-documents an existing codebase into reference docs. Critically, its output can substitute for Phase 1–2 artifacts, letting a brownfield project enter at Phase 3 (Solutioning) with generated context. For a live system, this is often the actual first move — not greenfield market/domain research.

bmad-generate-project-context (GPC, Analyst) produces project-context.md, which acts as the "constitution" for your project: technical preferences and implementation rules every agent must follow. You can hand-write it at the project root or in _bmad-output/, or auto-generate it by scanning the codebase. For research specifically, a good project-context.md means technical-research passes start already knowing your stack, conventions, and constraints — less re-explaining, sharper findings.

Practical brownfield stance:

• Market/domain research is frequently skippable for internal brownfield work — don't run the full Phase-1 suite out of completeness reflex.
• The useful pair is usually document-project (understand the system) + technical-research (decide one thing), optionally with bmad-investigate (§4) when the question is "why is it behaving like this."
• Feed project-context.md into research runs so findings respect your actual constraints from the first token.

13. Worked Example: A User Recommendation System

This section shows how BMAD research actually works on a real build. The feature: a user recommendation system that suggests content based on a user's view history. It's the ideal case study because it spans all three things research handles — algorithm choice, failure modes, and infrastructure — and because it exposes the hard boundary between what BMAD researches and what your data has to decide.

Framing first (the whole point). A recommender is ~90% technical research, ~10% domain, 0% market (internal feature, not a product to position). And per §10, BMAD will structure and survey the problem — it will not benchmark EMA-vs-two-tower on your click logs. Everything below is decision-support; the empirical answers come from eval, not the research workflow (see §13.7).

13.1 Decompose the feature into research passes

Don't run one giant "research a recommender" prompt — it produces a shallow essay. Split into focused technical-research passes, each a real decision:

Then reconcile the technical passes in a Party Mode panel (§13.5) — the fan-out-then-convene pattern from §9.4, and the way to work around the missing built-in technical sub-agents.

13.2 Pass A — scoring & representation

Real question: how do you turn "what a user viewed" into "what to show next," and keep it fresh without recomputing everything?

Options to compare and the substance you're validating:

• EMA user-preference vector — maintain u ← α·e_item + (1−α)·u per interaction (or time-decayed weight = exp(−λ·Δt)). α (or a half-life) is the recency knob. Cheap, updates online in O(1), naturally forgets stale interests. Known weakness: interest blending — a user who reads both tax law and IP law collapses into a muddy centroid that recommends neither well. That single fact is usually what pushes toward the next option.
• Recent-k / multi-vector — keep the last k interaction embeddings and retrieve by max-similarity (late-interaction style) instead of a single centroid; preserves distinct interests at higher retrieval cost. (Where ColBERT / MUVERA / late-interaction thinking connects.)
• Two-tower learned model — user tower + item tower trained on engagement; best relevance ceiling but adds training infrastructure and a cold-start dependency on engagement data.
• Hybrid — content-based ANN candidate generation (works cold) + collaborative signal in rerank once engagement exists. The usual pragmatic landing spot.

Retrieval mechanics: content/item embeddings in OpenSearch k-NN (HNSW); the user vector (EMA aggregate or multi-vector) drives an ANN query for the candidate set. Dimensionality and quantization (PQ / IVF-PQ) are the recall-vs-cost knobs.

Observation: research lays out this tradeoff space cleanly. It cannot tell you the right α or whether EMA is "good enough" for your users — that's §13.7.

13.3 Pass B — the failure modes (where recommenders actually break)

Duplicates are the headline, but they're one of a family. A good findings doc gives a layered defense and says where each layer sits and what it misses:

The layering insight research should surface: these are ordered stages (candidate-gen → seen-filter → dedup/cluster-collapse → MMR/diversity → impression-discount), and each layer only catches its own class — MMR won't catch exact dupes, seen-set won't catch near-dupes. That ordering is the finding. (The exploration slot in the last row is a one-step special case of a fuller sequential formulation — see §14 for modeling the recommender as a Markov Decision Process.)

13.4 Pass C — AWS architecture & deployment

Real question: how does all of the above run in production within latency, cost, and DR constraints? What good findings look like:

• Two paths. A batch/offline path (embed the corpus, backfill the vector index, precompute where possible) and an online path (per-request retrieve → filter → rerank). Keeping heavy work off the request path is the p95 story.
• State placement. EMA user vectors in DynamoDB (fast point read on the request, O(1) write on interaction, TTL to forget dormant users); content vectors in OpenSearch k-NN; a recommendation Lambda orchestrates retrieve → seen-filter → dedup → MMR rerank → Content API hydrate.
• Embedding generation. Batch backfill (SageMaker Batch Transform / Bedrock) for the corpus; near-real-time for newly ingested content via a Lambda on the ingest event.
• Latency budget is spent mostly in ANN retrieval + rerank; MMR over a bounded candidate set is cheap, a cross-encoder rerank is where you'd blow the budget (ties to Scenario A / reranking decisions).
• Edge & DR. CloudFront + ALB in front; origin-layer DR determines what happens to the rec call when a region degrades — fail to a popularity/editorial fallback rather than erroring.
• Top cost levers. Vector quantization (PQ) vs. recall, embedding-recompute frequency, and DynamoDB read/write capacity mode.

13.5 Reconcile the passes in a panel

Use §9. Open a Party Mode room in subagent mode and feed in the three findings docs — because the passes interact (a multi-vector user model changes ANN cost, which changes the AWS budget, which changes whether a cross-encoder rerank is affordable):

bmad-party-mode --mode subagent
# panel: relevance-engineer, latency-hawk, ops-SRE, cost-owner, eval-skeptic
# input: pass-A-scoring.md, pass-B-failure-modes.md, pass-C-aws.md

The eval-skeptic persona matters most here — its job is to keep asking "how will we know this is better?", forcing the design toward §13.7. (Heed §9.2: make sure the panel can actually read the three docs.)

13.6 What chains downstream

The three findings docs + the panel keepsake feed:

• bmad-create-prd — synthesizes them into requirements (it ingests research findings as input, §2).
• bmad-create-architecture — Pass C becomes the architecture backbone; Passes A/B become the algorithm and quality sections.
• The eval plan — where the real work starts.

13.7 The honest boundary — what the research cannot tell you

This is the "how it actually works here" punchline. BMAD technical research produces a decision-support package: options, tradeoffs, a layered design, cited sources (only if the host has web access — §10), and a proposed evaluation plan.

It does not, and cannot, tell you:

• the right α / half-life for your EMA,
• the right cosine threshold for near-dup collapse or λ for MMR,
• whether EMA beats two-tower on your users,
• your actual p95 under load.

Those are empirical and belong to eval-driven development, not research — adversarial multi-evaluator loops, offline replay against view history, and online A/B. The division of labor:

Research designs the experiment and narrows the option space. Your data runs the experiment and picks the winner.

A recommender is precisely where teams mistake a confident research doc for an answer and skip the eval. The research is real and valuable — it stops you building the wrong thing — but the tuning constants that decide whether the feature is good come from measurement, and the research's job ends at handing you a sound experiment to run.

14. Markov Decision Processes and Stochastic Dependency Graphs

14.1 Does BMAD support this natively? No — and it's worth being precise about why

BMAD's dependency model is a deterministic directed graph with conditional branches and mandatory gates — the graph TD planning flow (idea → optional research → brief → PRD → architecture → gates → implementation), soft preceded-by/followed-by sequencing in module-help.csv, and scale-adaptive heuristic routing to a track (Quick Flow / BMad Method / Enterprise). There is:

• no transition probability between milestones (a gate is pass/fail by review, not P(pass | state, action)),
• no reward/value function or policy being optimized,
• no MDP/POMDP solver, no stochastic scheduler, no Bayesian dependency graph.

"Scale-adaptive intelligence" and bmad-investigate's "evidence-graded findings" sound probabilistic but are heuristic/qualitative grading, not a Markov model. So if the question is "can I hand BMAD a stochastic milestone graph and have it solve for an optimal policy" — no, that's outside its design.

That said, MDPs and stochastic dependency graphs are relevant to your work in two distinct, genuinely useful ways. Neither is a built-in feature; one is a research subject BMAD is well-suited to investigate, the other is an overlay you build on top of BMAD's deterministic graph.

14.2 The concept — what a Markov Decision Process actually is

An MDP is the standard mathematical model for sequential decision-making under uncertainty: an agent repeatedly observes a state, picks an action, receives a reward, and lands (probabilistically) in a new state — and it wants to choose actions that maximize reward over the long run, not just right now.

The Markov property. The defining assumption: the future depends only on the current state, not the whole history — P(s_{t+1} | s_t, a_t, s_{t-1}, …) = P(s_{t+1} | s_t, a_t). In practice you engineer the state to make this true. If recent history matters (it does for a recommender), fold it into the state — and note an EMA vector already summarizes history into one carry-forward number, which is exactly a Markov-friendly representation.

The five pieces ⟨S, A, P, R, γ⟩.

• S — the set of states the world can be in.
• A — the actions available.
• P(s′ | s, a) — the transition model: the probability of the next state given the current state and action. The stochastic heart of the thing.
• R(s, a) (or R(s,a,s′)) — the reward for taking an action.
• γ ∈ [0,1) — the discount factor: a reward t steps away is worth γ^t of face value. γ near 0 = myopic (only now matters); γ near 1 = far-sighted.

What "solving" an MDP means: finding a policy. A policy π(a | s) maps states to actions. The quantity you care about is the expected discounted return from following π:

V^π(s) = E[ r_0 + γ·r_1 + γ²·r_2 + …  | start in s, follow π ]

V^π(s) is the state-value function; Q^π(s,a) is the action-value function (value of taking a in s, then following π). Solving the MDP = finding π* that maximizes value in every state.

The Bellman equation is the recursion that makes this tractable — value is immediate reward plus discounted value of where you land:

V*(s)   = max_a [ R(s,a) + γ · Σ_{s′} P(s′|s,a) · V*(s′) ]
Q*(s,a) =         R(s,a) + γ · Σ_{s′} P(s′|s,a) · max_{a′} Q*(s′,a′)

How you actually solve it — two worlds:

1. Model known (you have P and R). Use dynamic programming: value iteration (sweep the Bellman update to convergence) or policy iteration (alternate evaluate-policy / improve-policy). This is the milestone graph of §14.4 — few states, estimable transitions, solvable in a few lines.
2. Model unknown (you only get to act and observe). This is reinforcement learning. Q-learning updates value estimates from sampled transitions:

   Q(s,a) ← Q(s,a) + α·[ r + γ·max_{a′} Q(s′,a′) − Q(s,a) ]
   

   Policy-gradient methods instead push π directly toward higher return. This is the recommender of §14.3 — you can't write down how a user will respond, so you learn a policy from interaction logs.

Exploration vs. exploitation. With an unknown model, the agent must choose between exploiting the action that looks best now and exploring an uncertain one that might be better and teaches you something. ε-greedy, UCB, and Thompson sampling are the standard strategies — and this tension is exactly the diversity/exploration knob from §13.3, now with a name and a theory.

POMDP — when you can't see the true state. If the agent only gets a noisy observation rather than the state itself, it's a Partially Observable MDP. Almost every real system is one: you never observe a user's true intent, only their clicks. The fix is a belief state (a summary of history — again, the EMA vector is a crude one) used in place of S.

Two intuition anchors before we apply it:

• The bandit is the smallest MDP. A multi-armed bandit is an MDP with one state and γ = 0 — pure "which action pays best right now," no sequential effect. A contextual bandit adds a state (context) but still γ = 0. Set γ > 0 and let actions change the next state and you have a full MDP. That ladder (bandit → contextual bandit → MDP/RL) is the natural adoption path.
• Discounting is a modeling choice, not a detail. γ encodes how much you care about the future — it's where product intent enters the math.

14.3 Applying it #1 — the recommender as an MDP (a research subject, not a BMAD feature)

The strong fit, extending §13 directly. A recommender is a textbook (PO)MDP: the user is the environment, and each recommendation nudges the user into a new state whose response you can't fully predict.

The tuple ⟨S, A, P, R, γ⟩ instantiated:

Myopic vs. sequential — the concrete difference. A greedy ranker (your EMA+MMR design) maximizes immediate reward: show the highest-predicted-click item now. That can quietly degrade the state — hammer someone with the same high-CTR category and they narrow, get bored, and eventually churn. An MDP with γ > 0 will sometimes show a slightly lower-CTR item because it improves the future state — it discovers a new interest, keeps the session alive, protects long-term retention. Same catalog, same infra, different objective: sequence-value instead of click-value. That is the entire reason to reach for an MDP here.

It's really a POMDP, and that's good news for reuse: you never see true intent, but the belief state you can build is the EMA/user vector from §13. So MDP/RL layers on top of §13's representation and serving rather than replacing it — the user vector becomes (part of) S, the ANN candidate set constrains A, and the ranker becomes a learned policy.

The adoption ladder (crawl → walk → run):

1. Contextual bandit (γ = 0) over the rerank slot — the smallest useful step, and a principled version of the ε-greedy exploration you already flagged.
2. Slate bandit / slate MDP — handle that you recommend a set, where items interact (diversity, position bias); SlateQ-style decomposition tames the combinatorial action space.
3. Full RL with γ > 0 optimizing a long-horizon reward (retention, return visits) — highest ceiling, highest cost.

The hard part is evaluation, not the algorithm — off-policy evaluation (OPE). You can't A/B-test every candidate policy on live users. OPE estimates a new policy's value from logged data collected under the current policy, via inverse-propensity scoring (IPS) and doubly-robust estimators. This is the practical gate on RL recommenders and the thing a research pass should design before anyone trains a policy.

Pitfalls research should surface (all real, all expensive to learn late):

• Reward shaping / gaming — optimize raw dwell and you breed clickbait; optimize clicks and you breed rage-bait. The reward is the product's values.
• Credit assignment / delayed reward — retention arrives days later; attributing it to a recommendation three sessions ago is genuinely hard.
• Feedback loops — the policy shapes the data that trains the next policy; unchecked, it collapses diversity (the filter bubble, now self-reinforcing).
• Serving & stability — a learned policy in the request path adds latency, a model-serving dependency, and the risk of a bad policy reaching users.

How BMAD fits: research it with a technical-research pass (prompt A8 in Appendix A) — "MDP/RL vs. heuristic, and if RL, what's the OPE plan and the crawl-walk-run path?" The §13.7 boundary applies with extra force: whether RL beats EMA+MMR on your data is empirical, and the cost of being wrong is higher. Research defines the experiment (state/action/reward design, the OPE methodology, rollout guardrails); your logs decide whether to climb the ladder.

14.4 Applying it #2 — modeling BMAD's own milestones as a stochastic graph (an overlay you build)

The literal reading of the question — a stochastic dependency graph between milestones — and where the model-known branch of §14.2 applies. BMAD gives you the graph structure for free (phases, gates, and the preceded-by/followed-by edges in module-help.csv); you supply the probabilities and solve it outside BMAD.

Two versions, increasing power:

• Markov chain (no decisions) — model gate outcomes as random transitions to estimate schedule risk; this is essentially stochastic PERT.
• MDP (with decisions) — add actions at each gate — {proceed, iterate/rework, cut scope, run more research} — and solve for the optimal policy: given where the plan is and how risky the next gate looks, what's the cost-minimizing move?

A worked example. Model each gate as a transition over {pass, rework}:

              p_pass = 0.7
   [Arch gate] ───────────────▶ [Implementation]
       │  ▲
p_rework│  │ 0.3   (loop back, pay rework cost c_r)
       ▼  │
   [Re-architect]

• Expected attempts to clear a gate with pass-probability p is geometric: 1/p. At p = 0.7 that's ≈ 1.43 attempts; at p = 0.5 it's 2.0. Each extra attempt costs c_r (rework time).
• Chain three independent gates at p = 0.7: P(clean run) = 0.7³ ≈ 0.34 — only a third of projects clear all three without rework. That number alone reframes how you plan.
• Expected total rework ≈ Σ_gates (1/p_g − 1)·c_r — a concrete, defensible schedule-risk figure.

The elegant payoff — value of information. Treat "run a technical-research pass" as an action that changes a transition probability: it raises the architecture gate's p_pass (fewer late re-architecture loops). Then:

value of the research  ≈  (expected rework cost WITHOUT research)
                        − (expected rework cost WITH research)
research is worth it   ⇔   that difference  >  the research's own cost

If a research pass lifts the architecture gate from p = 0.5 to p = 0.7, expected attempts drop from 2.0 to ≈ 1.43 — saving ≈ 0.57 rework cycles at that gate. If one rework cycle costs more than the research pass, research pays for itself. This turns §13.7's qualitative "research stops you building the wrong thing" into a decision-theoretic ROI — the value-of-information calculation at the heart of MDP planning.

How to solve it: the graph is tiny (a handful of states), so value iteration by hand or ~20 lines of NumPy yields the optimal gate policy; pure schedule risk is a PERT / Monte-Carlo run.

What BMAD does / doesn't do: it emits the milestone graph and the artifacts; it does not estimate the probabilities or solve anything. You pull the edges (from module-help.csv or the planning flow), attach p values from your own historical delivery data (the weak link — garbage in, garbage out), and solve it in a notebook. A tidy loop: use bmad-technical-research to design the risk model, then own it yourself. Keep the honest caveat: the map is not the territory — a clean expected-value number built on three guessed probabilities is worse than no number.

14.5 Honest summary

• Native BMAD support for MDP / stochastic dependency graphs: none. Its model is a deterministic gated DAG with heuristic routing.
• The concept (§14.2) is sequential decision-making under uncertainty — ⟨S,A,P,R,γ⟩, solved for a policy via dynamic programming (model known) or RL (model unknown), with exploration-vs-exploitation and partial observability as the two complications that always show up in practice.
• MDP as a recommender technique (§14.3): fully applicable — the sequential, long-horizon generalization of the greedy ranker in §13, a natural bmad-technical-research subject, with off-policy evaluation as the real gate.
• Stochastic graph over BMAD's own milestones (§14.4): a valid overlay you build on the graph BMAD gives you — useful for quantifying plan risk and computing the value-of-information ROI of research itself, but solved outside BMAD.

15. Observations and Findings

Strengths

• Artifact-first design is the real value. Research isn't a chat you lose — it's a document the PRD/architecture workflows consume. The compounding across phases is the point.
• Market research is genuinely multi-agent and produces broad, structured coverage with little hand-holding.
• Clean separation of the three types maps to actually-different questions and stops you conflating "is it buildable" with "does anyone want it."
• Scale-adaptive: depth adjusts to project complexity, so bug-fix-sized work isn't pushed through enterprise-weight research.

Weaknesses / watch-outs

• Technical research is single-agent — weakest of the three for depth and speed. For high-stakes bets, plan to iterate and supply strong constraints. It amplifies a sharp prompt; it won't out-think a vague one.
• Depth options are opaque across versions — confirm at runtime.
• Command drift between short (bmad-*) and module-qualified (bmad-bmm-*) forms; bmad-help is the source of truth.
• subagent grounding regression (§9.2) — verify tool access before trusting a panel on codebase-grounded questions.
• Fresh-chat discipline is load-bearing — the most common cause of degraded, costlier output.
• Grounding is not guaranteed (§10) — research is only as current as the host's tools.

Complementary modules

• Creative Intelligence Suite (cis) — facilitated ideation frameworks (SCAMPER, reverse brainstorming, reframing) for before you have a crisp question to research.
• BMad Builder (bmb) — author your own .agent.yaml sub-agents / workflows; the sanctioned path to a personal "architecture-evaluator" sub-agent that closes the technical-research gap.
• Test Architect (tea) — risk-based test strategy and NFR assessment; research-adjacent when the question is "how will we validate this."

16. Practical Tips and Best Practices

1. For technical research, over-specify constraints. Latency budgets, runtime fit, "no new platform," existing infra you must reuse. The single agent leans on your framing.
2. State the decision, not just the topic. "Which of A/B/C and why" produces a usable doc; "tell me about reranking" produces an essay.
3. One decision per pass. Fan out across passes, then convene a panel (§9.4).
4. Match Party Mode mode to intent — session for ideation, subagent for honest critique/focus groups, agent-team for hands-off debate.
5. Verify subagent grounding before trusting a panel on real code/data (§9.2).
6. Route durable research to docs/, cycle-specific to planning-artifacts/.
7. Feed project-context.md into research so findings respect your stack from the first token (§12).
8. Offload to Web Bundles when tokens matter — run the heavy research phase on a flat-rate GPT, bring artifacts back (§11).
9. Take the keepsake HTML and attach it to the ticket; it's a real artifact.
10. Remember the boundary — research designs the experiment; your data picks the winner (§13.7).
11. Use bmb to close the sub-agent gap if you do technical research often on the same stack.
12. Always fresh chat per workflow — quality and cost both depend on it.

17. Quick Reference

Research commands

# Fresh chat, then:
bmad-agent-analyst          # load Analyst (Mary); exposes MR / DR / TR

bmad-market-research        # MR — competitors, sizing, trends (5 sub-agents)
bmad-domain-research        # DR — terminology, regulation, SME depth
bmad-technical-research     # TR — feasibility, architecture, libs (single-agent)

bmad-help                   # source of truth for what's installed / what's next

Research-adjacent

bmad-document-project           # DP  — reverse-document an existing codebase (brownfield)
bmad-generate-project-context   # GPC — write project-context.md ("constitution")
bmad-investigate                # forensic, evidence-graded system analysis (v6.8)
bmad-brainstorming              # guided ideation (before research)
bmad-party-mode                 # multi-persona room (--mode session|auto|subagent|agent-team)
bmad-customize <skill>          # pin defaults, house rules, persistent facts

BMM agents (persona · key menu codes)

Output → _bmad-output/planning-artifacts/ (cycle) or docs/ (durable). Feeds → bmad-create-prd, bmad-product-brief, bmad-create-architecture.

18. Glossary

BMAD terms

• BMM — BMad Method module; the flagship four-phase agile-AI module.
• Analyst (Mary) — the agent that runs all three research workflows.
• Skill — a workflow packaged as a SKILL.md + sharded step files; runs synchronously in the host IDE.
• Step file — step-XX-*.md, loaded just-in-time to keep context lean.
• module-help.csv — catalog mapping skills to names, menu codes, phases, output locations; powers bmad-help next-step routing.
• Party Mode — multi-persona conversation; four modes govern independence.
• Sub-agent — a specialized agent spawned within a workflow (market research) or a party (subagent mode).
• Scale-Adaptive Intelligence — automatic planning-depth adjustment by project complexity; expressed as Quick Flow / BMad Method / Enterprise tracks.
• project-context.md — the project "constitution": rules all agents follow.
• memlog — working-memory primitive carrying state across steps in a run.
• Web Bundles — planning skills packaged as Gemini Gems / ChatGPT GPTs for flat-rate, off-IDE planning.
• Gate — a mandatory checkpoint before advancing phases.

Recommender terms (used in §13)

• EMA — exponential moving average; u ← α·e + (1−α)·u for recency-weighted user profiles.
• Half-life / α / λ — the recency-decay knobs (interaction-count or time-based).
• Vectorization / embedding — mapping content/users to vectors for similarity search.
• ANN / HNSW / IVF-PQ / PQ — approximate nearest-neighbor search and its index/quantization variants (recall-vs-cost).
• Two-tower — separate learned encoders for users and items.
• MMR — Maximal Marginal Relevance; diversity rerank trading relevance against redundancy.
• Impression discounting — down-weighting recently-shown-not-clicked items.
• Inverse-propensity weighting — correcting popularity bias.
• Cold start — no history for a new user/item.
• Eval-driven development — deciding tuning constants by measurement (offline replay, adversarial evaluators, online A/B), not by research.

MDP and stochastic-planning terms (used in §14)

• MDP — Markov Decision Process; ⟨states, actions, transition probabilities, reward, discount γ⟩ for sequential decisions under uncertainty.
• Markov property — the future depends only on the current state, not the full history; you engineer the state to make this hold.
• Policy (π) — the state → action mapping an MDP solution produces (which slate to show in a given context).
• Value function (V, Q) — expected discounted return from a state (V) or a state-action pair (Q) under a policy.
• Bellman equation — the recursion V*(s) = max_a[R + γ·Σ P·V*] that defines optimal value and underlies every solution method.
• Value iteration / policy iteration — dynamic-programming solvers used when the model (P, R) is known.
• Reinforcement learning (Q-learning, policy gradient) — learning a policy from experience when the model is unknown.
• Exploration vs. exploitation — the trade between the best-known action and an uncertain one that might be better (ε-greedy, UCB, Thompson sampling).
• POMDP / belief state — partially observable MDP; the true state is inferred from observations via a history summary (a recommender rarely sees real intent, so it's really a POMDP; the EMA vector is a crude belief state).
• Contextual bandit — a one-step MDP (γ = 0); the frame behind the exploration / ε-greedy slot in §13.3.
• Discount factor (γ) — weights future vs. immediate reward; γ > 0 is what makes a recommender optimize long-term engagement over next-click.
• Off-policy evaluation (OPE) — estimating a new policy's value from logged data without deploying it live (IPS, doubly-robust); the practical gate on RL recommenders.
• SlateQ — a slate-decomposition value-learning approach for recommending sets of items under an RL/MDP formulation.
• Value of information — the expected cost/benefit of acquiring information (e.g. running research) before deciding; the ROI frame in §14.4.
• Stochastic dependency graph / PERT — a milestone graph whose edges carry probabilities (gate pass/rework), used to reason about plan time/cost risk (§14.4).

19. Appendix A: Copy-Paste Research Prompts

Ready-to-run prompt bodies. Start a fresh chat, load the Analyst, invoke the workflow, then paste. Replace bracketed parts.

A1 — Technical research (generic template)

bmad-technical-research

Decision: [the one thing to decide].
Options to compare:
1. [option] — [one line]
2. [option] — [one line]
3. [option] — [one line]
Constraints: [latency budget], [runtime/language fit], [existing infra to reuse],
[no-go's].
Output: comparison across [criteria], a recommended default, and a minimal
validation/PoC path.
Depth: [read the options the workflow offers].

A2 — Recommender Pass A (scoring & representation)

bmad-technical-research
Decision: how to represent a user's taste from view history and score candidates.
Options: (1) EMA user vector u←α·e+(1-α)·u + HNSW ANN; (2) recent-k multi-vector /
late-interaction; (3) two-tower learned; (4) hybrid (content ANN + collaborative
rerank). Constraints: O(1)-ish online update (no nightly full recompute),
Java/Spring service, OpenSearch vector store in place, tight p95.
Output: compare on freshness, multi-interest handling, cold-start, infra cost,
online-update cost; recommend a default + migration path.

A3 — Recommender Pass B (failure modes)

bmad-technical-research
Decision: mitigation strategy for recommender failure modes, view-history based.
Cover with a concrete mitigation + pipeline stage: exact duplicates, near-duplicates,
no-diversity, cross-session repetition, popularity bias, cold start, staleness.
Constraints: OpenSearch retrieval, view history queryable (Athena), content has a
cluster/grouping field. Output: a layered defense table, the order to apply layers,
and what each layer misses.

A4 — Recommender Pass C (AWS architecture)

bmad-technical-research
Decision: AWS reference architecture, batch + online paths. Cover: embedding
generation (batch backfill vs near-real-time on ingest), OpenSearch k-NN index +
quantization tradeoff, EMA user-state store (DynamoDB + TTL), serving path (rec
Lambda → ANN retrieve → seen-filter → dedup → MMR rerank → Content API hydrate),
CloudFront+ALB with origin-layer DR, warm/cold path, caching, cost drivers.
Constraints: tight p95, no GPU in request path, existing HNSW index.
Output: component/data-flow description, DR posture, top 3 cost levers.

A5 — Domain research (unfamiliar space)

bmad-domain-research
Domain: [the space]. I need: the terminology practitioners use, how "relevant"/
"good" is judged here vs. elsewhere, regulatory/ethical constraints, and the
evaluation lens I'd otherwise miss.

A6 — Market research (validation)

bmad-market-research
Space: [the category]. Who are the incumbents and adjacent players, the current
trend line, and where are the gaps a fast-follower could exploit?

A7 — Reconcile passes in a panel

bmad-party-mode --mode subagent
Convene: relevance-engineer, latency-hawk, ops-SRE, cost-owner, eval-skeptic.
Inputs (read these files): pass-A-scoring.md, pass-B-failure-modes.md, pass-C-aws.md.
**A8 — Recommender as MDP / RL (should we?)**

bmad-technical-research Decision: should the ranking stage be formulated as an MDP/RL problem (optimizing long-term engagement) or stay a greedy EMA+MMR heuristic (optimizing the next slate)? Frame: S = user vector + session + recent impressions; A = slate; P = stochastic user response; R = engagement/retention; γ = long-term weight. Compare: contextual bandit (one-step), slate-decomposition value learning (SlateQ-style), and the heuristic baseline. Constraints: no live experimentation on every policy (need off-policy evaluation), tight p95, existing OpenSearch/DynamoDB serving. Output: when RL is worth its cost, the off-policy eval plan, serving/complexity implications, and a recommended crawl→walk→run path from the heuristic baseline.

---

## 20. Appendix B: Custom Party Definitions

Author these with `bmad-party-mode, create a new party` (it writes to your overrides
via `bmad-customize`). The exact override-file format is version-specific and the
skill writes it for you — provide the personas and scene below when prompted.
Personas need *teeth*: specific values, first-move behavior, and a blind spot.

**B1 — Reranking / Scoring Decision Panel** (run `--mode subagent`)

- **Rel (Relevance Engineer).** Fights for measured relevance lift (nDCG, MRR,
  recall@k). Opens by asking for the offline metric and the candidate set size.
  Blind spot: discounts operational cost.
- **Pax (Latency Hawk).** Rejects anything that risks the p95 budget or puts a GPU
  in the request path. First question is always "what does this add to tail
  latency?" Blind spot: would ship a fast but mediocre ranker.
- **Sol (Ops/SRE).** Guards index rebuild cost, operational surface, and failure
  modes under region degradation. Blind spot: conservative to a fault.
- **Cass (Cost Owner).** Ranks options by cost-per-query and recompute frequency;
  challenges anything without a cost line. Blind spot: penny-wise on relevance.
- **Vera (Eval Skeptic).** Won't accept any recommendation without a concrete
  experiment that could disprove it. Says "how will we *know*?" in the first thirty
  seconds. Blind spot: can stall a decision that's good enough.

*Scene:* a design review for the recommender's ranking stage; the team must pick one
approach and a validation plan by end of session, and the eval skeptic has veto over
"just ship it."

**B2 — View-History Focus Group (from data)** (run `--mode subagent`)

Hand the room **anonymized user view-history clusters**; it builds representative
personas that react to a proposed recommendation strategy *from their own
behavior*, instead of one model rubber-stamping. Example seeded members:

- **The Specialist.** Reads deep in one narrow area; hates diversity injection that
  pulls them off-topic. Reacts badly to exploration slots.
- **The Generalist.** Grazes across many areas; punished by a single-centroid EMA
  that averages their interests into mush.
- **The Returner.** Comes back weekly; furious when shown items they already read
  (tests your seen-set/impression-discounting).
- **The New Arrival.** Little history; the cold-start canary.

*Scene:* each persona is shown a sample recommendation slate produced by the
candidate strategy and reacts from their own goals; the room surfaces which user
type each design choice helps or hurts.

> **Grounding note (§9.2):** for B2 to be worth anything, the spawned personas must
> actually consume the cluster data you provide — verify tool/file access, or paste
> the cluster summaries directly into the room.

---

## 21. Sources and Recency

Synthesized from the current BMAD-METHOD docs and repository
(docs.bmad-method.org, the `bmad-code-org/BMAD-METHOD` repo, its releases/issues,
and the DeepWiki module reference), plus third-party testing writeups, reflecting
the V6 / BMM state through mid-2026.

Version-sensitive or unverified items, flagged inline where they appear:

- **Research depth option values** — defined in workflow step files, not public
  docs; not verified. Read the options at runtime or ask `bmad-help`.
- **Short vs. `bmm`-qualified command form** — varies by build; `bmad-help` is
  authoritative.
- **The `subagent` grounding regression (§9.2)** and the open request for true
  native-subagent parallelism — both tracked upstream and actively changing;
  re-verify grounding on your build before trusting `subagent` mode for factual,
  code-grounded work.
- **The ~$200 full-cycle token figure (§11)** — one independent measurement, not an
  official number; treat as order-of-magnitude.
- **Custom party override-file format (Appendix B)** — the skill writes it; the
  exact schema is version-specific.

The recommender engineering content in §13 (EMA, MMR, ANN/quantization, the AWS
shape) is stable engineering knowledge, independent of BMAD version.

*Confirm anything load-bearing against your installed version with `bmad-help`.*

Source: Similarity-Calculations · Similarity-Calculations.md · updated 2026-07-23 · 🔒 secret gist

Synced verbatim from gist.github.com/bl9.

Is binary_clip Using Cosine Similarity? A Ground-Up Explanation

This document answers one question about the mew2026 OpenSearch index:

The binary_clip field is configured with space_type: innerproduct. Is that effectively cosine similarity, or plain dot product? And does using faiss with 16x compression mean the vectors are already normalized?

Short answer, proven with your own data in Section 8: the vectors are not normalized (their lengths are ~9–11, not 1), so this field computes raw dot product, which is not cosine similarity. faiss and the compression level have nothing to do with normalization.

This version assumes no prior background. Every term is defined, the origin of the name is explained (why "L2", where "Euclidean" comes from, what "inner product" means), and every concept has a worked numeric example. If you already know the basics, skip Section 0 and start at Section 1.

Table of Contents

0. Background: the words and ideas you'll need
1. The building block: what a vector is
2. What a "norm" is, and where "L2" comes from
3. Three ways to measure similarity (with a hand-worked example)
4. The key insight: normalization makes all three agree
5. Why innerproduct specifically requires you to normalize
6. Why faiss and 16x compression do NOT mean "normalized"
7. How to verify it yourself (the exact steps)
8. The proof, using your five sampled vectors
9. Conclusion
10. How to fix it
11. Glossary
12. References

0. Background: the words and ideas you'll need

Before any math, here is the cast of characters. Each is expanded later; this is the map.

Embedding. A machine-learning model reads something complex (an image, a sentence) and outputs a fixed-length list of numbers that captures its "meaning" in a form computers can compare. That list is called an embedding or embedding vector. Similar inputs get similar lists. Your binary_clip field stores one 512-number embedding per news photo.

CLIP. The model that produced these embeddings. CLIP stands for Contrastive Language–Image Pre-training (OpenAI, 2021). It was trained on image–caption pairs so that a picture and its description land at nearby points. A useful consequence: you can search images by text, because both live in the same 512-dimensional space.

Vector / dimension. "Vector" is just the math word for an ordered list of numbers. Each number is one dimension or component. We live in 3 dimensions (left-right, up-down, forward-back), but math places no limit — 512 dimensions simply means "a list of 512 numbers." You can't picture 512-D, but every formula that works in 2-D works identically in 512-D; the examples below use 2-D so you can check them by hand.

Norm. A norm is any rule that measures the "size" or "length" of a vector, written with double bars: ‖v‖. There are several norms; the one everyone means by default is the L2 / Euclidean norm (Section 2).

Similarity metric. The rule for scoring how alike two vectors are. In OpenSearch this is the field's space_type. The three that matter — dot product, cosine, Euclidean — are Section 3.

Normalization. Rescaling a vector so its length becomes exactly 1, keeping its direction. This is the single operation the entire question turns on (Section 4).

faiss. The software library OpenSearch uses under the hood to store vectors and find nearest neighbors quickly. The name stands for Facebook AI Similarity Search. It is a search engine for vectors, not a data-cleaning tool — it does not modify your numbers (Section 6).

HNSW. The specific index structure faiss builds: Hierarchical Navigable Small World graph. It's a shortcut that avoids comparing your query against every stored vector. It affects speed, not the numbers (Section 6).

Quantization / "16x compression". Storing each number in fewer bits to save memory, by rounding it to a coarse set of allowed values. "16x" means 16× smaller than the original. It approximates your numbers; it does not rescale them (Section 6).

With that map in hand, we start from the smallest piece.

1. The building block: what a vector is

CLIP turns an image into an embedding vector: a list of 512 numbers. Picture each vector as an arrow from the origin (the zero point) into space. Two properties describe any arrow:

• Direction — which way it points. For CLIP, direction carries the meaning: similar images point similar ways.
• Magnitude / length — how long the arrow is, written ‖v‖. For CLIP this is mostly a model artifact and carries little meaning.

What your actual data looks like (first 8 of the 512 numbers from document MT1IMGOST00034M2TL):

[-0.0992932, 0.3014491, -0.123002, -0.3986786, 0.4534476, -0.470669, 0.1424519, 0.0870281, ...]

Most components are small (±0.1–0.5). But a few, further along, are large (one is −3.33). That mixture is the tell we'll return to.

Why "dimension" here isn't the everyday meaning. In daily life "dimension" means a physical axis. In linear algebra it just means "one slot in the list." 512 dimensions = 512 slots. Nothing is spatial about it; it's bookkeeping. The reason we still use spatial words ("length", "direction", "angle") is that the formulas are the same ones you learned for 2-D and 3-D — they extend to any number of slots unchanged.

2. What a "norm" is, and where "L2" comes from

A norm answers "how big is this vector?" It's a function that takes a vector and returns a single non-negative number: its size. There is more than one sensible way to measure size, so there is a family of norms.

The Lᵖ family (this is where the letters and numbers come from)

The general formula, with a chosen exponent p:

‖v‖ₚ = ( |v₁|ᵖ + |v₂|ᵖ + ... + |vₙ|ᵖ ) ^ (1/p)

Different values of p give different norms:

• p = 1 → the L1 norm (a.k.a. Manhattan or taxicab distance): just add up the absolute values. It's called "taxicab" because it's how far a taxi drives on a grid of streets — you can't cut diagonally through buildings.
• p = 2 → the L2 norm (a.k.a. Euclidean norm): square each, sum, take the square root. This is ordinary straight-line length.
• p = ∞ → the L∞ norm: just the single largest absolute component.

So "L2" literally means "the Lᵖ norm with p = 2." The 2 is the exponent in the formula. And the "L" honors Henri Lebesgue, the French mathematician after whom the Lᵖ spaces of functions are named; the vector norms inherit the letter. (Pedantic note: for finite lists like ours the strictly correct symbol is a lowercase ℓ — "little-ell-2" or ℓ² — while capital L is for continuous functions, but in engineering "L2" is used for both and everyone understands it.)

Why L2 is also called "Euclidean"

Euclid of Alexandria (~300 BCE) wrote the Elements, the foundation of geometry. Straight-line distance in flat space is "Euclidean" in his honor. The L2 formula is nothing more than the Pythagorean theorem (a² + b² = c²) extended to many dimensions:

length of arrow = sqrt( sum of squares of its components )

That's why squaring (exponent 2) and then square-rooting appears: it's Pythagoras. The squaring is what makes it "straight-line" distance rather than grid distance.

One worked example, all three norms

Take v = [3, 4]:

L1  (p=1):  |3| + |4|                 = 7          (add absolute values)
L2  (p=2):  sqrt(3² + 4²) = sqrt(25)  = 5          (Pythagoras — the default "length")
L∞ (max):   max(|3|, |4|)             = 4          (largest component)

import numpy as np
v = np.array([3.0, 4.0])
np.sum(np.abs(v))     # L1  -> 7.0
np.linalg.norm(v)     # L2  -> 5.0   (np.linalg.norm defaults to p=2)
np.max(np.abs(v))     # Linf-> 4.0

The same three on the real 8-number slice above: L1 = 2.076, L2 = 0.8543, L∞ = 0.4707 — different measurements of the same vector.

For the rest of this document, "norm" means L2 (the default everywhere in vector search), and the L2 norm is the single most important calculation here: the test in Section 8 is literally "compute the L2 norm of your real vectors and check whether it equals 1."

3. Three ways to measure similarity (with a hand-worked example)

A similarity metric (OpenSearch's space_type) turns two vectors A and B into one number saying how alike they are. We compute all three on the same pair so the differences are visible.

Example pair: A = [3, 4], B = [4, 0]. (‖A‖ = 5, ‖B‖ = 4.)

3a. Dot product (a.k.a. inner product)

Multiply matching components, sum them:

dot(A, B) = A₁·B₁ + A₂·B₂ = 3·4 + 4·0 = 12

Why two names — "dot product" and "inner product"? "Dot product" comes from the notation A · B (a literal dot). "Inner product" is the more general, abstract term from linear algebra for this whole category of operation; the dot product is the standard inner product on ordinary (Euclidean) space. OpenSearch's innerproduct is exactly this: A · B. The two names mean the same thing here.

The crucial identity — dot product blends direction and length together:

dot(A, B) = ‖A‖ · ‖B‖ · cos(θ)      (θ = the angle between the two arrows)

So a longer vector inflates all its dot products. This is the metric your field uses.

3b. Cosine similarity

Divide the dot product by both lengths — this cancels magnitude and leaves only the angle:

              dot(A, B)        12
cos(A, B) = ───────────── = ─────── = 0.6
             ‖A‖ · ‖B‖       5 · 4

Trig refresher — what "cosine" and "angle" mean here. Cosine is the trigonometric function that, for an angle θ, ranges from +1 at 0° (arrows point the same way), through 0 at 90° (arrows perpendicular — called orthogonal, meaning "unrelated"), to −1 at 180° (arrows point opposite ways). "Angle between two 512-D arrows" sounds exotic, but any two arrows — no matter how many dimensions — always lie in a single flat plane together, and the angle in that plane is an ordinary 0°–180° angle. Cosine similarity is just the cosine of that angle. That's why it's the natural measure of direction alone: it ignores how long the arrows are and reports only how aligned they are.

3c. Euclidean distance (L2 distance)

The straight-line distance between the two arrow tips (smaller = more similar). It reuses the Pythagorean/L2 idea from Section 2, applied to the difference A − B:

d(A, B) = ‖A − B‖ = sqrt((3−4)² + (4−0)²) = sqrt(1 + 16) = sqrt(17) ≈ 4.123

Your field is not using this (space_type is innerproduct, not l2); it's shown for contrast, and to make clear the Euclidean-vs-cosine question is already closed by the config string alone.

All three in code:

import numpy as np
A = np.array([3.0, 4.0]); B = np.array([4.0, 0.0])
A @ B                                            # dot       -> 12.0   ('@' is dot product in numpy)
A @ B / (np.linalg.norm(A) * np.linalg.norm(B))  # cosine    -> 0.6
np.linalg.norm(A - B)                            # euclidean -> 4.1231

Why length "leaking in" actually matters

Make A twice as long (same direction): A2 = [6, 8].

dot(A2, B) = 6·4 + 8·0 = 24        # DOUBLED — purely because A got longer
cos(A2, B) = 24 / (10 · 4) = 0.6   # UNCHANGED — direction didn't change

Under dot product, a document can outrank another just for being "longer," even with identical direction. Under cosine it can't. Hold onto this — it is exactly what's happening in your index.

4. The key insight: normalization makes all three agree

Normalizing rescales a vector to length 1 while preserving direction:

v_normalized = v / ‖v‖

Divide every component by the vector's own L2 length. The result is a unit vector (length 1). Geometrically, all unit vectors sit on the surface of a sphere of radius 1 centered at the origin — the unit sphere — so normalizing is "slide the arrow tip in/out until it touches the unit sphere, without turning it."

Worked example. Normalize A = [3, 4] (‖A‖ = 5):