Source: bmad_e2e · bmad_e2e.md · updated 2026-08-02 · 🔒 secret gist

Synced verbatim from gist.github.com/bl9.

BMAD-METHOD v6 — End-to-End Guide

A working reference for every BMAD skill: what it does, when to reach for it, what prompt to type, what comes back, and how it fits the pipeline.

Sourced from the official v6 docs (docs.bmad-method.org) — Agents, Skills, Core Tools, and Workflow Map references — plus notes on where the docs are internally inconsistent.

Table of Contents

1. Mental Model
   • 1.1 Skills vs. Agent Menu Triggers
   • 1.2 Where Skills Live
   • 1.3 Install and Discovery
2. The Full Map
3. Core Module — The Eight Always-Installed Skills
   • 3.1 bmad-help
   • 3.2 bmad-advanced-elicitation
   • 3.3 bmad-review
   • 3.4 bmad-customize
   • 3.5 bmad-brainstorming
   • 3.6 bmad-deep-recon
   • 3.7 bmad-forge-idea
   • 3.8 bmad-party-mode
   • 3.9 Choosing Between the Four Thinking Skills
4. BMM Agents
5. Phase 1 — Analysis
6. Phase 2 — Planning
7. Phase 3 — Solutioning
8. Phase 4 — Implementation
9. The Four Entry Paths
10. Artifact Flow
11. Worked Example — Multilingual Search Feature
12. Deprecations, Renames, and Gotchas
13. Cheat Sheet
14. Appendix A — The Research Firewall
    • A.1 The Rule, Verbatim
    • A.2 What It Prevents
    • A.3 Worked Example — With and Without
    • A.4 What a Subagent Brief Actually Contains
    • A.5 Where the Firewall Applies
    • A.6 The Escape Hatches
    • A.7 Limits — What It Does Not Guarantee
    • A.8 Provenance — PR #2611
15. Appendix B — Why the Firewall Works: Vocabulary, Philosophy, and Scenarios
    • B.1 Start Here — One Idea You Already Use
    • B.2 The Vocabulary
    • B.3 Discovery vs. Justification
    • B.4 Why Human Brains Need This
    • B.5 How Other Fields Solved It
    • B.6 The Machine-Learning Version You Already Know
    • B.7 Where BMAD Sits in That Lineage
    • B.8 Six Scenarios on a Recommendation System
    • B.9 What the Firewall Does Not Fix
    • B.10 Glossary and Sources

1. Mental Model

BMAD's premise is context engineering: AI agents make inconsistent decisions when they lack structured context, so each phase produces documents that become the input context for the next. The PRD tells the architect which constraints matter; the architecture tells the dev agent which patterns to follow; spec files give focused, complete context for implementation.

The framework is scale-adaptive. There is no single mandatory sequence — a one-line bug fix and a greenfield platform both run through BMAD, but they touch very different numbers of skills.

1.1 Skills vs. Agent Menu Triggers

Two ways to start work. They are not alternatives to each other so much as two ergonomics for the same underlying workflow files.

flowchart TD
    A[You want to do something] --> B{Do you know which<br/>workflow you want?}
    B -->|Yes| C["Type the skill name<br/><code>bmad-prd</code>"]
    B -->|"No — already mid-conversation<br/>with an agent"| D["Type the trigger code<br/><code>PRD</code>"]
    C --> E[Workflow runs directly<br/>no persona loaded]
    D --> F[Agent starts the same workflow<br/>persona and context preserved]
    E --> G[Same underlying workflow file]
    F --> G

Practical rule: use skills when you know the destination. Use triggers when you're already deep in a session with an agent and don't want to lose the conversational context by switching.

1.2 Where Skills Live

The installer writes one skill directory per agent, workflow, task, and tool. Each directory holds a SKILL.md that instructs the AI to load the corresponding source file.

.claude/skills/
├── bmad-help/
│   └── SKILL.md
├── bmad-prd/
│   └── SKILL.md
├── bmad-agent-dev/
│   └── SKILL.md
└── ...

The directory name is the skill name. bmad-agent-dev/ registers the skill bmad-agent-dev.

1.3 Install and Discovery

# Standard install
npx bmad-method install

# Prerelease (higher churn)
npx bmad-method@next install

# Non-interactive config override, repeatable
npx bmad-method install --set bmm.output_folder=_bmad-output

# See what options a module exposes
npx bmad-method install --list-options bmm

The canonical list of what you actually have installed:

ls .claude/skills/ | grep bmad

Everything in this guide is the v6 default set. Your installation only contains skills for the modules you selected. If a skill in this document doesn't exist for you, re-run the installer and check your module selection.

2. The Full Map

flowchart TD
    subgraph CORE["CORE MODULE — available in every phase, always"]
        H["<b>bmad-help</b><br/>what do I do next"]
        AE["<b>bmad-advanced-elicitation</b><br/>refine recent output"]
        RV["<b>bmad-review</b><br/>multi-lens review"]
        CU["<b>bmad-customize</b><br/>TOML overrides"]
    end

    subgraph P1["PHASE 1 — ANALYSIS (optional)"]
        BS["bmad-brainstorming"]
        FI["bmad-forge-idea"]
        DR["bmad-deep-recon"]
        PB["bmad-product-brief"]
        PF["bmad-prfaq"]
    end

    subgraph P2["PHASE 2 — PLANNING"]
        PRD["bmad-prd"]
        UX["bmad-ux"]
        SP["bmad-spec"]
    end

    subgraph P3["PHASE 3 — SOLUTIONING"]
        AR["bmad-architecture"]
        PC["bmad-generate-project-context"]
        ES["bmad-create-epics-and-stories"]
        IR["bmad-check-implementation-readiness"]
    end

    subgraph P4["PHASE 4 — IMPLEMENTATION"]
        SPL["bmad-sprint-planning"]
        BD["<b>bmad-build</b><br/>convergence point"]
        BA["bmad-build-auto"]
        CR["bmad-code-review"]
        SS["bmad-sprint-status"]
        CC["bmad-correct-course"]
        RT["bmad-retrospective"]
    end

    BS --> PB
    FI --> PB
    DR --> PB
    PB --> PRD
    PF --> PRD
    PRD --> UX
    PRD --> SP
    UX --> AR
    SP --> AR
    AR --> PC
    PC --> ES
    ES --> IR
    IR --> SPL
    SPL --> BD
    SP -.->|"stories.yaml"| BA
    BD --> CR
    BD --> SS
    SS --> CC
    SS --> RT
    BA --> BD

    PRD -.->|"skip everything —<br/>small scoped work"| BD

The dotted line matters: clear, well-scoped work enters bmad-build directly. Phases 1–3 add context; they do not select a different implementation workflow. Skipping them is intended behavior, not a shortcut.

3. Core Module — The Eight Always-Installed Skills

Four kernel tools plus four thinking skills. No agent session required — type the skill name and it runs.

Doc inconsistency: the Core Tools page prose says "seven core skills" while its own tables list eight (four kernel + four thinking). The tables are correct.

3.1 bmad-help

Your intelligent guide to what comes next. Inspects project state, detects what's been done, and recommends the next required or optional step.

How it works

flowchart LR
    A["bmad-help<br/>+ optional NL query"] --> B[Scan project for artifacts<br/>PRD, architecture, stories]
    B --> C[Detect installed modules<br/>and their workflows]
    C --> D[Rank next steps<br/>required first, then optional]
    D --> E[Output: prioritized list<br/>each with skill command]

Prompt examples

bmad-help

bmad-help I have a SaaS idea and know all the features. Where do I start?

bmad-help What are my options for UX design?

bmad-help I inherited a Django monolith with no docs. What's my entry point?

Expected output

A prioritized list, something like:

Detected: no PRD, no architecture, no stories. BMM + CIS installed.

REQUIRED NEXT
  1. bmad-prd        — You have a clear feature set; go straight to
                       Create mode. Produces prd.md + addendum.md.

OPTIONAL — worth considering first
  2. bmad-product-brief — If the strategic framing isn't settled,
                          this feeds prd.md and reduces re-explanation.
  3. bmad-deep-recon    — If competitive positioning is uncertain.

SKIP FOR NOW
  - bmad-brainstorming  — You said you know the features.

Why it matters more than the phase list: bmad-help reads the actual files on disk. Following a static sequence blindly produces documents you don't need. bmad-help evolves as you install modules, so it also surfaces third-party module capabilities the docs don't cover.

3.2 bmad-advanced-elicitation

Push the LLM to reconsider, refine, and improve its recent output. BMad's shared refinement checkpoint — other skills invoke it at natural pauses, and you can call it directly.

How it works

flowchart TD
    A[Target the most recent output<br/>unless you point it elsewhere] --> B[Offer a short menu of<br/>best-fit elicitation methods]
    B --> C{You choose<br/>one or more}
    C --> D[Apply methods against the target]
    D --> E[Hand back the improved version]
    E --> F[Invoking flow resumes<br/>where it paused]

Prompt examples

bmad-advanced-elicitation

bmad-advanced-elicitation run a pre-mortem on that architecture section

bmad-advanced-elicitation red team the failure modes in the retry logic above

bmad-advanced-elicitation apply first principles to the caching assumption

Named methods you can request directly: Socratic, first principles, pre-mortem, red team.

Expected output

A method menu, then the enhanced content:

Best-fit methods for this architecture section:
  1. Pre-mortem        — assume it failed in prod; work backwards
  2. Red team          — attack the trust boundaries
  3. First principles  — rebuild the caching decision from constraints
  4. Socratic          — interrogate the unstated assumptions

> 1

PRE-MORTEM — assuming this shipped and failed within 90 days:
  ...
REVISED SECTION:
  ...

Use it when: output feels shallow or generic, you're finalizing something critical, or you want a specific analytical lens by name.

3.3 bmad-review

Multi-lens review over any diff, doc, or artifact. Each lens is a distinct method and stance. Every finding reports in one canonical shape. Zero findings is a valid outcome — it never pads to look thorough.

The shipped lenses

The two editorial lenses hold your content sacrosanct: they never challenge your ideas, only how they're organized and expressed, and they propose rather than execute. Prose runs on top of structure findings when both are selected.

How it works

flowchart TD
    A[Load content] --> B[Identify type: diff / file /<br/>function / document — code or docs]
    B --> C{Lenses specified?}
    C -->|Yes| D[Run named lenses]
    C -->|No| E[Every enabled lens whose<br/>applicability + conditions fit]
    D --> F[Announce the plan]
    E --> F
    F --> G[Run independent lenses in parallel<br/>via subagents where supported]
    G --> H[Run dependent lenses on top<br/>e.g. prose over structure]
    H --> I[Assemble one findings array<br/>overlap = signal, not duplication]

Inputs

Prompt examples

bmad-review my uncommitted changes

bmad-review the diff on branch feat/hnsw-tuning, verification gap lens only

bmad-review docs/architecture.md with structure and prose, reader_type llm

bmad-review src/reranker.py — also_consider: multilingual tokenization edge cases

Expected output

JSON findings array where each finding carries lens, location, trigger_condition, guard_snippet, potential_consequence — and/or a markdown report grouped by lens. Editorial lenses render a findings table you accept or reject row by row, plus an estimated reduction when structural changes are proposed.

[
  {
    "lens": "verification-gap",
    "location": "src/reranker.py:88-104",
    "trigger_condition": "Query locale differs from index locale and fallback path executes",
    "guard_snippet": "assert scores.shape[0] == len(candidates)",
    "potential_consequence": "Silent truncation of the candidate set; recall drop invisible to existing tests"
  }
]

Note: you rarely invoke this manually in a full flow. Code review workflows in other modules run the code lenses automatically, and the document workflows (PRD, UX, architecture, product brief) run the editorial lenses as their finalize step. Custom lenses can be added — and shipped ones tuned or disabled — through the skill's customize.toml.

3.4 bmad-customize

Create and verify customization overrides. Change how an installed agent or workflow behaves without hand-authoring TOML.

How it works

flowchart LR
    A[Natural language<br/>description of change] --> B[Scan installed BMad skills<br/>for customizable surfaces]
    B --> C[Select the right override scope]
    C --> D["Write override files<br/>under <code>_bmad/custom/</code>"]
    D --> E[Verify merged configuration]

Prompt examples

bmad-customize make the dev agent always run our lint config before declaring a story done

bmad-customize add a persistent fact: our OpenSearch cluster is 2.13, no k-NN nmslib engine

bmad-customize disable the prose lens in bmad-review for this project

bmad-customize add a menu item to the architect agent for ADR generation

Expected output

TOML override files under _bmad/custom/, plus a verification pass on the merged config.

What you can override: persistent facts, activation hooks, custom menu items, agent memory/prompts (via *.customize.yaml merging), review lenses, and any module config option.

3.5 bmad-brainstorming

Generate diverse ideas through interactive creative techniques. A facilitated session that loads proven ideation methods from a technique library.

How it works

flowchart TD
    A[Topic or problem statement<br/>+ optional context file] --> B[Set up session]
    B --> C[Load technique from method library<br/>SCAMPER, reverse brainstorming, etc.]
    C --> D[Generate ideas]
    D --> E{10 ideas since<br/>last shift?}
    E -->|Yes| F["Anti-bias protocol:<br/>shift creative domain"]
    F --> C
    E -->|No| G{100+ ideas?}
    G -->|No| D
    G -->|Yes| H[Organize by technique]
    H --> I["brainstorm.html keepsake<br/>+ optional brainstorm-intent.md<br/>+ .memlog.md"]

Prompt examples

bmad-brainstorming

bmad-brainstorming ways to cut cold-start latency in a multilingual recommender

bmad-brainstorming onboarding for a legal research tool — context: docs/personas.md

Expected output

• brainstorm.html — self-contained keepsake of the session
• brainstorm-intent.md — optional, for downstream skills to consume
• .memlog.md — session record

The quantity target is the point. The workflow pushes for 100+ ideas because, in BMAD's framing, the useful material appears in ideas 50–100 — after you've exhausted the obvious. The anti-bias protocol exists to stop you clustering in one creative domain.

3.6 bmad-deep-recon

Decision-grade research on any subject, three ways. The only core skill that goes outside the model's memory.

Absorbed the former bmad-market-research, bmad-domain-research, and bmad-technical-research workflows as research types. Old IDs still forward.

The three modes

flowchart TD
    A[Your ask] --> B[Detect mode + infer research type]
    B --> C{Mode}

    C -->|DRAFT| D["Write a deep-research prompt<br/>for your own tool<br/>(ChatGPT DR, Gemini DR, Perplexity)"]
    D --> D2[You run it there<br/>on your flat-rate subscription]
    D2 --> E

    C -->|PROCESS| E["Ingest a finished report<br/>→ cited summary downstream<br/>skills consume directly"]

    C -->|RUN| F[Plan at one gate]
    F --> G[Fan out firewalled<br/>research assistants in parallel]
    G --> H[Verify claims as they land]
    H --> I

    E --> I["research.md with metadata frontmatter<br/>+ optional self-contained HTML briefing"]

    I --> J{Existing report?}
    J -->|REFRESH| K[Update without re-researching]
    J -->|DEEPEN| K

Six typed research packs. Each loads its own prioritized dimensions, source craft, and freshness rules: market, domain, technical, competitive, user-voice, literature.

Prompt examples

bmad-deep-recon

bmad-deep-recon draft mode — HNSW vs IVF-PQ tradeoffs for 50M multilingual
embeddings under a 40ms p99 budget

bmad-deep-recon process this report: ~/Downloads/gemini-dr-vector-db.md

bmad-deep-recon run — competitive research on legal research AI tools,
decision: whether to build or license citation extraction

bmad-deep-recon refresh docs/research/vector-db-2026-05/

Expected output

• research.md with metadata frontmatter and inline citations
• Optional self-contained HTML briefing

Mode selection — cost and fidelity tradeoff:

Use it when: a decision should rest on evidence instead of the model's memory, or you're choosing between named options and want a structured comparison.

The phrase "firewalled research assistants" in the Run-mode description refers to the research firewall — the trust mechanism that keeps your own project documents out of the evidence chain. See Appendix A.

3.7 bmad-forge-idea

Pressure-test an idea until it hardens, proves out, or dies cheaply. An adversarial interrogator drives a half-formed idea one question at a time.

How it works

flowchart TD
    A[The idea — any domain] --> B[Establish the goal up front<br/>steer questioning to match]
    B --> C[One question at a time<br/>in dependency order]
    C --> D[Put a recommended answer<br/>on the table to push against]
    D --> E[Two voices per branch:<br/>one from your installed roster,<br/>one conjured by the topic]
    E --> F[Challenge fuzzy terms<br/>test claims against project material]
    F --> G{More branches?}
    G -->|Yes| C
    G -->|No| H{Verdict}
    H -->|HARDENED| I["forged-idea.md distillate<br/>+ forge-report.html"]
    H -->|KILLED| J[forge-report.html]
    H -->|CLEARER| J

Prompt examples

bmad-forge-idea

bmad-forge-idea we should replace our BM25 fallback entirely with dense retrieval

bmad-forge-idea building an internal eval harness rather than adopting one —
goal: decide in one session, I'm biased toward building

bmad-forge-idea leaving my job to do this full time

Expected output

• forge-report.html — self-contained keepsake, every run
• forged-idea.md — distillate, only when the idea hardens (and optional even then)

Three landing states: Hardened, Killed, or Clearer.

The design intent is that killing is a success. Stating your bias up front ("I'm biased toward building") gives the interrogator something specific to attack. The two-voice mechanic — one persona from your installed agent roster, one conjured by the topic itself — is what keeps it from collapsing into agreement.

3.8 bmad-party-mode

Orchestrate multi-agent group discussions. Loads all installed BMad agents and facilitates a conversation where each contributes from their expertise and personality.

How it works

flowchart TD
    A[Topic or question<br/>+ optional persona list] --> B[Load agent manifest<br/>all installed personalities]
    B --> C[Analyze topic → select<br/>2-3 most relevant agents]
    C --> D[Agents take turns<br/>natural cross-talk and disagreement]
    D --> E[Rotate participation for<br/>diverse perspectives over time]
    E --> F{"goodbye / end party / quit"}
    F -->|No| D
    F -->|Yes| G[Session ends —<br/>nothing persisted]

Prompt examples

bmad-party-mode

bmad-party-mode should we ship the reranker behind a feature flag or
gate it on the A/B result?

bmad-party-mode our search latency SLO — I want Winston and Amelia specifically

Expected output

Real-time multi-agent conversation with maintained personalities. No artifact is written. Exit with goodbye, end party, or quit.

Use it when: you need multiple expert perspectives, want agents to challenge each other's assumptions, or you're exploring something spanning multiple domains. It is exploratory and pre-artifact — if you want findings you can act on, use bmad-review or bmad-forge-idea instead.

3.9 Choosing Between the Four Thinking Skills

The four are easy to confuse. They differ on where the input comes from and what shape the output takes.

flowchart TD
    Q{What's the bottleneck?}
    Q -->|"I don't know the facts"| DR["<b>bmad-deep-recon</b><br/>external evidence<br/>→ cited research.md"]
    Q -->|"I have no options"| BS["<b>bmad-brainstorming</b><br/>divergence, many→<br/>→ brainstorm.html"]
    Q -->|"I have one idea and<br/>I'm too attached to it"| FI["<b>bmad-forge-idea</b><br/>convergence, →one<br/>→ verdict + report"]
    Q -->|"I'm only seeing this<br/>from one angle"| PM["<b>bmad-party-mode</b><br/>breadth of perspective<br/>→ no artifact"]

The key distinctions:

• Deep-recon vs. the other three — facts vs. thinking. Deep-recon is the only one that can tell you something you didn't already know.
• Brainstorming vs. forge-idea — opposite directions on the same axis. Brainstorming expands, forge contracts.
• Forge-idea vs. party-mode — forge is adversarial and terminal (produces a verdict). Party is exploratory and unstructured (produces discussion). Forge also brings a voice conjured by the topic, not just your installed roster.
• Party-mode vs. bmad-review — party is speculative and pre-artifact. Review runs structured lenses over something you've already written.

They compose. A realistic sequence: bmad-brainstorming (get options) → bmad-deep-recon (check which are real) → bmad-forge-idea (kill the weak survivors) → bmad-product-brief (write up what's left).

4. BMM Agents

The default BMM (Agile suite) agents installed with BMad Method. Each agent is available as a skill, generated by the installer. The skill ID invokes the agent; triggers are the short menu codes and fuzzy matches shown in each agent's menu.

Trigger decode:

How a trigger session looks

sequenceDiagram
    participant You
    participant Skill as bmad-agent-pm
    participant John as John (persona)
    participant WF as Workflow file

    You->>Skill: bmad-agent-pm
    Skill->>John: Load persona + activate menu
    John->>You: Menu: PRD, CE, IR, CC
    You->>John: PRD
    John->>WF: Load PRD workflow config
    WF->>John: Steps + prompts
    John->>You: Step 1 - in character, asking for input
    You->>John: (answers)
    Note over John,WF: ...steps continue...
    John->>You: prd.md written. Next: CE?
    You->>John: CE
    Note over John: Same session, no context loss

Prompt examples

bmad-agent-pm

bmad-agent-dev

Then within the session:

BD

CR

Notes

• QA test generation is handled by the bmad-qa-generate-e2e-tests workflow skill, available through the Developer agent. The full Test Architect (TEA) lives in its own module.
• The Technical Writer (Paige) is on hiatus - she will return more capable. Project documentation lives on via the DP (Document Project) trigger through the Analyst, or by invoking bmad-document-project directly.
• Optional modules (BMB, CIS, TEA, Game Dev Studio) add their own agents and skills only if selected at install.

5. Phase 1 - Analysis

Explore the problem space and validate ideas before committing to planning. All Phase 1 steps are optional. They feed context into the PRD.

flowchart LR
    A["Vague idea"] --> BS[bmad-brainstorming]
    BS --> FI[bmad-forge-idea]
    A --> FI
    FI -->|Killed| X["Stop - you saved<br/>weeks of work"]
    FI -->|Hardened / Clearer| DR[bmad-deep-recon]
    A --> DR
    DR --> PB[bmad-product-brief]
    PB --> PF[bmad-prfaq]
    PB --> P2["to Phase 2"]
    PF --> P2

bmad-product-brief

Prompt examples

bmad-product-brief

bmad-product-brief a cross-language document retrieval layer for legal
research - source it from docs/research/multilingual-2026-07/research.md

Expected output

_bmad-output/
├── brief.md          <- strategic vision, problem, audience, success
├── addendum.md       <- supporting detail that would bloat the brief
└── .memlog.md        <- session record

Plus any HTML or presentation hydration you request.

Downstream link: bmad-prd can source-extract from product-brief.md during Discovery, which reduces re-explanation and keeps the two documents aligned. Neither skill requires the other - start with bmad-prd directly if you already know what you're building.

bmad-prfaq

Working Backwards - the press-release-first exercise. Stress-tests the concept from the customer's side before you've built anything.

Prompt examples

bmad-prfaq

bmad-prfaq write the launch announcement for the multilingual search feature
as if it shipped, then interrogate the gap

Expected output: prfaq-{project}.md

Also available as the WB trigger on the Analyst agent.

6. Phase 2 - Planning

Define what to build and for whom.

flowchart TD
    A["brief.md / prfaq / raw intent"] --> PRD[bmad-prd]
    PRD --> I{Which intent?}
    I -->|Create| C["prd.md + addendum.md + .memlog.md<br/>coached discovery from scratch"]
    I -->|Update| U["Reconcile with a change signal<br/>surfacing conflicts before applying"]
    I -->|Validate| V["validation-report.html + .md<br/>critique against configurable checklist"]
    C --> UX[bmad-ux]
    C --> SP[bmad-spec]
    U --> SP
    UX --> SP
    SP --> S["SPEC.md + companions<br/>optional stories.yaml"]
    S --> P3["to Phase 3"]

bmad-prd - three intents in one skill

State your intent when invoking, or the skill will ask.

Prompt examples

bmad-prd

bmad-prd create - cross-language retrieval for legal research.
Source from _bmad-output/brief.md

bmad-prd update - legal changed the data residency requirement to
EU-only storage for EU tenants

bmad-prd validate

Expected output by intent

The Update intent surfacing conflicts before applying is the part worth knowing - it's what stops requirement drift from silently overwriting earlier decisions.

The PRD workflow also includes steps for vision/differentiators and an executive summary.

bmad-ux

Produces a spine pair: DESIGN.md (visual) and EXPERIENCE.md (behavioral). Two files because visual decisions and behavioral decisions change at different rates and have different audiences.

Prompt examples

bmad-ux

bmad-ux the search results surface - cross-language results need to signal
which language the source doc is in without cluttering

Skip this for backend-only or infrastructure work.

bmad-spec

The canonical machine contract. A five-field kernel plus companion files, validated so every load-bearing source claim is preserved.

flowchart LR
    subgraph SPEC["SPEC.md - five-field kernel"]
        W["<b>Why</b><br/>the reason this exists"]
        C["<b>Capabilities</b><br/>what it must do"]
        K["<b>Constraints</b><br/>what bounds it"]
        N["<b>Non-goals</b><br/>what it explicitly won't do"]
        S["<b>Success signal</b><br/>how you know it worked"]
    end
    SPEC --> COMP["Companion files under<br/>specs/spec-{slug}/"]
    SPEC -.->|on request| ST["stories.yaml<br/>ordered, for autonomous dispatch"]

Accepts any intent input: a brief, a PRD, a meeting transcript, a brain dump, a design folder.

Prompt examples

bmad-spec

bmad-spec distill _bmad-output/prd.md into a spec

bmad-spec here's a transcript of yesterday's planning call - notes/2026-07-30.md

bmad-spec and break it into stories.yaml for autonomous dispatch

Expected output

_bmad-output/specs/spec-multilingual-retrieval/
├── SPEC.md          <- the five-field kernel
├── <companions>     <- preserved load-bearing source claims
└── stories.yaml     <- optional, ordered, for bmad-build-auto

Two things that make bmad-spec structurally important:

1. It is the only writer of SPEC.md. Other skills invoke it headless when they need to express or update intent - so the contract stays single-sourced.
2. It locks the WHAT before the HOW. Everything downstream reads from it.

Note: bmad-spec used to live in the core module. It now ships with BMM as a Phase 2 planning workflow.

7. Phase 3 - Solutioning

Decide how to build it and break work into stories.

flowchart TD
    A["SPEC.md / prd.md / DESIGN.md"] --> AR[bmad-architecture]
    AR --> SPINE["ARCHITECTURE-SPINE.md<br/>(hydrates to other outputs on request)"]
    SPINE --> PC[bmad-generate-project-context]
    PC --> CTX["project-context.md<br/>the project constitution"]
    CTX --> ES[bmad-create-epics-and-stories]
    ES --> EP["Epic files with stories"]
    EP --> IR[bmad-check-implementation-readiness]
    IR --> D{Gate}
    D -->|PASS| P4["to Phase 4"]
    D -->|CONCERNS| P4
    D -->|FAIL| BACK["Back to the gap -<br/>usually PRD or architecture"]
    BACK --> AR

bmad-architecture

Prompt examples

bmad-architecture

bmad-architecture - constraints: OpenSearch 2.13, no new infra, p99 under 40ms,
must degrade to BM25 on embedding service failure

Expected output: ARCHITECTURE-SPINE.md. The spine is the durable form; hydrate it into a full document or a presentation when you need to communicate it outward.

bmad-generate-project-context - the highest-leverage step

project-context.md works like a constitution for your project. It guides implementation decisions across all workflows and is automatically loaded by implementation workflows.

Two ways to get it:

• Manually - create _bmad-output/project-context.md with your technology stack and implementation rules
• Generate it - run bmad-generate-project-context to derive it from your architecture or existing codebase

flowchart LR
    A[Architecture doc] --> G[bmad-generate-project-context]
    B[Existing codebase] --> G
    G --> C["_bmad-output/project-context.md"]
    C -.->|auto-loaded| D[bmad-build]
    C -.->|auto-loaded| E[bmad-code-review]
    C -.->|auto-loaded| F[bmad-build-auto]

Prompt examples

bmad-generate-project-context

bmad-generate-project-context - derive from the existing codebase, we have no
architecture doc. Emphasize our error handling and logging conventions.

This is the single best-value step for an existing codebase. It's what stops agents re-litigating your conventions on every story.

bmad-create-epics-and-stories

Prompt examples

bmad-create-epics-and-stories

bmad-create-epics-and-stories from SPEC.md - keep epics under 5 stories each,
we ship weekly

Also: the CE trigger on the PM agent.

bmad-check-implementation-readiness

A genuine gate, not a formality. Returns PASS, CONCERNS, or FAIL.

bmad-check-implementation-readiness

Available on both the PM agent (IR) and the Architect agent (IR) - deliberately, since readiness failures usually trace to either a requirements gap or a design gap.

8. Phase 4 - Implementation

Every implementation path converges on bmad-build. It accepts direct intent, an issue, a specification, or a planned story, then chooses the clarification, planning, implementation, and review depth needed for that input.

flowchart TD
    subgraph IN["Inputs - all valid"]
        I1[Direct intent]
        I2[An issue<br/>JIRA-4412]
        I3[A specification<br/>SPEC.md]
        I4[A planned story<br/>from epic files]
    end

    IN --> BD["<b>bmad-build</b>"]
    BD --> CL[Clarify: ask for the<br/>choices it needs]
    CL --> PL[Plan: you approve or change]
    PL --> IM[Implement]
    IM --> RV[Review its own work<br/>runs bmad-review code lenses]
    RV --> OUT["spec-*.md + working code<br/>you can run and inspect"]

    CTX["project-context.md"] -.->|auto-loaded| BD

bmad-build - the convergence point

You keep control of the decisions that shape the result. Build asks for the choices it needs, then gives you a plan to approve or change, implements, reviews its own work, and shows you the result.

Prompt examples

bmad-build

bmad-build add exponential backoff with jitter to the embedding service client,
max 3 retries, fall through to BM25 on exhaustion

bmad-build story 2.3

bmad-build JIRA-4412

bmad-build from _bmad-output/specs/spec-multilingual-retrieval/SPEC.md

Expected output

• A spec-*.md capturing what it decided to build and why
• Working code you can run and inspect
• Review findings from the code lenses it ran on itself

The depth adapts to the input. A one-line intent gets light clarification and a short plan. A full planned story with architecture and project-context behind it gets a deeper plan and heavier review. Feeding it more context doesn't route you to a different workflow - it changes how bmad-build behaves.

bmad-build-auto - unattended loops

flowchart TD
    A["stories.yaml<br/>(from bmad-spec)"] --> B[bmad-build-auto]
    B --> C[Pick next story in order]
    C --> D[Run the bmad-build model<br/>unattended]
    D --> E[Review]
    E --> F{Pass?}
    F -->|Yes| G[Mark done, log]
    F -->|No| H[Log the failure]
    G --> I{More stories?}
    H --> I
    I -->|Yes| C
    I -->|No| J[Stop - report]

Prompt examples

bmad-build-auto

bmad-build-auto - run through stories.yaml, stop on the first review failure

Prerequisite: an ordered stories.yaml, which bmad-spec produces on request.

Judgment call: unattended loops amplify whatever your project-context.md says. Run it only after the constitution is accurate - otherwise you get several stories' worth of consistently wrong conventions.

bmad-sprint-planning

Run once per project to initialize tracking. Produces sprint-status.yaml.

bmad-sprint-planning

bmad-sprint-status

bmad-sprint-status

Returns story-by-story progress against sprint-status.yaml.

bmad-code-review

Ad hoc review of any code change - separate from the review bmad-build runs on itself. Produces findings + applied patches.

Prompt examples

bmad-code-review

bmad-code-review the diff against main

bmad-code-review PR 218 - focus on the concurrency changes

Reach for this when: reviewing code BMAD didn't write, or when bmad-build's built-in review depth wasn't enough for a risky change.

bmad-correct-course

For significant mid-sprint changes - a requirement shifted, an assumption broke, a dependency vanished. Produces an updated plan or re-routes you to an earlier phase.

Prompt examples

bmad-correct-course

bmad-correct-course - the embedding vendor deprecated the multilingual model
we planned on, effective in 60 days

Also: the CC trigger on the PM agent.

bmad-retrospective

Run after epic completion. Produces lessons learned.

bmad-retrospective

Also: the ER (Epic Retrospective) trigger on the Developer agent.

bmad-qa-generate-e2e-tests

Available through the Developer agent (QA) or directly.

bmad-qa-generate-e2e-tests

The full Test Architect (TEA) is a separate module with much deeper test architecture capability - install it if testing is a first-class concern rather than a step.

9. The Four Entry Paths

There is no single correct order. Pick the path by the shape of the work.

flowchart TD
    START{What are you doing?}

    START -->|"New product, nothing exists"| FULL
    START -->|"Small, well-understood change"| QUICK
    START -->|"Existing codebase, no BMAD artifacts"| BROWN
    START -->|"Batch of defined work, want it unattended"| AUTO

    subgraph FULL["FULL METHOD"]
        F1[bmad-brainstorming] --> F2[bmad-deep-recon]
        F2 --> F3[bmad-forge-idea]
        F3 --> F4[bmad-product-brief]
        F4 --> F5[bmad-prfaq]
        F5 --> F6[bmad-prd]
        F6 --> F7[bmad-ux]
        F7 --> F8[bmad-spec]
        F8 --> F9[bmad-architecture]
        F9 --> F10[bmad-generate-project-context]
        F10 --> F11[bmad-create-epics-and-stories]
        F11 --> F12[bmad-check-implementation-readiness]
        F12 --> F13[bmad-sprint-planning]
        F13 --> F14[bmad-build loop]
        F14 --> F15[bmad-retrospective]
    end

    subgraph QUICK["QUICK PATH"]
        Q1["bmad-build<br/>(that's it)"]
    end

    subgraph BROWN["EXISTING CODEBASE"]
        B1[bmad-document-project] --> B2[bmad-generate-project-context]
        B2 --> B3{Scope?}
        B3 -->|Small| B4[bmad-build]
        B3 -->|Large| B5[rejoin at bmad-prd]
    end

    subgraph AUTO["AUTONOMOUS"]
        A1[bmad-spec] --> A2["stories.yaml"]
        A2 --> A3[bmad-build-auto]
    end

Path 1 - Full Method (greenfield, Level 3-4)

Interleave freely: bmad-code-review for risky changes, bmad-sprint-status between stories, bmad-correct-course when something shifts.

Path 2 - Quick Path (well-scoped change)

bmad-build <your intent>

That is the whole path. bmad-build accepts raw intent and picks its own clarification, planning, implementation, and review depth. Skipping steps 1-13 is the intended behavior, not a compromise.

Path 3 - Existing Codebase

flowchart LR
    A[bmad-document-project<br/>or DP via Analyst] --> B[bmad-generate-project-context]
    B --> C{Scope of the work?}
    C -->|"Bug fix, small feature"| D[bmad-build]
    C -->|"New subsystem"| E[bmad-prd] --> F[bmad-architecture] --> G[...]

The two-step front-load (bmad-document-project then bmad-generate-project-context) is what makes every later BMAD invocation aware of your conventions. Do it once.

Path 4 - Autonomous

bmad-spec ... and break it into stories.yaml
bmad-build-auto

Only after project-context.md is accurate.

Three things that matter more than the order

1. Run bmad-help between phases. It inspects actual artifacts on disk and tells you what's genuinely next. That beats following any static list, including this one.
2. bmad-advanced-elicitation isn't a step. Call it any time an output feels thin.
3. bmad-review isn't a step either. Document workflows already invoke its editorial lenses as their finalize step, and code workflows run the code lenses automatically.

10. Artifact Flow

Every document becomes context for the next phase. Without this structure, agents make inconsistent decisions.

flowchart TD
    R["research.md<br/><i>bmad-deep-recon</i>"] --> BR["brief.md + addendum.md<br/><i>bmad-product-brief</i>"]
    BI["brainstorm-intent.md<br/><i>bmad-brainstorming</i>"] --> BR
    FG["forged-idea.md<br/><i>bmad-forge-idea</i>"] --> BR

    BR -->|"source-extracted<br/>during Discovery"| PRD["prd.md + addendum.md<br/><i>bmad-prd</i>"]
    PF["prfaq-{project}.md<br/><i>bmad-prfaq</i>"] --> PRD

    PRD --> UX["DESIGN.md + EXPERIENCE.md<br/><i>bmad-ux</i>"]
    PRD --> SPEC["SPEC.md + companions<br/><i>bmad-spec</i>"]
    UX --> SPEC

    SPEC --> ARCH["ARCHITECTURE-SPINE.md<br/><i>bmad-architecture</i>"]
    PRD --> ARCH

    ARCH --> PCTX["project-context.md<br/><i>bmad-generate-project-context</i>"]
    CODE["existing codebase"] --> PCTX

    ARCH --> EPICS["epic files with stories<br/><i>bmad-create-epics-and-stories</i>"]
    SPEC -.->|"optional"| SY["stories.yaml"]

    EPICS --> RDY["PASS / CONCERNS / FAIL<br/><i>bmad-check-implementation-readiness</i>"]
    RDY --> SST["sprint-status.yaml<br/><i>bmad-sprint-planning</i>"]

    SST --> BUILD["spec-*.md + code<br/><i>bmad-build</i>"]
    PCTX -.->|auto-loaded| BUILD
    SY --> BAUTO["<i>bmad-build-auto</i>"] --> BUILD

    BUILD --> RETRO["lessons learned<br/><i>bmad-retrospective</i>"]

Reading the diagram: solid arrows are the documented feeds. Dotted arrows are automatic or optional. The important observation is that project-context.md feeds sideways into implementation rather than flowing down the chain - it's a constraint layer, not a stage.

Default output location: _bmad-output/. Configurable via --set bmm.output_folder=....

11. Worked Example - Multilingual Search Feature

A realistic end-to-end run for an existing production search system, showing which steps earn their keep and which don't.

Context: existing OpenSearch-backed search service. Task: add cross-language retrieval so an English query surfaces relevant French and German documents. Team of three. Codebase exists; no BMAD artifacts yet.

flowchart TD
    S1["<b>1.</b> bmad-document-project<br/><i>one-time, existing codebase</i>"] --> S2
    S2["<b>2.</b> bmad-generate-project-context<br/><i>the constitution</i>"] --> S3
    S3["<b>3.</b> bmad-deep-recon (draft mode)<br/><i>embedding model choice needs evidence</i>"] --> S4
    S4["<b>4.</b> bmad-forge-idea<br/><i>challenge 'dense retrieval replaces BM25'</i>"] --> S5
    S5["<b>5.</b> bmad-spec<br/><i>lock the WHAT</i>"] --> S6
    S6["<b>6.</b> bmad-architecture<br/><i>the fallback path is a real decision</i>"] --> S7
    S7["<b>7.</b> bmad-create-epics-and-stories"] --> S8
    S8["<b>8.</b> bmad-sprint-planning"] --> S9
    S9["<b>9.</b> bmad-build loop"] --> S10
    S10["<b>10.</b> bmad-code-review<br/><i>on the reranker change only</i>"] --> S11
    S11["<b>11.</b> bmad-retrospective"]

    SK["SKIPPED:<br/>brainstorming - direction is known<br/>product-brief / prfaq - internal feature<br/>prd - spec is sufficient at this scope<br/>ux - backend only"]

    style SK fill:none,stroke-dasharray: 5 5

Step-by-step

1. Document the existing project

bmad-document-project

Output: a structured read of the current architecture, entry points, and conventions.

2. Generate the constitution

bmad-generate-project-context - derive from the codebase. Emphasize:
our OpenSearch client wrapper is the only allowed query path, all latency-
sensitive code paths must have a p99 assertion in tests, no new infra.

Output: _bmad-output/project-context.md, auto-loaded by every later implementation workflow.

3. Research the one thing you don't know

bmad-deep-recon draft mode - multilingual embedding models for legal-domain
retrieval at 50M docs. Decision: which model, and whether to index per-language
or into a shared space. Constraints: 40ms p99, OpenSearch 2.13 k-NN.

Output: a deep-research prompt you paste into your flat-rate subscription. Then:

bmad-deep-recon process ~/Downloads/dr-multilingual-embeddings.md

Output: research.md, cited, in the shape downstream skills consume.

Why draft+process rather than run: the fan-out in run mode burns metered IDE tokens on work a flat-rate subscription does equally well. Run mode earns its cost when you need verification-as-claims-land inside the IDE loop.

4. Kill the bad version of the idea

bmad-forge-idea we should replace BM25 entirely with dense retrieval -
goal: decide this session. I'm biased toward the clean architecture.

Output: forge-report.html. Likely verdict: Clearer - hybrid, with BM25 as the fallback path, because exact-match legal citation lookup degrades badly under pure dense retrieval.

Stating your bias up front is what makes this useful. The interrogator attacks the specific thing you're attached to.

5. Lock the WHAT

bmad-spec - from research.md and the forge report. Multilingual retrieval
with BM25 fallback.

Output: _bmad-output/specs/spec-multilingual-retrieval/SPEC.md with the five-field kernel. Non-goals matter most here - explicitly excluding query translation, for instance, prevents scope creep three stories in.

6. Architecture, because the fallback is a real decision

bmad-architecture - constraints from SPEC.md plus: must degrade to BM25 on
embedding service failure without a user-visible error, and the degradation
must be observable in metrics.

Output: ARCHITECTURE-SPINE.md.

7-8. Break down and initialize

bmad-create-epics-and-stories from SPEC.md - epics under 5 stories, weekly ship cadence

bmad-sprint-planning

9. Build loop

bmad-build story 1.1

bmad-build story 1.2

Between stories:

bmad-sprint-status

10. Extra review where risk concentrates

bmad-code-review the reranker diff - verification gap lens especially,
also_consider: behavior when query locale and index locale disagree

11. Close the epic

bmad-retrospective

What got skipped and why

12. Deprecations, Renames, and Gotchas

Merged into bmad-review (old IDs forward)

• bmad-editorial-review
• bmad-editorial-review-prose
• bmad-editorial-review-structure
• bmad-review-adversarial-general
• bmad-review-edge-case-hunter
• bmad-review-verification-gap

The editorial lenses inside bmad-review replace the separate editorial skill entirely.

Merged into bmad-deep-recon (old IDs forward)

• bmad-market-research
• bmad-domain-research
• bmad-technical-research

These are now research types selected by the pack loader, not separate workflows.

Removed outright

• bmad-shard-doc
• bmad-index-docs

Moved

• bmad-spec - core module to BMM Phase 2

Auto-removed on upgrade

Pre-v6.2.0 wrapper skills bmad-bmm-* and bmad-agent-bmm-* are deleted automatically on upgrade so they stop erroring with missing-file warnings.

dev vs. build naming

The docs are internally inconsistent on this. Some pages reference bmad-dev / bmad-dev-auto and a "Quick Dev" explanation page; others reference bmad-build / bmad-build-auto and a "Build" page. The current reference material uses build. Check which your installed version actually generated - ls .claude/skills/ | grep -E 'build|dev' settles it in one command.

Gotchas worth knowing

Cost lever - Web Bundles

Web bundles package selected BMad skills as Google Gemini Gems and ChatGPT Custom GPTs. Do the upfront planning work - brainstorming, product briefs, PRDs, PRFAQs, UX specs, market and industry research - in your web LLM subscription, then bring the polished artifacts into the IDE for implementation.

Planning runs on a flat-rate subscription instead of metered IDE tokens. On a long engagement that is a meaningful saving, and the artifacts are identical.

Current shelf: brainstorming, product brief, PRFAQ, PRD, UX, market and industry research. Available at bmadcode.com/web-bundles.

13. Cheat Sheet

Everything, one place

Core (always installed)

bmad-help                      what do I do next
bmad-advanced-elicitation      refine recent output
bmad-review                    multi-lens review
bmad-customize                 TOML overrides
bmad-brainstorming             divergent ideation
bmad-deep-recon                external evidence
bmad-forge-idea                adversarial pressure test
bmad-party-mode                multi-agent discussion

BMM agents

bmad-agent-analyst             Mary       BP MR DR TR CB WB DP
bmad-agent-pm                  John       PRD CE IR CC
bmad-agent-architect           Winston    CA IR
bmad-agent-dev                 Amelia     BD QA CR SP ER
bmad-agent-ux-designer         Sally      CU

Phase 1 - Analysis (optional)

bmad-brainstorming             brainstorm.html
bmad-forge-idea                forge-report.html [+ forged-idea.md]
bmad-deep-recon                research.md [+ HTML briefing]
bmad-product-brief             brief.md + addendum.md
bmad-prfaq                     prfaq-{project}.md

Phase 2 - Planning

bmad-prd                       prd.md + addendum.md | validation-report.html
bmad-ux                        DESIGN.md + EXPERIENCE.md
bmad-spec                      SPEC.md + companions [+ stories.yaml]

Phase 3 - Solutioning

bmad-architecture              ARCHITECTURE-SPINE.md
bmad-generate-project-context  project-context.md
bmad-create-epics-and-stories  epic files
bmad-check-implementation-readiness   PASS / CONCERNS / FAIL

Phase 4 - Implementation

bmad-build                     spec-*.md + code
bmad-build-auto                unattended iterations
bmad-sprint-planning           sprint-status.yaml
bmad-sprint-status             progress
bmad-code-review               findings + patches
bmad-correct-course            updated plan
bmad-retrospective             lessons learned

Utility

bmad-document-project          structured read of an existing codebase
bmad-qa-generate-e2e-tests     e2e test generation

Decision table

Verify your own install

ls .claude/skills/ | grep bmad          # canonical list
ls .claude/skills/ | grep -E 'build|dev' # settle the naming question
cat .claude/skills/bmad-build/SKILL.md   # read what a skill actually does
npx bmad-method install --list-options bmm

Compiled from the BMAD-METHOD v6 documentation. Where this guide states an opinion - which steps to skip, when draft mode beats run mode, sequencing judgment - that is analysis, not doctrine. Your installed version is the authority on what exists; bmad-help is the authority on what's next.

Appendix A - The Research Firewall

The single most consequential design decision in bmad-deep-recon, and the one most likely to be misread. "Firewall" here has nothing to do with network security. It is an epistemic boundary: a rule about what may count as evidence.

A.1 The Rule, Verbatim

From src/core-skills/bmad-deep-recon/SKILL.md, stated as one of two standing rules inherited verbatim by every subagent:

1. Never conclude from training data alone. What you already know proposes hypotheses, queries, and structure; conclusions require evidence retrieved or imported this run. A claim you cannot evidence is stated as an unverified belief or not at all.

2. The research firewall. Project context - briefs, PRDs, code, memory, {workflow.persistent_facts} - shapes what to ask, never what is true. It is inadmissible as evidence: every claim in a research artifact traces to a digest or import file with a source. Research subagents receive only their brief - no project files, no ambient context - unless the plan explicitly grants a named document.

One sentence: your project decides the questions; it is never allowed to be an answer.

A.2 What It Prevents

The failure mode is subtle because it produces output that looks like good research.

flowchart TD
    subgraph BAD["WITHOUT a firewall - motivated reasoning"]
        B1["PRD states:<br/>'HNSW handles 50M vectors at 40ms p99'"]
        B1 --> B2[Subagent reads the PRD<br/>as background context]
        B2 --> B3["Treats it as an established fact"]
        B3 --> B4["Searches to CONFIRM:<br/>'HNSW 50M scale success'"]
        B4 --> B5["Finds confirming vendor blog posts"]
        B5 --> B6["Report: 'Research confirms<br/>the approach is sound'"]
        B6 --> B7["<b>Your own assumption,<br/>laundered into a citation</b>"]
    end

    subgraph GOOD["WITH the firewall"]
        G1["PRD states the same thing"]
        G1 --> G2["Lead uses it to FRAME:<br/>'research HNSW p99 latency at 50M scale'"]
        G2 --> G3["Subagent receives only that question -<br/>never sees the PRD"]
        G3 --> G4["Searches openly:<br/>'HNSW latency benchmarks 50M'"]
        G4 --> G5["Finds production reports showing<br/>90-140ms at that scale on comparable hardware"]
        G5 --> G6["Report: 'Evidence contradicts<br/>a 40ms target at this scale'"]
        G6 --> G7["<b>The research can tell you<br/>you are wrong</b>"]
    end

    style B7 fill:#4a1f1f,color:#fff
    style G7 fill:#1f3d2a,color:#fff

The problem is not that the subagent lies. It is that a subagent holding your premise searches to support it. Query formulation is where the bias enters, long before synthesis. By the time a report is written, the sources genuinely do say what it claims - they were just selected by a question that assumed the answer.

The firewall's second effect is on citation integrity: because project files are inadmissible, every claim in research.md must trace to a digest or import file with a real publisher, date, and URL. A claim that traces only to your PRD cannot appear at all.

A.3 Worked Example - With and Without

Setup. You run bmad-deep-recon inside your search-service repo. The repo contains prd.md, ARCHITECTURE-SPINE.md, and project-context.md. Your architecture spine asserts a design decision you made six months ago.

Your input:

bmad-deep-recon run - should we move from per-language indices to a single
shared multilingual embedding space? Decision: whether to re-index 50M docs
this quarter.

What the lead orchestrator does with your project files

The brief that actually reaches a subagent

TOPIC: multilingual retrieval - shared embedding space vs per-language indices
DECISION: whether to re-index a 50M-document corpus this quarter

YOUR QUESTIONS (dimension 2 of 4):
  - What recall tradeoffs are reported between shared multilingual embedding
    spaces and per-language indices at 10M+ document scale?
  - Which failure modes are documented for cross-lingual retrieval in
    specialized domains?

SEARCH SURFACES: harness web search; <MCP tools if installed>
PREFERRED SOURCES: <from customize.toml>   BANNED: <from customize.toml>

PACK CRAFT (technical): read retrospective threads not launch threads;
favor accounts with production numbers over advocacy; before citing a pain
point check whether it was since fixed; freshness - versions <= 1 mo,
ecosystem signals <= 6 mo, landscape <= 12 mo (AI-adjacent <= 3 mo)
TWO-SOURCE CLASSES: performance/scale numbers; claims that an approach failed

BUDGETS: 8 sources, ~10 tool calls
QUERY CRAFT: short queries (<= ~5 words) beat hyper-specific ones; broaden
when sparse, narrow when abundant; never repeat an identical query on the
same tool; after every tool result, pause and evaluate before firing again

EPISTEMICS (verbatim):
  1. Never conclude from training data alone.
  2. The research firewall. Project context shapes what to ask, never what
     is true. It is inadmissible as evidence.

RETURN: a digest, not raw results - findings as claims, each with
{claim, source, publisher, pub_date, accessed, confidence, class};
plus leads worth chasing and what you looked for and could not find.

Notice what is absent. No PRD. No architecture spine. No project-context. No mention that you already run per-language indices, or that you have a preference. The subagent cannot tell whose project this is or which answer would be convenient.

What comes back

A digest file at {doc_workspace}/digests/recall-tradeoffs-r1-1.md, written to disk the moment it lands:

- claim: Shared multilingual spaces show 3-8% recall degradation vs
         per-language indices on domain-specific corpora
  source: <url>       publisher: <name>       pub_date: 2026-03
  accessed: 2026-08-01   confidence: medium   class: performance
- claim: Degradation narrows to under 2% when a per-language reranker
         is retained on top of a shared retrieval stage
  source: <url>       publisher: <name>       pub_date: 2026-01
  accessed: 2026-08-01   confidence: medium   class: performance
LEADS: hybrid stage architectures; "reranker retention" as a mitigation
NOT FOUND: no published figures above 20M documents in a legal-domain corpus

That last line - "absence of evidence is a finding" - is a firewall consequence. Without it, a subagent holding your context would be tempted to fill the gap from your own numbers.

How it lands in the report

The lead writes research.md from digest files only. Your project reappears only in the Recommendations section, where findings are bound to the artifacts that consume them - the pack's Feeds entries. So the flow is:

flowchart LR
    P["Project files<br/>PRD, spine, context, code"] -->|"shapes questions"| Q[Dimensions and briefs]
    Q --> S[Subagents behind the firewall]
    S -->|"digest files with sources"| D["digests/"]
    D -->|"only source of claims"| R["research.md"]
    P -.->|"NEVER a source of claims"| R
    R -->|"recommendations bound<br/>to your artifacts"| P

    style P fill:#2a2a3d,color:#fff

Project context flows in at framing time and back at recommendation time. It never flows into the evidence chain in the middle.

A.4 What a Subagent Brief Actually Contains

Per references/run.md, exactly these items - and the list is closed:

A.5 Where the Firewall Applies

Not just the Run-mode research fan-out. Every subagent in the skill runs behind it.

flowchart TD
    L["Lead orchestrator<br/>(holds project context -<br/>uses it only to frame)"]

    L -->|"brief only"| R1[Research assistant 1]
    L -->|"brief only"| R2[Research assistant 2]
    L -->|"brief only"| R3[Research assistant N]
    L -->|"digest files only"| V["Verifier subagent<br/>fresh context"]
    L -->|"conclusion + budget,<br/>NO supporting evidence,<br/>NO run context"| RT["Red-team skeptic<br/>fresh context"]
    L -->|"the import file only"| EX["Extraction subagent<br/>(Process mode)"]
    L -->|"research.md only"| CC["Citation checker<br/>fresh context"]

    R1 --> DG["digests/"]
    R2 --> DG
    R3 --> DG
    EX --> DG
    DG --> V
    V --> RM["research.md"]
    RT --> RM
    CC --> RM

    style L fill:#2a2a3d,color:#fff

A related isolation, worth noting: assistants are also firewalled from each other. Round 1 assistants do not see each other's returns. When two isolated assistants independently land on the same finding, that convergence is real signal. If they had shared context, agreement would mean nothing.

A.6 The Escape Hatches

The firewall is a default, not a wall without a door.

A.7 Limits - What It Does Not Guarantee

Worth stating plainly if you are relying on it.

• It is prompt-enforced, not sandboxed. The rule is text in SKILL.md that the lead is instructed to inherit into every brief. Nothing mechanically prevents a leak. There is no assertion in recon_kit.py that briefs are context-free, and no test in test_recon_kit.py covering it - the 6 shipped tests cover citations cross-check, tally last-status-wins, staleness windows, slug determinism, and URL escaping.
• It does not protect against a leading question. The firewall keeps your documents out. It cannot keep your framing out. "Research why shared embedding spaces underperform" is a biased dimension, and the firewall passes it through untouched. Framing quality is on you and the plan gate.
• It does not fix bad sources. Isolation improves independence, not source quality. That job belongs to the source-quality card - prefer primary sources, treat answer engines as single aggregators and chase their citations, resolve conflicts by recency and publisher quality rather than averaging.
• persistent_facts is a self-inflicted hole. Anything you add there travels into framing on every run. Empty is the safe default for a reason.

A.8 Provenance - PR #2611

The firewall arrived with the research consolidation.

Replaced bmad-market-research, bmad-domain-research, and bmad-technical-research - three near-duplicate step-file trees whose step-06-*-synthesis.md files alone were ~450-490 lines each, three times over.

The PR body states the firewall as one of two standing rules under "Epistemics and reliability," alongside files-first persistence and verification-at-landing. It also notes persistent_facts defaults to empty, and that the skill moved to src/core-skills/ so research reaches non-software installs.

Where to read it in your own install:

cat .claude/skills/bmad-deep-recon/SKILL.md              # the two standing rules
cat .claude/skills/bmad-deep-recon/references/run.md     # brief contents, fan-out
cat .claude/skills/bmad-deep-recon/references/verification.md   # verifier + red-team
cat .claude/skills/bmad-deep-recon/customize.toml        # persistent_facts default

Appendix B - Why the Firewall Works: Vocabulary, Philosophy, and Scenarios

Appendix A described what the research firewall is and how BMAD implements it. This appendix explains why the idea exists at all - where it comes from, what problem in human reasoning it solves, how a dozen other fields arrived at the same answer independently, and what it looks like across several situations on a recommendation system.

No background assumed. Every term is defined before it is used.

B.1 Start Here - One Idea You Already Use

Before any philosophy, here is the firewall in a form that will already be familiar from machine learning.

You would never evaluate a model on its training data.

Not because it would be dishonest. Because the number would be meaningless. A model that has seen an example can reproduce the answer without having learned anything general. The evaluation has to run on data the model has never seen, or it measures memory rather than skill.

The research firewall is a train/test split for reasoning.

• Your project documents are the training data - they shaped what the system expects.
• Your research question is the evaluation.
• If the researching agent has read your documents, its "finding" may just be reproducing your assumption. The number is meaningless in exactly the same way.
• So the researcher runs on held-out context: the question, and nothing else.

flowchart LR
    subgraph ML["What you already do"]
        T1[Training data] --> M1[Model]
        M1 --> E1[Evaluate on<br/>HELD-OUT test set]
        E1 --> R1["A number you can trust"]
        T1 -.->|"leakage =<br/>meaningless score"| E1
    end

    subgraph FW["What the firewall does"]
        T2["Project docs<br/>PRD, spine, code"] --> M2[Lead orchestrator]
        M2 -->|"question only"| E2["Research subagent<br/>HELD-OUT context"]
        E2 --> R2["A finding you can trust"]
        T2 -.->|"leakage =<br/>meaningless finding"| E2
    end

That dotted line has a name in ML: data leakage. The firewall exists to prevent the reasoning equivalent.

Everything below is why that instinct is correct, and how many different fields discovered it separately.

B.2 The Vocabulary

These words get used interchangeably in ordinary speech and mean quite different things here.

Idea

Anything that occurs to you. There is no quality bar - an idea is just a thought that showed up.

"Maybe we should try a two-stage retrieval pipeline."

Could be excellent. Could be terrible. Nobody has checked yet, and nothing about the word "idea" implies anyone will.

Hunch (also: intuition, gut feeling)

A belief you hold without being able to state your reasons. The feeling is real; the articulation is missing.

"Something's wrong with the French results. I can't point at what. I just know."

Hunches are frequently correct - they often compress genuine experience you absorbed without consciously recording it. That is why experienced engineers have better hunches than new ones. But a hunch is the output of a process you cannot inspect, which is exactly why it cannot serve as proof to anyone else, including your future self.

Hypothesis

A guess stated specifically enough that a result could prove it wrong.

That last clause is the whole definition. Compare:

The test is called falsifiability, associated with the philosopher Karl Popper (1902-1994). His point: a claim that no possible observation could contradict is not a strong claim, it is an empty one. "The recommender works better on Tuesdays because of user energy" survives every possible result, which means it explains nothing.

Why this matters for the firewall: a well-formed hypothesis is what your project documents legitimately contribute. Your architecture spine can supply the guess. It cannot supply the measurement.

Theory

The genuinely confusing word, because it means opposite things in casual and technical speech.

Almost nothing in software engineering rises to the scientific sense. When you say "my theory is the cache is cold on first request," you mean hypothesis. That is fine - just know that "theory" in a paper means something else entirely.

Evidence

An observation from outside your own head that bears on whether a claim is true.

The "outside your own head" part is load-bearing. Your recollection that recall dropped is not evidence; it is testimony about a memory. The eval run's output file is evidence. A published benchmark is evidence. A sentence in your own PRD asserting a number is not evidence, because your PRD is a record of what you concluded, not of what was observed.

Warrant / warranted

You have good reason to believe something. It has earned belief.

The same sentence can be warranted or unwarranted depending entirely on how you came to hold it:

"Shared multilingual embeddings hurt recall on legal text."

• Unwarranted: you noticed some French results looked off six months ago
• Warranted: you ran a controlled eval on 10k labelled legal queries and measured a 7.2% drop

The sentence is identical. The warrant is not. This is the distinction the firewall protects, and it is invisible from the sentence alone - which is precisely why documents are dangerous. A doc records the sentence and drops the warrant.

Claim

A statement that asserts something is true. In BMAD's vocabulary, a claim is specifically a research finding tracked in the memlog with {claim, source, publisher, pub_date, accessed, confidence, class}.

The skill's phrasing: "A claim is a sentence with a source. Publisher, publication date, access date. No naked numbers." A number with no source attached is not a claim in this system - it is noise that looks like a claim.

Admissible / inadmissible

Borrowed from courtroom procedure. Admissible evidence is material a court will allow the jury to consider. Inadmissible material may be entirely true and still be excluded, because the process by which it arrived is not trustworthy.

Two examples from law:

• Hearsay - "my colleague told me the vendor's benchmark showed 40ms." Possibly true. Excluded, because the person who actually observed it isn't there to be questioned.
• Fruit of the poisonous tree - evidence obtained through an unlawful search is thrown out even when it proves guilt, because admitting it would reward the bad process.

Both share a principle: courts regulate the pipeline, not just the conclusion. A system that only checked conclusions would be gameable by anyone who controlled the inputs.

When BMAD says project context is "inadmissible as evidence," it is using the word precisely. Your PRD may be entirely correct. It is still excluded, because material that originated inside the project cannot be used to validate the project.

Prior and likelihood

From Bayesian reasoning - two ingredients of an updated belief:

• Prior - what you believed before looking. Where you point your attention.
• Likelihood - what the new data says.

Both are legitimate. The error is letting the prior masquerade as data - counting your existing belief twice, once as a belief and once as evidence for itself.

The firewall in one line of this vocabulary: your project supplies the prior; only retrieved sources supply the likelihood.

Bias

Not "prejudice" in the moral sense. A systematic deviation - an error that leans consistently in one direction rather than scattering randomly.

Random error averages out with more samples. Systematic error does not: run a biased process a hundred times and you get a hundred results wrong in the same direction, plus false confidence from the consistency. That is why bias is treated as a design problem rather than an effort problem.

B.3 Discovery vs. Justification

The distinction

The philosopher Hans Reichenbach (1891-1953) drew a line in Experience and Prediction (1938) between two questions people constantly blur:

The claim: how you arrived at an idea has no bearing on whether it is correct.

This cuts in both directions, and both directions matter:

• A ridiculous origin does not make an idea wrong. Dreams, analogies, and accidents have produced real discoveries.
• An impressive origin does not make an idea right. Deep expertise, careful reasoning, and a senior title are all origins. None is evidence.

The second half is the one that bites in engineering. "The staff engineer decided this after a lot of thought" is a discovery story. It feels like justification. It is not.

Three examples

The lottery dream

You dream of the number 17. You buy a ticket. It wins.

• Discovery: a dream. Absurd, and it worked.
• Justification: the draw.
• The dream was never evidence. You would still be foolish to bet your savings on tomorrow night's dream - even though last night's paid out. The origin worked once and is still worthless as proof.

Kekule's snake

August Kekule reported working out the ring structure of benzene after a reverie in which a snake seized its own tail.

• Discovery: a daydream about a snake.
• Justification: X-ray crystallography and a century of chemistry.
• No chemist believes in the benzene ring because of the snake. The snake produced a candidate; the instruments produced the warrant.

Your architecture doc

Six months ago you noticed French results looked weak. You formed a hypothesis - shared multilingual embeddings hurt recall on legal text - and wrote it into ARCHITECTURE-SPINE.md, then built per-language indices.

• Discovery: your hunch. Entirely legitimate, and probably good judgment.
• Justification: never happened. You did not run the eval.
• The trap: once written in a document, a hunch stops looking like a hunch. Six months on, ARCHITECTURE-SPINE.md reads like a statement of fact. Nothing in the sentence carries a marker saying "this was a feeling on a Tuesday."

flowchart TD
    H["A hunch<br/>'French results feel weak'"] --> W["Written into a document"]
    W --> T["Time passes"]
    T --> L["<b>Reads as established fact</b><br/>The warrant is gone;<br/>the sentence remains"]
    L --> D["Cited in decisions,<br/>quoted in reviews,<br/>fed to an AI agent"]
    D --> C["<b>Circular:</b> the project<br/>validates itself"]

    style L fill:#4a3a1f,color:#fff
    style C fill:#4a1f1f,color:#fff

This is not a failure of rigor by anyone. It is what documents do: they preserve conclusions and discard provenance. Every long-lived codebase contains decisions whose original justification nobody can reconstruct.

Why an AI agent makes it worse

A human reading ARCHITECTURE-SPINE.md may remember it was a guess. They were there. An agent has no such memory - it reads the sentence and receives a confident assertion with no epistemic status attached. Then it goes looking for support.

The firewall is a mechanical answer to a mechanical problem: if the agent cannot read the document, it cannot mistake the document's confidence for evidence.

B.4 Why Human Brains Need This

The firewall would be unnecessary if reasoning systems, human or artificial, were naturally even-handed. A century of psychology says otherwise.

Confirmation bias

The tendency to seek, notice, and remember information that supports what you already believe - and to skip past what doesn't.

Peter Wason demonstrated it cleanly in 1960. Subjects were told the sequence 2, 4, 6 followed a rule, and asked to discover the rule by proposing their own triples, receiving only yes/no feedback.

Almost everyone guessed "ascending even numbers," then tested 8-10-12, 20-22-24, 100-102-104 - each answered "yes," each confirming their guess. Very few tried 1-2-3 or 5-4-3, which is what it takes to find out you're wrong.

The actual rule was simply any increasing sequence. Nearly all confident answers were wrong, and the confidence came from a long run of yeses.

This is the exact failure the firewall prevents. An agent holding your hypothesis generates confirming queries - "shared embeddings recall problems legal" - and every result comes back yes. The problem is never in the sources. It is in which questions got asked.

Motivated reasoning

Ziva Kunda (1990) showed that wanting a conclusion changes how thoroughly you scrutinise evidence for it. People do not simply believe what they want - they construct justifications, applying real reasoning, just unevenly. Evidence against a preferred conclusion gets audited hard; evidence for it gets waved through.

An agent inside your repo inherits a preference: your project has clearly already chosen. Even without intent, the asymmetric scrutiny follows.

Anchoring

Tversky and Kahneman (1974): an initial number distorts subsequent estimates, even when it is obviously arbitrary. Subjects who saw a spun wheel land on a high number gave higher estimates for an unrelated quantity than those who saw a low number.

If a subagent reads "sub-40ms p99 required," 40ms becomes the anchor. Sources near it read as confirming; sources far from it read as edge cases about different setups. The red-team subagent in BMAD receives the conclusion but explicitly no supporting evidence - that is anti-anchoring by construction.

The streetlight effect

From the joke about the drunk searching for his keys under a lamppost because the light is better. Research goes where searching is easy rather than where the answer is.

Your documents make certain searches easy - they hand you vocabulary, product names, framings. An agent with your docs searches your vocabulary. An agent without them has to find the field's own terms, which is where unfamiliar options live. This is a substantial part of the firewall's practical value: the third option you had not considered is usually filed under words your project never uses.

HARKing

Norbert Kerr (1998) named it: Hypothesizing After the Results are Known. You run an analysis, notice a pattern, then present it as though you had predicted it in advance.

The result looks far stronger than it is, because a prediction that survives a test is meaningful, while a pattern found in data you already have is just a description of that data. Sequence is doing hidden work.

The research version: run a search, notice which findings fit the plan, present the report as if those were the questions all along. BMAD counters with files-first - digests are written the moment they land, before anyone knows which way the conclusion falls, and the memlog is append-only. Retroactive tidying leaves a trace.

The garden of forking paths

Gelman and Loken (2013) described this subtler cousin: even a researcher who never consciously fishes for results makes dozens of small analytic choices - which subgroup, which cutoff, which outliers to drop - and each choice, made in the presence of a preferred conclusion, tilts slightly the same way. No single decision is misconduct. The aggregate is a foregone conclusion.

In research: which sources to open, which to skim, which to call authoritative, which contradiction is "an edge case." All defensible individually.

The pattern across all six: none requires bad faith. Every one is what ordinary careful reasoning does when the reasoner already holds the answer. Which is why "be objective" fails as a remedy, and why every field below reached for structure instead.

B.5 How Other Fields Solved It

The same answer, arrived at independently, across fields that never talked to each other: do not ask the reasoner to ignore information - withhold the information.

Blinding (medicine)

The assessor is not told which treatment a subject received.

• Single-blind - the subject doesn't know
• Double-blind - neither subject nor assessor knows
• Triple-blind - the statistician analysing the data doesn't know either

The reason is not suspicion of doctors. It is that knowing changes judgment below the level of intention. A physician who knows this patient got the real drug reads an ambiguous chart slightly more generously - and would sincerely deny doing so, because they are not aware of it.

A striking early instance: the 1784 French royal commission investigating Franz Mesmer's "animal magnetism," which included Benjamin Franklin and Antoine Lavoisier. Subjects were blindfolded and told they were or were not being magnetised, sometimes falsely. Effects tracked what subjects believed, not what was actually done. Widely cited as one of the first blinded experiments - and it worked by controlling information, not by asking anyone to be fair.

Map to BMAD: the research subagent is the blinded assessor. It is not told which answer the project is hoping for.

Chinese walls / information barriers (finance and law)

A bank advising Company A on a confidential merger also trades Company A's stock. If the traders learn of the merger, trading on it is illegal.

The remedy is not a memo asking traders not to use inside information. It is structural: separate systems, separate floors, restricted document access, monitored crossings, and a documented process for "wall-crossing" someone deliberately when there is a legitimate need.

Law firms do the same to isolate teams acting for opposing clients. The term is increasingly rendered as ethical wall or information barrier.

Map to BMAD: the lead orchestrator is inside the wall (it holds your project files); research subagents are outside it. The explicit named grant at the plan gate is the wall-crossing procedure - deliberate, logged, exceptional.

Rules of evidence (law)

Courts regulate how material arrives, not only whether it seems true. Hearsay is excluded even when accurate; unlawfully obtained evidence is excluded even when conclusive; chain of custody requires documenting every hand a piece of evidence passed through, because unbroken provenance is what makes it trustworthy.

Map to BMAD: digests/ and imports/ are the chain of custody. Every claim traces to a file with publisher, publication date, and access date. The recon_kit.py citations check is a mechanical audit that no claim entered without one - the "no naked numbers" rule, enforced.

Separation of duties (accounting, security)

The person who approves a payment cannot also issue it. The developer who writes a deployment cannot also approve it. Not because anyone is presumed dishonest, but because concentrating both powers removes the check regardless of intent. Also called maker-checker or four-eyes.

Map to BMAD: the agent that frames the research is not the agent that answers it. Whoever holds the hypothesis does not get to grade it.

Peer review and the devil's advocate

Journals send work to reviewers who did not produce it; many blind reviewers to author identity so reputation doesn't substitute for argument.

The devil's advocate is older still - the advocatus diaboli was a formal office in Catholic canonisation proceedings, a person whose assigned job was to argue against the candidate. Someone was paid to attack the conclusion, because volunteering objections against a consensus is unreliable.

Map to BMAD: the red-team pass in references/verification.md is a paid devil's advocate. A fresh-context skeptic receives the conclusion and a search budget, no supporting evidence, no run context, and hunts for disconfirming material. A conclusion that survives keeps its strongest counter-argument on the record; one that doesn't is revised before the report states it.

Preregistration (science, post-2011)

Researchers publicly record their hypothesis and analysis plan before collecting data. This makes HARKing and forking-path drift visible - the record of what you intended to test exists independently of what you found.

Map to BMAD: the plan gate plus the append-only memlog. The plan is logged as a decision before acquisition begins; mid-run scope changes are logged as further decision entries rather than silently applied.

Adversarial collaboration

Daniel Kahneman promoted this: two researchers who disagree design a study together, agreeing in advance what result each would accept as losing. Neither side gets to specify the test alone.

Map to BMAD: the closest analogue is the select shape's weighted decision matrix, where scoring is shown rather than only totals - so a reader can re-weight and reach a different verdict. The skill's own phrasing: a matrix the user can re-weight is worth more than a verdict they must trust.

Norms of science

Robert Merton (1942) described norms that scientific communities enforce, two of which are directly relevant:

• Organized skepticism - claims are subject to structured criticism as a matter of course, not as an insult
• Disinterestedness - findings are judged apart from what the finder gains from them

Older and blunter: the Royal Society's motto nullius in verba - roughly, take nobody's word for it. And Richard Feynman's formulation, from his 1974 "cargo cult science" address: the first principle is that you must not fool yourself, and you are the easiest person to fool.

flowchart TD
    P["<b>The shared problem</b><br/>A reasoner who already holds<br/>the answer will find support for it -<br/>without intending to"]

    P --> M["Medicine<br/><b>Blinding</b><br/>don't tell the assessor"]
    P --> F["Finance / Law<br/><b>Chinese wall</b><br/>separate the systems"]
    P --> L["Courts<br/><b>Admissibility</b><br/>regulate the pipeline"]
    P --> A["Accounting<br/><b>Separation of duties</b><br/>split the powers"]
    P --> S["Science<br/><b>Peer review,<br/>preregistration,<br/>devil's advocate</b>"]
    P --> ML["Machine learning<br/><b>Held-out test set</b><br/>never evaluate on train"]

    M --> B["<b>BMAD</b><br/>The research firewall"]
    F --> B
    L --> B
    A --> B
    S --> B
    ML --> B

    style P fill:#2a2a3d,color:#fff
    style B fill:#1f3d2a,color:#fff

None of these fields asks the participant to try harder. Every one of them changes what information reaches whom.

B.6 The Machine-Learning Version You Already Know

Recommender and search work has its own vocabulary for exactly this principle. The firewall is not a new idea being imported into your field - it is an idea your field already enforces rigorously, applied to a place where nobody was enforcing it.

Data leakage

Information from outside the training set slips into training, and the model scores brilliantly on your evaluation and poorly in production.

Classic forms in recommender work:

• Target leakage - a feature encodes the label. "Number of times this item was clicked" as a feature for predicting clicks
• Temporal leakage - training on data from after the evaluation window. The model has seen the future
• Group leakage - the same user appears in both train and test, so the model memorises that user instead of generalising to new ones
• Preprocessing leakage - fitting a normaliser or an embedding on the full dataset before splitting, so test statistics bleed into training

Every one is a case of the answer reaching the thing that was supposed to derive the answer independently. That is the firewall's failure mode, exactly.

Held-out sets and temporal splits

The remedy is structural. You do not tell the model to ignore the test set - you make the test set unreachable. For recommenders specifically, a random split is usually the wrong structure, because recommendation is a prediction about the future: the standard is a temporal split, training on everything before time T and evaluating after it.

Note what that reflects: the split is designed so the evaluation can fail. A random split makes the number look better. Practitioners choose the harder split because the flattering number is worthless.

That choice is the firewall's whole philosophy. A research process configured so it cannot contradict you is the random split of reasoning - it will always report good news.

A/B testing discipline

Your work already runs on these rules:

• Fixed sample size and analysis plan in advance. Peeking at a running test and stopping when it turns significant inflates false positives dramatically - this is preregistration under a different name
• A holdout group that receives nothing, so you can measure against reality rather than against your previous variant
• Guardrail metrics defined before launch, so you cannot pick the metric that happens to have moved
• Novelty effects - a new model looks better for two weeks because it is new. Longer horizons exist so the flattering early number does not decide

Every item is a mechanism for stopping the person who wants a result from getting it by accident.

The offline/online gap

Every recommender team knows the pattern: offline metrics improve, the online test shows nothing. Offline evaluation is scored against logged data produced by the old system, so it systematically rewards models that agree with the incumbent. The evaluation is contaminated by the thing it was supposed to judge.

That is the firewall's problem statement in your own domain. An agent researching inside your repo is running offline evaluation against logs produced by your existing decisions - and it will systematically reward conclusions that agree with your incumbent design.

B.7 Where BMAD Sits in That Lineage

Every mechanism in the skill maps to one of the traditions above.

One line summarises the design: BMAD did not invent a new epistemology. It took the standard toolkit for stopping motivated reasoning and wired it into an agent harness.

B.8 Six Scenarios on a Recommendation System

Concrete runs on the kind of system you work on - OpenSearch, HNSW vector retrieval, embeddings, reranking, A/B evaluation, multilingual content. Each shows the same structure: what your project supplies, what the subagent receives, what goes wrong without the wall, and what specifically you gain.

Scenario 1 - Vector index parameters

The situation. ARCHITECTURE-SPINE.md says: "HNSW with M=16, efConstruction=200 gives us adequate recall at acceptable latency." Those numbers came from a benchmark on 2M documents eighteen months ago. The corpus is now 50M. You ask whether to retune.

What your project legitimately supplies (discovery): that HNSW parameters are the question; that the corpus is 50M; that OpenSearch 2.13 is the engine; that p99 latency is the binding constraint. All of this shapes what to ask.

What the subagent receives:

QUESTION: How do HNSW recall and latency characteristics change between
2M and 50M vector corpora? What M / efConstruction / efSearch values do
production deployments report at 50M+ scale?
BUDGET: 8 sources
TWO-SOURCE CLASSES: performance and scale numbers
FRESHNESS: versions and compatibility <= 1 month; landscape <= 12 months

Note the absence: no M=16, no efConstruction=200, no mention that anyone already chose values.

Without the wall. The subagent reads the spine, searches "HNSW M=16 efConstruction 200 recall," and finds material discussing those specific values - because people do use them. Report: your configuration is within the commonly recommended range. True, and useless. It never surfaces that the recommended range is a function of corpus size and dimensionality, because it was never asked a question shaped that way.

With the wall. The open question surfaces the actual relationship: graph connectivity requirements scale with corpus size, and efSearch - a query-time parameter your spine does not mention at all - is typically the dominant recall/latency lever in production. The report can now say your build-time parameters are probably not the interesting knob.

The concrete gain: you were asking about the wrong parameter. Only a subagent that had not been handed your parameter list could tell you that.

Scenario 2 - Choosing a reranker

The situation. You need a cross-encoder reranker. You have informally settled on a well-known open model because your team knows it. You want research before committing.

This is a select shape decision - the one place BMAD deliberately inverts the flow.

flowchart TD
    subgraph YOURS["Your project OWNS the requirements frame"]
        R1["Hard gates:<br/>self-hostable, multilingual,<br/>Apache/MIT licence"]
        R2["Weighted preferences:<br/>latency 40%, quality 30%,<br/>ops burden 20%, cost 10%"]
    end

    subgraph RESEARCH["Research OWNS the evidence"]
        E1[Candidate screen]
        E2[Score each finalist<br/>against the frame]
        E3[Cost and lock-in]
    end

    YOURS -->|"frame agreed FIRST"| E1
    E1 --> E2 --> E3
    E3 --> V["Weighted matrix -<br/>scoring shown, re-weightable"]
    YOURS -.->|"NEVER supplies<br/>a score"| E2

    style YOURS fill:#2a2a3d,color:#fff

references/selection.md is explicit: requirements come from the project and the user, and web research does not set requirements. But the moment a candidate is scored, the firewall applies fully.

Without the wall. The subagent knows your team's preferred model. Its screening queries centre on that model; competitors appear framed as alternatives to it. The matrix is built, your model wins, and the exercise has laundered a preference into a decision.

With the wall. Screening runs against the field, not against your favourite. Your model may still win - and if it does, you now have a defensible reason rather than a familiar one. The named runner-up and the conditions under which it wins instead is the deliverable you could not have produced yourself, because you did not know what the alternatives were good at.

The concrete gain: the pack's two-source classes force pricing and performance figures to be independently confirmed, and the matrix is re-weightable - so when someone challenges your latency weighting six months later, you adjust a number rather than redo the research.

Scenario 3 - Multilingual embeddings (the running example)

The situation. ARCHITECTURE-SPINE.md says shared multilingual embeddings hurt recall on legal text. That was a hunch, six months ago, never measured. You are asking whether to re-index 50M documents into a shared space.

Without the wall.

Queries: "shared multilingual embeddings recall problems legal", "why per-language indices better domain specific"

Report: Research confirms shared embedding spaces degrade recall on specialised corpora. Per-language indexing remains the stronger choice. [3][7][11]

Every citation is real. The sources genuinely say that. The bias entered at query formulation, which is why the output is indistinguishable from good research. You have received your own six-month-old hunch with footnotes attached - and it now looks more authoritative than when you wrote it, because it has citations.

With the wall.

Question: What recall differences are reported between shared multilingual embedding spaces and per-language indices at 10M+ documents?

Report: 3-8% degradation on domain-specific corpora - narrowing to under 2% when a per-language reranker is retained on top of shared retrieval. No published figures above 20M documents in legal-domain corpora.

Three concrete gains:

1. A third option. Hybrid architecture - shared retrieval, per-language reranking - was in neither your doc nor your question. It was found because the search was open
2. A quantified tradeoff. 3-8% versus under 2% is a number you can weigh against re-indexing cost. "It hurts recall" is not
3. An honest gap. "No published figures above 20M in legal-domain corpora" tells you the evidence thins out exactly where your corpus lives - so the responsible next step is your own eval, not more reading. A contaminated run would have filled that silence with your own numbers

Scenario 4 - Post-incident, under pressure

The situation. Recall@10 dropped 12% overnight after a cluster upgrade. Everyone believes the upgrade caused it. You run research while the incident is open.

This is where the firewall matters most, because urgency is when confirmation bias is strongest. Under time pressure, the first plausible explanation gets adopted and everything after it becomes confirmation.

Without the wall. The subagent reads your incident notes - "recall dropped after upgrade to 2.13" - and searches "OpenSearch 2.13 recall regression". It finds something, because in a large project there is always some open issue mentioning recall. Report: a known regression may be responsible. The team spends a day on a rollback.

With the wall. The question goes out as: What changes to k-NN query behaviour, scoring, or default parameters were introduced in OpenSearch 2.13 relative to 2.11? An open question about what changed, rather than a search for a culprit.

Now the report can surface a default parameter change - a silently altered ef_search default, say - which is not a bug, would never appear under "regression," and is fixed with a config line rather than a rollback.

The concrete gain: the difference between hunting for evidence of a suspect and asking what actually changed. Also note the type pack's craft rule here: before citing a pain point, check whether it was since fixed - an old complaint against a current version is a false claim. An eighteen-month-old GitHub issue is exactly what a motivated search surfaces first.

Scenario 5 - Cold start for new items

The situation. New items get no impressions for days. Your PRD asserts: "cold start is a content-embedding problem - we need better item embeddings."

Without the wall. Every query is about embeddings, because the PRD framed the problem as an embedding problem. The report describes content-based embedding approaches for cold start. Entirely accurate, and it never questions the framing.

With the wall. The dimension goes out as: What approaches do production recommender systems use to surface items with no interaction history, and what tradeoffs are reported for each?

The answer space is much wider than embeddings: explicit exploration budgets, bandit approaches, position-boosting for new inventory, popularity priors, hybrid fallbacks. Several are ranking-policy changes rather than representation changes - cheaper to ship and independently testable.

The concrete gain: the firewall protected you from a premise you did not notice you had made. Your PRD had already converted "new items don't surface" into "our embeddings are inadequate," and once written, that conversion was invisible. This is the streetlight effect in practice - your documents made the embedding search easy, and the ranking-policy literature uses vocabulary your project never introduced.

Scenario 6 - Evaluation methodology

The situation. You want to validate your A/B design before a quarter-long test - metric choice, split strategy, minimum detectable effect.

This one is delicate, because your design is project context.

Without the wall. The subagent reads your test plan and searches for support for the metrics you chose. It confirms that NDCG@10 is standard, that your split is common, that your MDE is reasonable. All true. It does not tell you that your metric is known to be insensitive to the specific change you are testing, because it was never asked that question - it was asked to evaluate your plan, and evaluating a plan means finding the plan's merits.

With the wall. Two open dimensions:

1. Which offline metrics are reported to correlate with online engagement in recommender A/B tests, and which are reported not to?
2. What sample-size and duration considerations do practitioners report for detecting small ranking-quality changes?

Now the offline/online correlation literature can reach you - including the widely reported finding that offline ranking metrics correlate poorly with online outcomes for certain classes of change. That is a finding about your plan that a search for support of your plan would never surface.

The concrete gain: this is the offline/online gap from B.6, operating one level up. Your evaluation design was itself being evaluated against logged assumptions.

What the six have in common

flowchart LR
    A["Your project<br/>supplies the question"] --> B["Firewall"]
    B --> C["Open search finds<br/>what the field actually says"]
    C --> D1["A parameter you<br/>weren't asking about"]
    C --> D2["A third option<br/>nobody proposed"]
    C --> D3["A gap in the evidence<br/>where your case lives"]
    C --> D4["A premise you<br/>didn't know you had"]

    style B fill:#1f3d2a,color:#fff

In none of the six did the firewall make the research more thorough. It made it capable of disagreeing. Every gain above came from a question being open rather than loaded - and in five of six, the useful finding was something nobody at the company would have thought to ask for.

B.9 What the Firewall Does Not Fix

Stated plainly, since the mechanism is easy to over-trust.

It does not stop a leading question. The firewall keeps your documents out. It cannot keep your framing out. "Research why shared embeddings underperform" sails straight through - the bias is in the question, and the question is the one thing the firewall is designed to let past. This is the load-bearing weakness. Framing quality rests on you and on the plan gate, which is why that gate is the skill's one hard stop.

It is prompt-enforced, not sandboxed. The lead orchestrator holds all your project context and is trusted to assemble clean briefs. No assertion in recon_kit.py and no test in test_recon_kit.py covers it. A leak fails silently - no error, just a slightly agreeable report. It sits closer to convention-enforced than architecture-enforced.

It does not improve source quality. Isolation buys independence, not credibility. That job belongs to the source-quality card: prefer primary sources; treat answer engines as single aggregators and chase their citations; resolve conflicts by recency, consistency, and publisher quality - never by averaging.

It does not fix training-data bias. Standing rule 1 exists as a separate rule precisely because the firewall does not cover it. Isolation from your documents is not isolation from what the model already believes about HNSW.

persistent_facts is a self-inflicted hole. Anything placed there travels into framing on every run. [] is the safe default for a reason.

It cannot make a decision for you. Research narrows uncertainty; it does not choose. A report saying "3-8%, narrowing to under 2% with a reranker" still leaves you weighing re-indexing cost against recall against team capacity. That weighting is yours, and it should be.

B.10 Glossary and Sources

Terms, in the order they were introduced

Named works, for anyone who wants to read further

• Hans Reichenbach, Experience and Prediction (1938) - the discovery/justification distinction
• Karl Popper, Logik der Forschung (1934; English: The Logic of Scientific Discovery, 1959) - falsifiability
• Robert Merton, "The Normative Structure of Science" (1942) - organized skepticism, disinterestedness
• Peter Wason, "On the failure to eliminate hypotheses in a conceptual task" (1960) - the 2-4-6 experiment
• Amos Tversky and Daniel Kahneman, "Judgment under Uncertainty: Heuristics and Biases" (Science, 1974) - anchoring
• Richard Feynman, "Cargo Cult Science" (Caltech commencement address, 1974) - you are the easiest person to fool
• Ziva Kunda, "The Case for Motivated Reasoning" (Psychological Bulletin, 1990)
• Norbert Kerr, "HARKing: Hypothesizing After the Results are Known" (1998)
• Andrew Gelman and Eric Loken, "The Garden of Forking Paths" (2013)
• 1784 French Royal Commission on animal magnetism (Franklin, Lavoisier, Bailly) - an early blinded experiment

Where to read BMAD's own version

grep -A4 "research firewall" .claude/skills/bmad-deep-recon/SKILL.md
cat .claude/skills/bmad-deep-recon/references/run.md            # brief contents
cat .claude/skills/bmad-deep-recon/references/verification.md   # verifier, red team
cat .claude/skills/bmad-deep-recon/references/selection.md      # the deliberate inversion
grep -n "persistent_facts" .claude/skills/bmad-deep-recon/customize.toml

Historical and academic attributions in this appendix are summarised from general knowledge rather than retrieved sources; the named works are given so they can be checked directly. The BMAD implementation details are quoted from the skill files shipped in PR #2611 and are verifiable in any v6 install.