Tool registry design, and tool descriptions as prompt surface

What it is

A tool is a function the model can invoke, exposed to it as a name, a description and a parameter schema. A tool registry is the system that decides which tools exist, which are visible for a given request, how they are described, how they are versioned, and what happens when one fails.

The reframing that makes this a design topic rather than a plumbing one: every tool definition is prompt text, sent on every request, and it competes with everything else for the model's attention.

20 tools x ~180 tokens of definition = 3,600 tokens
Sent on EVERY step of an agent loop.
A 9-step task pays it 9 times: 32,400 tokens of tool definitions.

So a tool description is not documentation for a human reader. It is a prompt fragment whose job is to make the model choose correctly, it has a token cost paid repeatedly, and it should be written and tested accordingly.

What this is confused with: an API and a tool are not the same interface. An API is designed for a programmer who reads documentation, holds context across calls, and can compose primitives. A tool is invoked by a model that sees only the description, in the middle of a task, with no memory of the last time it used it. The correct tool surface is usually coarser than the API it wraps, and treating "expose the API" as the design is the most common mistake.

The problem it solves

Wrong-tool selection scales badly with tool count. Measured on the same agent with the same task set:

tools available    wrong-tool rate    task success
     6                  4%               91%
    12                  9%               86%
    25                 16%               77%
    40                 18%               67%

Two mechanisms. The tool-definition prefix grows, so it consumes a larger share of the context and pushes everything else toward the positional trough (see context rot). And the model is choosing among more similar-looking options, several of which plausibly match.

Descriptions are the second variable and are usually left to whoever wrote the function. These are two real definitions of the same underlying capability:

# The API-mirroring version
Tool("get_user_records",
     description="Retrieves user records from the database.",
     params={"user_id": "string", "include_deleted": "boolean",
             "fields": "array", "limit": "integer"})

# The model-facing version
Tool("lookup_customer",
     description=(
         "Look up a customer's account details by their customer ID "
         "(format: C-12345) or email address. Returns name, plan, "
         "signup date and current status. "
         "Use this before answering any question about a specific customer. "
         "Does NOT return orders or billing history: use lookup_orders "
         "or lookup_invoices for those."),
     params={"identifier": "Customer ID (C-12345) or email address"})

The second is longer and produces measurably better selection, because it answers the three questions the model is actually asking: what does this return, when should I use it, and how is it different from the neighbouring tool.

The third problem is failure semantics. A tool that returns [] for both "no results" and "malformed query" gives the model no way to distinguish retry from rethink, so it retries the same malformed query. That is the loop failure on the agent failure modes page, caused by a tool rather than by the model.

Mechanics

Writing a tool description

Five elements, in this order:

1. What it does, in the caller's vocabulary, not the implementation's.
2. What it returns, specifically enough to know if it answers the question.
3. WHEN to use it: the trigger condition.
4. What it does NOT do, naming the neighbouring tool.
5. Constraints: rate limits, side effects, idempotency.

Tool(
    name="issue_refund",
    description=(
        "Issue a refund for a specific order. "                          # 1
        "Returns a refund ID and the expected settlement date. "         # 2
        "Use only after confirming the order exists and the customer "   # 3
        "is entitled to a refund under the policy. "
        "Does NOT cancel a subscription (use cancel_subscription) and "  # 4
        "does NOT reverse a chargeback (escalate to a human). "
        "SIDE EFFECT: moves money. Not idempotent: calling twice issues " # 5
        "two refunds. Requires approval above $500."
    ),
    params={
        "order_id": "Order ID, format O-12345",
        "amount_cents": "Amount in CENTS. 4250 means $42.50.",
        "reason": "One of: duplicate_charge, item_damaged, not_received, other",
    })

Element 4 is the one that most improves selection accuracy, because most wrong-tool errors are between two tools with adjacent meanings. Saying explicitly what a tool is not for, and naming its neighbour, resolves exactly that confusion.

The amount_cents parameter description is not pedantry. Unit ambiguity in tool parameters is a live source of production incidents, and 4250 means $42.50 costs six tokens.

Progressive disclosure: showing fewer tools

The most effective lever on selection accuracy is reducing the choice set per request.

class ToolRegistry:
    def visible_for(self, request, agent_state) -> list[Tool]:
        tools = []
        for tool in self.all_tools:
            if not tool.available_in(agent_state.phase):     # phase-gated
                continue
            if not tool.permitted_for(request.principal):    # permission-gated
                continue
            tools.append(tool)
        return tools

Three gating strategies, and they compose:

Phase gating. An agent in an "investigating" phase does not need write tools; one in "executing" does not need search tools. Halving the visible set for each phase is common.

Permission gating. A tool the caller is not authorised to use should not appear at all. Filtering after the model chooses is both slower and worse: the model wastes a step choosing something it cannot use, and the refusal message is a confusing observation.

Retrieval over tools. Above roughly 50 tools, embed the descriptions and retrieve the top 10 to 15 for the request:

def visible_for(self, request) -> list[Tool]:
    if len(self.all_tools) <= 15:
        return self.all_tools
    candidates = self.tool_index.search(request.text, k=12)
    return candidates + self.always_visible      # never hide the core tools

The always_visible set matters: a retrieval miss that hides the tool the task needs is worse than a slightly larger prefix, so core tools and any "escalate to a human" tool are never retrieval-gated.

Granularity: coarser than the API

The instinct is to expose primitives and let the model compose them. In practice composite tools that match a task boundary outperform primitives:

# Primitives: 4 tools, 4 model calls, 4 chances to get it wrong.
get_customer(id) -> Customer
get_orders(customer_id) -> list[Order]
get_order_items(order_id) -> list[Item]
get_shipment(order_id) -> Shipment

# Composite: 1 tool, 1 call, matched to what is actually asked.
get_customer_order_summary(customer_id, limit=5) -> {
    customer: {...}, recent_orders: [{items: [...], shipment: {...}}]
}

Task: "when will the customer's latest order arrive?"
  primitives:  4 tool calls, ~11s, 3 opportunities for a wrong call
  composite:   1 tool call, ~2s, and the result contains the answer

Design tools around tasks, not around your data model. The composite version also returns a coherent object rather than fragments the model must join, which removes a class of reasoning error.

The counter-pressure is that composites are less flexible and multiply as tasks vary. The rule: a composite for every task the agent does routinely, primitives available for the tail.

Errors are prompt surface too

# Useless: the model cannot distinguish these cases.
return []

# Useful: the model knows what to do next in each case.
return ToolResult(
    ok=False,
    error_type="NOT_FOUND",
    message="No order O-99999 exists. Check the ID, or use "
            "lookup_orders(customer_id) to list this customer's orders.",
    retryable=False)

return ToolResult(
    ok=False,
    error_type="RATE_LIMITED",
    message="Rate limited. Retry after 8 seconds.",
    retryable=True, retry_after_s=8)

return ToolResult(
    ok=False,
    error_type="INVALID_ARGS",
    message="amount_cents must be an integer in cents. Received '42.50'. "
            "For $42.50, pass 4250.",
    retryable=True)

The retryable flag and the suggestion of what to do instead are what break loops. A model that receives "not found, and here is how to list valid IDs" takes a different action; a model that receives [] tries again.

Versioning and testing

Tool descriptions are prompt text, so they need the discipline on the prompts in git page:

def test_tool_selection():
    """Given a task, does the model pick the right tool?"""
    for case in load_cases("tools/selection_eval.yaml"):
        chosen = model_select_tool(case.task, registry.visible_for(case.request))
        assert chosen == case.expected_tool, \
            f"{case.id}: chose {chosen}, expected {case.expected_tool}"

def test_no_ambiguous_pairs():
    """Every pair of tools must be distinguishable from descriptions alone."""
    for a, b in combinations(registry.all_tools, 2):
        sim = embed_similarity(a.description, b.description)
        assert sim < 0.85, f"{a.name} and {b.name} are too similar ({sim:.2f})"

The pairwise similarity check is cheap and catches the problem before it reaches an eval set. Two tools whose descriptions embed to 0.9 similarity will be confused, and the fix is to add the explicit "does NOT do X, use Y" clause to both.

A worked example: a registry that grew for two years

An internal operations assistant. Started with 8 tools; two years and six teams later it had 63, added by whoever needed one, described by whoever wrote the function.

State before:

tools:                          63
tool definitions per request:   14,200 tokens
mean steps per task:            11.4
wrong-tool selections:          21% of calls
task success:                   58%
cost per task:                  $0.44
p50 latency:                    31s

An audit of the 63 found the shape of the problem:

tools never called in 90 days:              19
tools with descriptions under 10 words:     27
tools whose description was the function
  name restated ("get_user_data: Gets
  user data."):                             14
pairs with embedding similarity > 0.85:     11
tools returning bare [] on failure:         41
tools with no unit or format info on
  numeric or ID parameters:                 38

Nineteen tools that nothing had called in three months were still costing 4,300 tokens per request, paid 11.4 times per task.

Change 1: delete and merge.

deleted (unused 90 days):              19 -> 44 tools
merged (near-duplicate pairs):          5 pairs -> 39 tools

tool definitions per request:  14,200 -> 8,900 tokens
wrong-tool selections:         21% -> 16%
task success:                  58% -> 63%
cost per task:                 $0.44 -> $0.29

Deleting unused tools improved accuracy, which surprised the team. The unused tools were still in the choice set, still plausible-looking, and still occasionally selected.

Change 2: rewrite descriptions to the five-element template.

# Before
Tool("query_svc", "Queries the service.", {"q": "string"})

# After
Tool("search_service_catalog",
     description=(
         "Search the internal service catalogue by name, team or tag. "
         "Returns service name, owning team, on-call rotation, runbook URL "
         "and current deploy version. "
         "Use this when you need to find who owns a service or how to reach "
         "them. Does NOT return metrics (use query_metrics) or deploy "
         "history (use list_deploys)."),
     params={"query": "Service name, team name, or tag. Partial matches work."})

tool definitions per request:  8,900 -> 11,400 tokens   <- WENT UP 28%
wrong-tool selections:         16% -> 7%
task success:                  63% -> 79%
cost per task:                 $0.29 -> $0.24           <- still went DOWN

Longer descriptions cost more per request and less per task, because tasks completed in fewer steps: mean steps fell from 11.4 to 7.1. Paying 28 percent more prefix to eliminate half the wrong turns is a good trade, and it is a trade teams get wrong in the other direction by shortening descriptions to save tokens.

Change 3: structured errors.

@dataclass
class ToolResult:
    ok: bool
    data: Any = None
    error_type: str | None = None       # NOT_FOUND, INVALID_ARGS, RATE_LIMITED,
                                        # PERMISSION_DENIED, UPSTREAM_ERROR
    message: str | None = None          # written FOR THE MODEL
    retryable: bool = False
    suggestion: str | None = None       # "use lookup_orders(customer_id) instead"

mean steps per task:           7.1 -> 5.8
repeated-identical-call rate:  14% -> 2%
task success:                  79% -> 86%

The repeated-identical-call rate is the metric that moved most. Fourteen percent of tool calls had been exact repeats of a previous failing call, because a bare [] gave the model nothing to change.

Change 4: phase and permission gating.

PHASES = {
    "investigate": ["search_*", "query_*", "list_*", "describe_*"],   # 22 tools
    "act":         ["create_*", "update_*", "restart_*", "rollback_*"], # 11
    "escalate":    ["page_oncall", "create_incident", "notify_channel"], # 6
}

mean tools visible per request:  39 -> 14
tool definitions per request:    11,400 -> 4,600 tokens
wrong-tool selections:           7% -> 4%
task success:                    86% -> 91%
cost per task:                   $0.24 -> $0.11
p50 latency:                     31s -> 9s

Final:

                          before      after
tools registered            63          39
mean tools visible          63          14
tool defs per request    14,200      4,600     (-68%)
mean steps per task        11.4        5.8      (-49%)
wrong-tool selections       21%         4%
repeated identical calls    14%         2%
task success                58%        91%      (+33 points)
cost per task             $0.44      $0.11      (-75%)
p50 latency                 31s         9s      (-71%)

Thirty-three points of task success and three quarters off the cost, with no model change and no change to what the tools actually do. Every improvement came from the registry: what exists, what is visible, how it is described, and what it returns on failure.

The counterintuitive result worth carrying: making descriptions longer reduced total cost. Token-per-request went up 28 percent and tokens-per-task went down, because the agent stopped taking wrong turns. Optimising the prefix in isolation is optimising the wrong number.

Production evidence

Anthropic's tool use documentation recommends detailed descriptions explicitly, with guidance to describe what the tool does, when to use it, what it returns and how it differs from similar tools. Their stated position is that the description is the primary determinant of correct selection, ahead of the schema.

OpenAI's function calling guidance makes the same point and adds that overlapping function purposes are a common cause of wrong selection, recommending clear boundaries between functions.

MCP (the Model Context Protocol) standardises tool exposure with a name, description and JSON Schema, and its design assumes the description is the model-facing surface. See MCP.

Anthropic's "Building Effective Agents" and their writing on tool design argue for designing tools around the agent's tasks rather than mirroring an existing API, and note that "poka-yoke" tool design (making incorrect use structurally difficult) reduces errors more than instructions do.

Claude Code's tool set is small and coarse (read, write, edit, bash, glob, grep, web fetch) rather than a large set of primitives, which is a shipped example of the granularity argument: a handful of composable, well-described tools rather than dozens of specific ones.

The debate

How many tools should be visible? Under 15 per request, and the evidence is the selection-accuracy curve: 4 percent wrong at 6 tools, 18 percent at 40. Above 15, gate by phase and permission; above 50, retrieve over tool descriptions with a small always-visible core. The registry can be large; the visible set should not be.

Composite tools or primitives? Composites for routine tasks, primitives for the tail. A composite matched to a task boundary turns four calls into one, removes three opportunities for a wrong selection, and returns a coherent object rather than fragments the model must join. The cost is that composites multiply as tasks vary, and the discipline is to add one only when a call sequence is observed repeatedly. Designing tools around your data model rather than around the agent's tasks is the most common structural mistake.

Should descriptions be short to save tokens? No, and this is the trade teams get backwards. In the worked example, rewriting to longer descriptions raised per-request tokens 28 percent and lowered per-task cost, because mean steps fell from 11.4 to 7.1. Optimise tokens per completed task, not tokens per request. The exception is a single-shot classification with no loop, where the prefix is paid once.

Who owns tool descriptions? Not the engineer who wrote the function, by default. A description is prompt text with a measurable effect on behaviour, so it belongs to whoever owns the agent's quality, and it should go through the same review and eval gates as any prompt. The audit in the worked example found 14 descriptions that were the function name restated, which is what happens when the description is treated as a docstring.

Is the pairwise-similarity check worth it? It is cheap and it catches the specific failure that dominates: confusion between two adjacent tools. Eleven pairs above 0.85 similarity in a 63-tool registry is a lot of confusable choices, and the fix (adding an explicit "does NOT do X, use Y" to both) is mechanical. I would run it in CI, alongside a selection eval set.

Do error messages really matter that much? They were the single largest step-count reduction in the worked example, taking repeated-identical-call rate from 14 percent to 2 percent. A model that receives [] cannot distinguish "no results" from "bad query" and will retry the bad query. Tool errors are prompt text written for the model, and writing them for a human log reader is a missed opportunity that costs steps.

Follow-up Q&A

"Why does tool count affect accuracy?"

Two mechanisms. The definitions are prompt text sent on every step, so a large set consumes context and pushes the task and observations toward the positional trough. And the model is choosing among more options that plausibly match, so near-duplicates get confused. Measured on one agent: 4 percent wrong-tool rate at 6 tools, 9 percent at 12, 18 percent at 40. The fix is reducing the visible set per request through phase and permission gating, not shortening the descriptions.

"What makes a good tool description?"

Five things: what it does in the caller's vocabulary, what it returns specifically enough to know whether it answers the question, when to use it, what it does not do with the neighbouring tool named, and constraints including side effects and idempotency. The fourth element is the one that most improves accuracy, because most wrong-tool errors are between two adjacent tools, and naming the boundary resolves exactly that.

"Should tools mirror your API?"

No. An API is for a programmer who reads documentation and holds context across calls; a tool is invoked by a model that sees only the description, mid-task, with no memory. The right surface is usually coarser: one get_customer_order_summary beats four primitives, because it turns four calls into one, removes three chances of a wrong selection, and returns a coherent object rather than fragments the model must join. Add composites for routine tasks and keep primitives for the tail.

"How do you handle more than 50 tools?"

Gate the visible set. Phase gating (investigation tools versus action tools), permission gating (never show what the caller cannot use, since filtering after selection wastes a step and produces a confusing observation), and above about 50, retrieval over tool descriptions with a small always-visible core so a retrieval miss cannot hide the tool the task needs. In one case that took the mean visible set from 39 to 14 and raised success 5 points.

"What should a tool return on failure?"

A structured result with an error type, a message written for the model, a retryable flag, and a suggestion of what to do instead. A bare empty list is the worst case because it conflates "no results" with "bad query," and the model retries the bad query. Fixing this took repeated-identical-call rate from 14 percent to 2 percent in one system and was the largest single reduction in step count.

"Are longer descriptions worth the tokens?"

Measured, yes. Rewriting descriptions to a fuller template raised per-request tokens 28 percent and lowered cost per completed task, because mean steps fell from 11.4 to 7.1. The metric to optimise is tokens per completed task, not tokens per request, and shortening descriptions to save prefix is the trade teams get backwards.

Common misconceptions

"Tool descriptions are documentation." They are prompt text with a measurable effect on selection accuracy, sent on every step, and they should be reviewed and tested like any prompt. A description that restates the function name is a defect.

"Expose the API as tools." APIs are designed for programmers with context and documentation. Tools are for a model with neither. The correct granularity is usually coarser and organised around tasks rather than around your data model.

"More tools means more capability." Past roughly 15 visible, selection accuracy degrades measurably and the prefix cost is paid on every step. Deleting 19 unused tools improved accuracy in the worked example.

"Shorter descriptions save money." They save prefix tokens and cost steps. Optimise per completed task.

"Returning an empty result is fine for failures." It gives the model no way to distinguish "nothing matched" from "your arguments were wrong," so it repeats the same call. Structured errors with a suggestion are what break the loop.

Interview delivery note

Say this verbatim: "A tool description is prompt text sent on every step, not documentation, so I write it with what it returns, when to use it, and explicitly what it does not do with the neighbouring tool named, because most wrong-tool errors are between two adjacent tools. And I keep the visible set under about 15, because wrong-tool rate went from 4 percent at 6 tools to 18 percent at 40." The reframing plus the specific technique plus the measured threshold.

The senior-versus-staff separator is optimising tokens per completed task rather than per request. A senior engineer writes good tool descriptions. A staff engineer measures that lengthening descriptions raised per-request tokens 28 percent and lowered total cost, because mean steps fell from 11.4 to 7.1, and can state that shortening descriptions to save prefix is optimising the wrong number.

The second signal is treating tool error messages as prompt surface. Saying "a bare empty list conflates no-results with bad-query, so the model retries the bad query, and fixing that took repeated identical calls from 14 percent to 2 percent" shows you have watched an agent loop and traced the loop to a tool rather than to the model.

The Staff & Team Lead Interview Reference