Tool Design and MCP Integration
How to shape tool contracts so Claude chooses the right tool, recovers cleanly from failures, sees only the tools it needs, and connects to external systems through MCP without leaking secrets or wasting turns.
Tool design and MCP integration is where the exam tests whether Claude can act effectively in the world through the contracts you publish. The model never sees your backend code. It sees the tools array you send with each Messages request, where every tool carries a name, a description, and an input_schema that together determine both selection and execution. When those contracts are vague, overlapping, or too numerous, selection degrades, errors become unbranchable, and the agent burns turns recovering from misrouting that a few precise sentences would have prevented.
The domain is the tooling counterpart to agentic orchestration. Orchestration asks how you sequence capabilities and route information between agents. Tool design asks which capabilities you expose at all, how you shape their schemas so they are distinguishable, how you handle the four classes of failure so the agent knows whether to retry or to pivot, how you distribute tools across agents at four to five per turn, and how MCP servers are integrated so that shared configuration stays credential free and standard systems are not rebuilt as custom first steps. Expect scored scenarios that blend a minimal description with a plausible classifier, a generic error string with a retry loop, or an overloaded single agent with a second server that does not actually reduce the flat list.
This guide follows the five tasks exactly as tested. Task 2.1 makes selection reliable by writing production grade descriptions and fixing system prompt coupling. Task 2.2 gives every failure a structured envelope that separates access failure from valid emptiness and transient from validation, business, and permission. Task 2.3 reduces choice complexity by scoping per role, consolidating homogeneous variants into one parameterised tool, and using tool_choice modes correctly. Task 2.4 places MCP configuration where it belongs, keeps secrets out of the shared file with expansion, prefers community servers for standard systems, and makes MCP tools selectable through richer descriptions and Resources. Task 2.5 enforces the built in hierarchy of Grep for inside, Glob for names, and Edit with a unique anchor before Read plus Write, all through incremental discovery rather than bulk reading.
Tool Schema Design
Write production grade tool descriptions and strict schemas so Claude selects the right tool without guessing and executes it without type errors.
What you need to know
Tool descriptions are the primary mechanism through which Claude selects which tool to call for a given user request, and the interface definition as a whole determines whether that selection can be executed without error. The model never sees your backend implementation. It sees the JSON Schema you publish in the tools array of the Messages request, where each tool carries a name, a description, and an input_schema that describes its typed parameters. Selection and parameter synthesis flow from those three fields, and description quality dominates the selection half while schema hygiene dominates the execution half.
In current deployments, before any tool call is issued the model receives the full list of tool definitions alongside the conversation history and the system prompt. Claude's tool use reference describes the tools parameter as a list of tool definitions with name, description, and input_schema, and notes that while the schema is required for inference the descriptions are where you steer the model toward precise behaviour. When two tools have overlapping capability and their descriptions are minimal, for example get_customer described only as Retrieves customer information and lookup_order described only as Retrieves order details, the model cannot differentiate which to use when the user says check my order number 12345. Both read as retrieval tools over entities identified by an identifier like value, and with no distinguishing input formats, example queries, or boundary statements, selection defaults to plausibility rather than correctness, leading to systematic misrouting where order queries are dispatched to the customer tool.
A production grade description removes the guesswork with five elements, what the tool does stated unambiguously, what inputs it expects including formats and required versus optional fields, concrete example queries the tool handles well that anchor pattern matching, edge cases and limitations including what the tool does not do, and explicit boundaries that say when to use this tool versus any similar one in the set. Our material illustrates the difference directly. The minimal get_customer line gives no identifier type, no return shape, and no boundary, while the production variant states that it looks up a customer account by email, phone, or customer identifier, returns name, contact details, account status and loyalty tier, is for identity verification, and explicitly says do not use it for order queries where lookup_order should be used instead. The production variant for lookup_order mirrors that shape, stating that it requires order number in format number NNNNN or tracking identifier, returns order status, items, shipping details and refund eligibility, and excludes identity verification work.
Beyond individual description quality, the distribution of overlap matters. A generic analysis tool such as analyze_document that claims to analyse a document and return results is a single overloaded entry point for at least three distinct jobs that deserve distinct contracts, extracting structured data fields, summarising key arguments, and verifying whether a claim is supported by the source. Splitting the generic tool into extract_data_points, summarize_content, and verify_claim_against_source, each with a narrow description and typed input output contract, makes the selection decision crisp because the task description now aligns with one tool rather than with a vague umbrella. Renaming is a lighter version of the same fix. When two tools have confusingly similar names, giving one a behaviour anchored name such as extract_web_results and rewriting its description to be web specific resolves overlap at the interface without touching implementation.
System prompt wording can silently override good descriptions, which is why interface design cannot be reviewed in isolation. If the system prompt contains keyword sensitive instructions such as always check customer details before proceeding, the model may route any customer phrasing to get_customer regardless of retrieval context, creating an unintended tool association that no description broadening can correct without also revising the prompt. After improving descriptions, the correct step is to review the prompt for such cues and remove or clarify them, ensuring prompt and interface convey the same routing policy.
The decision framing that extends beyond this task is equally important. Descriptions are the fix when the agent has a workable number of tools and simply cannot tell two apart. When the toolkit is large, rewriting descriptions does not rescue selection because the agent is past the point where decision quality depends on wording and must instead have its tool count reduced, which is the subject of Task 2.3. Recognising which regime you are in prevents description work where distribution work is needed and vice versa.
How tool definitions travel to Claude
Tool definitions are transmitted in the Messages API request under the tools key as an array of objects, each containing name as a unique snake case identifier, description as free form natural language that the model uses to decide whether to call this tool, and input_schema as a JSON Schema object constraining what arguments Claude can pass. The input_schema uses standard JSON Schema keywords including type object, properties with per property type, description, enum, format, pattern, minimum, maximum, additionalProperties, and required as an array of keys that must be present. Optional flags on the tool object include strict as a top level boolean alongside name, description, and input_schema, which when set to true constrains token sampling so the resulting input field in the tool_use block is guaranteed to match the declared schema, with only the shape guaranteed not business validity.
The model response for a tool call is a message whose content array contains one or more tool_use blocks, each with type tool_use, id, name, and input matching the schema, and the stop_reason is tool_use when at least one such block is present. Claude returned tool results are sent back as user role messages containing tool_result blocks with tool_use_id copied from the originating block and content as a string, optionally with is_error when the tool signalled failure. This surrounding contract matters to schema design because the description and property level description fields are the only content the model sees before deciding to emit a tool_use block. Precision in those fields is therefore not documentation polish, it is the runtime control plane.
Schema hygiene that prevents execution failure
Every property should carry its own description with format guidance and examples, for example User identifier in USR dash XXXXX format such as USR dash 48721, and any parameter with a fixed set of valid values should use enum rather than relying on prose, because without an enum the model may invent values outside the allowed set. Required arrays should list only truly required fields, optional parameters with sensible defaults should be omitted from required so Claude can skip them, integer and number should be distinguished precisely, nesting should remain shallow with no more than two levels, and additionalProperties should be closed when stray keys would break backend validation.
A well defined schema that is also vague in description still misroutes, and a well described tool with a loose schema still fails at execution with type mismatches or missing fields. Strict tool mode as grammar constrained sampling guarantees that the input in the tool_use block matches the input schema exactly, including enum and type correctness, but it does not guarantee business validity. Hygiene and description are therefore a pair, and either alone leaves one half of the correctness equation uncovered.
From one vague tool to crisp selection via splitting and renaming
When a single tool must cover extracting data points, summarising, and verifying claims, its description cannot be crisp for any one of those jobs. The tool selection problem is then structural. Splitting replaces one generic tool with several narrow, well named tools, each with its own contract. After splitting, a summary task aligns with summarize_content and an extraction task aligns with extract_data_points, so selection is a match rather than a guess. This is a low infrastructure fix compared with adding a routing classifier that bypasses model judgement entirely.
Renaming is the lightest version of the same interface repair. When two tools have confusingly similar names, giving one a behaviour anchored name such as extract_web_results and rewriting its description to be web specific changes the lexical trigger the model pattern matches on. Keeping the original name and lengthening the description leaves the name ambiguity in place, so the fix must include the name change rather than only prose growth.
Prompt coupling and when wording stops rescuing selection
System prompt wording can reintroduce the ambiguity a description rewrite just removed. Keyword sensitive instructions that mention customer details or always check a particular entity create an unintended association that persists even after the description adds an explicit boundary. The interface fix must therefore be validated against prompt wording before being declared complete, with conflicting prompt phrasing removed or narrowed and stability remeasured on the same benchmark.
The four to five tool ideal and the hard ceiling beyond eighteen tools for a single agent establish a second regime. Where tools are few enough to handle yet two read alike, sharpen the descriptions as described above. Where many tools share a shape such as data in plus operation plus data out, consolidate into one parameterised tool rather than rewriting eighteen descriptions. Moving servers without merging does not change the flat list the model reasons over, so distribution must be evaluated on the merged prompt rather than per server.
Authoritative mechanism reference
The sections below expand the tested rules into the complete documented surface: ownership, lifecycle, version differences, production boundaries, and adjacent mechanisms.
Mechanism reference
This section documents every mechanism the task touches, at full depth, with exact field names, value spaces, ownership, and boundary conditions.
Mechanism reference: Tool definition anatomy
A tool is an object passed in the tools array of a POST /v1/messages request. The Messages API is the transport, and the tool definition is the contract that constrains what the model may emit inside a tool_use block. Three fields are required and always present. Every additional field is optional and layered on top without changing the core three.
name is a string that must be unique among the tools in the request and must use snake case. The API validates uniqueness. At runtime the model emits the same string in tool_use.name, and application code routes on it. The lesson set is explicit that a mismatch between the name declared in tools and the name forced via tool_choice produces a validation error before inference runs. A name such as get_customer versus lookup_order carries load-bearing information because the model uses both name and description together to decide relevance.
description is a free-form string with no length limit enforced by the schema, but with a token budget that matters in practice. Every token in every description is loaded into context on every turn. The lesson tool-definition-schemas quantifies a set of four to five well-described tools at roughly 17,000 to 18,000 tokens of context cost, which makes description verbosity a real constraint rather than a style preference. The guidance for what belongs in this string has been stable across documentation versions: state purpose unambiguously, state expected inputs with formats, give concrete example queries, enumerate edge cases and what the tool does not do, and state explicit boundaries versus sibling tools. The reference page lists exactly those five elements, and the lesson tool-definition-schemas covers them as the four jobs of a description plus boundary guidance, while agentic-eng-action-tools adds side effects as an explicit item to document.
input_schema is a JSON Schema object that must have type: "object". It is not a loose bag of hints. It is a schema that can be enforced by the API when strict mode is used, and it governs validation on every tool_use.input the model produces. The canonical shape is:
{
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "City name with optional state or country, e.g. 'San Francisco, CA' or 'Tokyo, Japan'"
},
"units": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "Temperature unit. Defaults to celsius if not specified."
}
},
"required": ["location"]
}Only type and properties are strictly required to form a valid schema, but a production schema also includes required and description per property, and often enum, additionalProperties, and type refinements. The sections below enumerate each subfield.
The reference page, the lesson, and the API documentation agree that name, description, and input_schema are the three load-bearing fields and that description dominates selection accuracy.
Mechanism reference: input_schema design at field-level depth
input_schema follows the JSON Schema object specification. The model treats it as a set of constraints on tool_use.input. The constraints listed here are the ones that exist in the API and that interact with strict mode.
type must be "object" at the top level. Individual properties use "string", "number", "integer", "boolean", "array", or "object". The distinction between integer and number is load bearing. The lesson tool-definition-schemas requires matching the schema type exactly to what the backend expects, because a schema that says integer but a backend that parses a string will accept the call and then fail on arrival. Where a value must be an integer identifier such as order_number, declare integer, not number. Where decimals are valid, use number.
properties is a map from parameter name to a schema fragment. Every property should include its own description that states format, constraints, and an example. The lesson explicitly shows "User ID in USR-XXXXX format, e.g. USR-48721" as the pattern. A property without a description leaves the model to infer format from the name alone, which is the same failure mode as a tool without a description, only at smaller scope.
required is an array of property names that must be present in tool_use.input. Only list a parameter here if the tool will fail without it. Optional parameters with sensible defaults must be omitted from required so the model can skip them when they are not relevant. An empty required array is valid and means every property is optional.
enum is an array of literal values that constrains a property to a closed set. Use it whenever a parameter has a fixed vocabulary. The lesson is explicit: do not rely on description alone for fixed values, because the model may still invent a value not in the allowed set. An enum of ["celsius", "fahrenheit"] prevents the model from emitting "metric" or "centigrade". The same pattern applies to status fields, action verbs, and tier names.
additionalProperties controls whether the model may emit properties not declared in properties. Setting it to false closes the object and forces the model to emit only declared keys. This is the recommended setting under strict mode, because open schemas leave a gap where unexpected keys can pass shape validation and then fail business validation downstream.
Nesting depth must be kept shallow. The lesson advises never exceeding two levels of nested objects, because deeper nesting reduces invocation reliability. Flatten complex structures where possible. If a tool logically needs a nested object, consider splitting the tool instead.
description per property is where identifier format belongs. A property named order_number with description "Order number in #NNNNN format, e.g. #12345" prevents the model from passing a raw integer or a tracking ID into the wrong field. The description is also where defaults are documented: "Temperature unit. Defaults to celsius if not specified."
Validation boundary: when strict mode is off, the model is guided by the schema but not constrained by it, so a required field can still be omitted or a value can arrive as string "2" instead of integer 2. When strict mode is on, the API constrains sampling so that only schema-valid outputs are possible, using grammar-constrained sampling. Strict mode guarantees shape, not business validity. A well-typed customer_id that references an account the caller cannot access still passes shape validation and must be rejected by application authorization code.
Mechanism reference: Optional top-level tool fields
Beyond name, description, and input_schema, the API defines a set of optional fields that layer capabilities onto the same definition without changing the core contract. The prompt for this task enumerates type, strict, defer_loading, input_examples, allowed_callers, cache_control, and eager_input_streaming as the fields to account for. Each is documented here with its exact semantics.
type identifies the tool kind. Most tools submitted by applications are custom tools (the default when omitted). Built-in server tools carry their own type values, such as web_search or code_execution, and are not declared with a full input_schema by the caller in the same way. For a custom tool, omitting type or setting it to "custom" is equivalent and has no effect on selection. Specifying a server-tool type without the corresponding beta or capability flags produces a validation error.
strict is a boolean flag set at the same level as name and description. When true, the API enforces the input_schema with grammar-constrained sampling so that tool_use.input is guaranteed to match the schema and the emitted tool_use.name is guaranteed to be one of the tools provided. The constrained schema subset in strict mode is the same subset used for structured outputs. Schemas that use unsupported constructs (deeply nested conditionals, some anyOf or oneOf patterns, or recursive references) must be simplified before strict mode accepts them. Strict mode does not replace server-side business validation.
defer_loading is a boolean that marks a tool as deferred. A deferred tool is not loaded into the model's context at the start of the turn. Only non-deferred tools and a lightweight tool_search capability are visible initially. When a task needs a deferred capability, the model searches by name or intent and only then is the matching definition expanded into context. This addresses context bloat when an application exposes many tools. An application can expose 50 or more tools while the model only pays the token cost for the ones it actually searches for and uses on a given turn. defer_loading is a beta capability and requires the corresponding beta header as of this writing.
input_examples is an array of 1 to 5 realistic sample invocations showing minimal, partial, and fully specified parameter combinations. These examples are attached to the tool definition and teach the model conventions that a JSON Schema cannot express, such as date formats, identifier prefixes, and which optional fields travel together. An input_examples entry is a complete input object that would be valid according to the input_schema. The lesson emphasizes that this field is distinct from few-shot prompting in the system prompt. Few-shot in the prompt adds tokens on every turn and does not fix why the model is confused, while input_examples on the tool definition adds targeted format guidance that travels with the schema and is consulted during selection.
allowed_callers controls whether a tool may be called from code execution sandboxes. The current documented value is ["code_execution_20250825"], which opts a tool into programmatic calling from Claude-written code running in the sandbox. When opted in, multi-step orchestration and filtering can happen inside code rather than as a chain of individual tool_use and tool_result round trips, which keeps bulky intermediate results out of the context. Without this flag the tool is only callable as a normal tool_use block. The flag interacts with the code execution tool family.
cache_control marks a tool definition for prompt caching. Setting cache_control: { "type": "ephemeral" } on a tool allows its prefix in the request (typically the tools array and system prompt) to be cached across turns so that repeated calls pay only for the uncached suffix. Tool definitions are a natural fit for caching because they are large, stable, and repeated verbatim. The token savings are most visible when the tool set is large and descriptions are detailed. Caching does not change selection semantics. It changes billing and latency.
eager_input_streaming is an optimization that allows the model to begin streaming tool_use.input field values before the full block is complete. The flag is only meaningful when streaming responses are consumed incrementally and the application can act on partial inputs (for example to pre-validate or prefetch). It does not change the final validation of the input against input_schema. When absent or false, the input is delivered atomically at the end of the tool_use block.
Mechanism reference: tool_use and tool_result shapes
The tool definition governs what the model may produce. The tool_use block is what the model actually produces, and tool_result is what the application returns. These three objects form a matched triple and share identifiers that must align exactly.
tool_use block. The model emits this inside the content array of an assistant message when stop_reason is tool_use. The shape is fixed:
{
"type": "tool_use",
"id": "toolu_01ABC123",
"name": "get_weather",
"input": {
"location": "Tokyo, Japan",
"units": "celsius"
}
}Three fields are always present. id is a unique identifier prefixed with toolu_. It is generated by the API and is the correlation key for the round trip. name is the tool name from the definition, and with strict mode enabled it is guaranteed to be one of the tools provided. input is the parameter object constructed according to input_schema, with the same type coercions and enum constraints the schema declared. When strict is true the input is guaranteed to match input_schema, otherwise it is best effort and application code must validate. Multiple tool_use blocks may appear in a single assistant message, and they may be interleaved with text blocks that contain a brief explanation such as "Let me check the weather for you".
The id prefix is load bearing. The application must not invent an identifier or reuse one across blocks. Each tool_use has its own id, and each result must echo that exact string.
tool_result block. The application emits this inside the content array of a user message that follows the assistant message containing tool_use. The ordering requirement is strict: the assistant message with tool_use blocks must be appended to the message history before the user message with tool_result blocks, otherwise the API rejects the request. The shape is:
{
"type": "tool_result",
"tool_use_id": "toolu_01ABC123",
"content": "Tokyo: 22C, Sunny, 55% humidity, Wind 15 km/h",
"is_error": false
}tool_use_id must match the originating id character for character. content carries the tool output. It may be a string or an array of content blocks. The API does not prescribe a schema for the output, the application chooses what to return, but lesson tool-definition-schemas and agentic-eng-action-tools both require that the description of the tool already told the model what the output will look like so it can interpret the result and decide what to do next. is_error is optional. When true it marks the result as an error and the lesson tool-choice-deep-dive recommends handling it by switching tool_choice to none to force a textual explanation rather than retrying the same failing tool.
Matching contract for parallel calls. If the model emits three tool_use blocks in one turn, the application must return three tool_result blocks in one user message, each with its own tool_use_id. A common bug is to return a single result for multiple calls or to split results across multiple user messages before re-querying the model. The loop must collect all results and send them together.
stop_reason values. The response's top-level stop_reason tells the application what to do next. The four values seen in a basic tool-use loop are end_turn (model finished, no pending calls, display text), tool_use (model wants to call tools before continuing, extract blocks and execute), max_tokens (response was cut off at the max_tokens limit, any tool_use at the tail may have an incomplete input and must be validated before execution), and stop_sequence (a custom stop string was hit, verify completeness). Three additional values apply to agents that use server tools or run near limits: pause_turn when a server-side tool hits its iteration cap and the correct action is to send the assistant content back as is to let it continue, refusal when the model declines for safety reasons with stop_details indicating the policy and the mitigation being a retry on a fallback model, and model_context_window_exceeded when the model runs out of context mid-generation, distinct from the client-side max_tokens cap. Ignoring stop_reason and assuming every turn contains tool calls, or assuming every turn ends the conversation, are both common loop bugs.
Mechanism reference: tool_choice interaction with schema design
tool_choice does not live inside the tool definition. It lives at the request level alongside tools and messages, but it directly governs how the model consults the definitions. Four values exist and each fundamentally changes termination behavior.
auto lets the model decide whether to use a tool. Either tool_use or end_turn is possible. This is the correct default for general-purpose assistants where some requests are purely conversational. The lesson set is consistent that auto also preserves the model's ability to emit a short explanatory sentence before a tool call, while forced modes do not. Leaving tool_choice unspecified is equivalent to auto.
any forces the model to use one of the available tools on every turn. No text-only response is allowed, and stop_reason will always be tool_use. This is correct for classifiers and routers that must act on every input, but it requires a maximum turn guard in any loop, because the loop will never terminate on its own if any is held constant.
none suppresses all tools. The model responds with text only and stop_reason is always end_turn. This is correct for greeting turns, confirmation prompts before destructive actions, and error explanations after a failed tool result.
Specific tool, expressed as {"type": "tool", "name": "get_weather"}, forces the model to call exactly that tool. The named tool must exist in tools or the request fails validation. This is correct for pipeline steps where the caller must guarantee which function runs.
Forcing via any or a specific name prefills the assistant turn to guarantee a tool call, which suppresses leading text and reasoning that would otherwise appear before the tool_use block. The lesson notes that this is observable and that applications which need a sentence of reasoning before a forced call should capture reasoning via the extended thinking channel or switch to auto with a strong nudge. The ordering implication for schema design is that descriptions must do their work without relying on free-form reasoning that only appears under auto.
Mechanism reference: Misrouting taxonomy
With the field-level mechanics in place, the failure modes the reference page describes can be named precisely.
Ambiguity misrouting occurs when two descriptions lack boundary statements. The model sees get_customer: "Retrieves customer information" and lookup_order: "Retrieves order details" and treats them as near synonyms. A user request containing both a customer signal ("check my order") and an order identifier ("#12345") provides no disambiguation through the description alone, so the model picks arbitrarily. The fix is to add accepted identifier formats, returned fields, and an explicit do-not-use clause to each description.
Granularity misrouting occurs when a single tool handles multiple distinct intents. analyze_document: "Analyses a document and returns results" conflates extraction, summarization, and verification. Even with a perfect description it remains one tool with one name, so the model cannot express which intent the caller actually wants. Splitting into three tools with distinct names and narrow schemas removes the ambiguity.
Name-collision misrouting occurs when the name alone is confusing. analyze_content tells the model nothing about whether the tool operates on web results, local files, or conversation history. Renaming to extract_web_results disambiguates at the name layer without changing implementation, because name and description are consulted together.
Prompt-override misrouting occurs when a system prompt contains a keyword that implicitly favors one tool. A prompt that says "always check customer details before proceeding" creates an association between any mention of a customer and the get_customer tool, even when the user intent is order tracking. Updating descriptions without auditing the prompt leaves this override in place.
Cardinality misrouting occurs when the tool count itself is the problem. The reference page and the lesson set agree that four to five tools is the sweet spot, six to eight is manageable only with carefully differentiated descriptions, and beyond that reliability drops. At very large counts the model's decision space is too wide for description quality to rescue, and the correct remedy is architectural, such as introducing a router tool that dispatches inside the application. Task 2.3 addresses that threshold and response.
Mechanism reference: Batch endpoints and parallel variants
The lesson tool-definition-schemas adds two scale patterns that are directly relevant to schema design because they determine whether the caller needs a new tool, a new parameter, or no change at all.
Batch parameter execution replaces N sequential calls that differ only by a parameter value with one call that accepts an array of values and returns an array of results. Three calls to run_analysis at two minutes each cost six minutes wall-clock when serialized, while one call to run_analysis_batch with scenarios: ["A", "B", "C"] completes in roughly two minutes by running server-side concurrently. The schema change is to replace a singular parameter with an array type and to declare the return as an array of corresponding results. Batch endpoints also require input size limits at the API layer so that an oversized batch is rejected or paginated rather than timing out.
Parallel variant over boolean switches. When a lightweight version of an existing tool is needed, adding a new tool such as search_orders_summary alongside search_orders_full is clearer to the model than adding a boolean includeDetails switch to one tool. Each variant gets its own description, input_schema, and usage guidance, the summary variant keeps the model from pulling heavyweight fields it does not need, and existing callers retain backward compatibility because no signature changed.
Mechanism reference: Output specifications and context hygiene
Tool outputs are not declared in input_schema. Whatever the application returns in tool_result.content is the output. The discipline is to describe the output in the description so the model knows what to expect and what to do next, and to keep the output itself concise and structured. The lesson advises returning summaries or paginated results rather than dumping full documents into the context window, and enforcing input size limits so that wide queries do not flood the window on the way in either. Error outputs deserve the same discipline, with a clear statement of what went wrong, why, and what to try next, rather than a raw stack trace.
Ownership map
Each layer in the stack owns a distinct set of guarantees. Mixing ownership causes bugs where the model is blamed for a backend validation failure or the application is blamed for a sampling constraint the API should have enforced.
| Guarantee | Owner | What happens if the owner is bypassed |
|---|---|---|
name uniqueness and snake case shape | API validation on POST /v1/messages | Request is rejected before inference with a validation error, the model never sees the malformed definition |
input_schema shape enforcement when strict: true | API grammar-constrained sampling | Without it, required fields can be omitted and values can arrive with wrong primitive types such as string "2" instead of integer 2 |
description quality and boundary statements | Application developer, at definition time | Model picks arbitrarily between overlapping tools, producing the misrouting failure the reference page centers on |
tool_use.id generation and toolu_ prefix | API runtime | Application code must not invent identifiers, the API guarantees uniqueness per block |
tool_use.input construction according to input_schema | Model, constrained by schema and strict | Without strict the model may emit a close but invalid shape that application code must catch |
tool_result.tool_use_id matching and content shape | Application loop code | A mismatch causes the API to reject the follow-on request, a wrong ordering causes a history error |
stop_reason interpretation and loop termination | Application loop code | Ignoring tool_use leaks raw JSON to the user, ignoring end_turn produces an infinite loop, ignoring max_tokens executes a truncated input |
tool_choice policy (auto, any, none, specific tool) | Application request, per turn | Setting any without a max turn guard loops forever, setting none when a tool is needed causes hallucination, forcing a nonexistent name fails validation |
defer_loading search and expansion | Model plus API context management | Without it, large tool sets blow the context budget, with it the model must know to search before calling a deferred tool |
allowed_callers code-execution dispatch | API sandbox plus application registration | A tool without the flag cannot be called from sandbox code, with it large intermediate results stay out of the context |
cache_control billing and reuse | API caching layer | No semantic change, but repeated tools prefixes become cacheable and latency drops |
eager_input_streaming partial delivery | API streaming transport | Without it inputs arrive atomically, with it the application can pre-validate or prefetch from partial fields |
| Business validity of a well-typed identifier | Application authorization and domain logic | strict guarantees shape, not authority, an authorized check must still happen before side effects |
| Output conciseness and truncation | Application tool handler | Dumping full documents into tool_result.content floods the window and degrades the next reasoning step |
The ordering of tool_use and tool_result is worth calling out separately because it is a common source of "API error on the second turn" reports. The history must read assistant message containing tool_use blocks, then user message containing tool_result blocks, then the next assistant message. Swapping the two or appending a tool_result without first preserving the assistant message breaks the required alternation.
Version and terminology currency
The reference page, the lessons, and the current Anthropic documentation use nearly identical vocabulary for the core triple, but surrounding terminology has moved. Candidates should know both forms to avoid hesitation on phrasing.
Tool versus function. The exam guide and early documentation sometimes say "function calling" while current docs say "tool use." These refer to the same mechanism. Tool definitions in the Messages API are functions the model may call. The field tools on the request and the block tool_use in the response are the current names, with function persisting only in informal description.
strict and grammar-constrained sampling. The reference page does not use the word strict, but the lesson tool-definition-schemas covers strict: true as a top-level tool property that places the sampling path under a schema-constrained decoder. This capability postdates the original tool-use launch which described input_schema as guidance rather than a guarantee. The currency note is that older writing says the model "should produce valid input according to the schema" while current writing with strict says the model "will produce only schema-valid input." The two statements describe the same system before and after the flag is enabled.
Advanced tool use additions. Three capabilities described in lesson tool-definition-schemas under the heading Advanced Tool Use: Search, Examples, and Programmatic Calling are beta additions that layer on top of the core contract. They map to the optional fields documented above: defer_loading for the Tool Search Tool, input_examples for Tool Use Examples, and allowed_callers for Programmatic Tool Calling. The lesson notes they are opt-in behind a beta header as of its writing, and that they are not prerequisites for correct basic definitions. Candidates who read only the reference page will not have seen these fields by name. The correct exam interpretation is that they are scaling solutions for large tool libraries, not prerequisites for fixing misrouting among a small set.
Tool Search Tool naming. The documentation heading "Tool Search Tool" and the flag defer_loading: true describe the same feature. An older informal description might say "deferred tools" while newer docs say "Tool Search." Both mean tools that are searchable rather than eagerly loaded.
cache_control and prompt caching. Prompt caching is documented as a distinct optimization from schema design, but it is included in the tool enumeration for this task because detailed descriptions are expensive. Lesson tool-definition-schemas lists cache_control alongside strict and defer_loading as part of the production surface, and Anthropic docs describe caching as applicable to tool definitions that are repeated verbatim.
Exam-guide versus current product names. The exam guide may refer to the tool set in terms of "functions" or describe batching and routing in prose, while the product surfaces concrete flags and tool kinds. The candidate should answer with the documented behavior, using whichever term the question stem uses, and should not treat a term mismatch as a trick.
Official versus community divergence
Where community material contradicts Anthropic documentation on this task, the documentation governs the answer. The divergences below are the ones relevant to tool schema design.
Few-shot as a first fix. Community explanations sometimes recommend adding five to eight few-shot examples to the system prompt as the primary remedy for misrouting. The official documentation and the lesson material both treat few-shot as symptom treatment that adds token overhead without fixing why the model is confused. The correct first step for the reference scenario is to expand descriptions. The community recommendation is not wrong in isolation, but the reference page explicitly marks it as the exam trap for this scenario.
Routing classifier as a first step. Community patterns sometimes jump to a dedicated routing layer that parses input before each turn and pre-selects a tool. Anthropic guidance and the lesson set both characterize this as over-engineered as a first step because it bypasses the model's natural language understanding and adds infrastructure the scenario did not require. The router tool pattern inside the application is endorsed as a cardinality solution, not as a misrouting fix for two ambiguous descriptions.
Tool consolidation versus splitting. Some community write-ups advocate consolidating get_customer and lookup_order into a single lookup_entity tool that dispatches internally. The reference page states this is valid as long-term architecture but wrong as a first step in front of a misrouting test where expanding descriptions costs far less. Lesson tool-definition-schemas reinforces that a new parallel tool is often preferable to a boolean switch on an existing one, but only when the need is for a lightweight variant, not as a way to paper over ambiguous descriptions.
Tool count thresholds. Community material sometimes quotes a single magic number for when selection breaks. The official-adjacent lesson material gives a graduated scale rather than a cliff: one to three tools is highest reliability, four to five is the sweet spot at moderate token cost, six to eight is manageable only with carefully differentiated descriptions, and nine or more is where selection errors increase significantly, with 18 stated as the hard ceiling where the model reliably fails. The exam tests the four to five ideal and the deferred routing response for large sets, not a single hard threshold in isolation.
Strict versus description quality. A community misconception holds that strict: true fixes misrouting. Strict guarantees that the emitted input matches input_schema in shape. It does not fix which tool the model chooses. That choice is driven by name and description. A strict schema with vague descriptions still misroutes reliably, it just misroutes with well-formed arguments.
Prompt wording as neutral. Community material often treats the system prompt as orthogonal to tool selection. Both the reference page and the lesson on tool_choice emphasize that keyword-sensitive instructions can silently override descriptions, and that only auto preserves explanatory text before a tool call. Leaving the prompt untouched after a description rewrite is explicitly identified as a trap.
The candidate's decision rule is simple. When a choice in a practice or exam scenario has one answer grounded in Anthropic docs or lesson guidance and another grounded only in a community pattern, pick the documented behavior and note the divergence.
Beyond the task statement
The reference page keeps focus narrow: descriptions fix misrouting among a workable number of tools, and system prompts can override them. The lesson set covers adjacent material that the reference page omits entirely, and each of these is relevant to how a candidate reasons about schema design in production.
JSON Schema depth and strict guarantees. The lesson tool-definition-schemas devotes a full subsection to per-property description, enum, required, additionalProperties, and shallow nesting. This is the vocabulary the exam uses to test whether a schema will actually constrain the model or merely hint at what it should do. Knowing the strict subset and that it uses the same constrained decoding as structured outputs matters for any scenario where a malformed call would throw at runtime.
The full stop_reason enum and tool_choice interplay. Lesson tool-use-blocks enumerates seven stop reasons, not just end_turn and tool_use, and lesson tool-choice-deep-dive shows how each tool_choice value maps to which reasons can appear and whether the loop can terminate naturally. A candidate who only knows end_turn versus tool_use will misdiagnose pause_turn as an error and will not understand why any loops forever without a max turn guard.
Forcing suppresses reasoning. The detail that any and specific-tool forcing prefill the assistant turn and suppress explanatory text, even when the prompt says "explain first," is omitted from the reference page but documented in tool-choice-deep-dive and tied to the extended thinking channel. This is the mechanism behind the prompt-override trap.
Batch and parallel variant patterns. Lesson tool-definition-schemas covers batch endpoints and parallel lightweight variants as scale responses. These are schema design decisions that determine whether the correct remediation is a schema edit, a new tool, or a new parameter. The reference page's splitting advice is a special case of this broader set.
4 to 5 tool rule and router pattern. The token cost table and the recommendation to use a router tool when the set exceeds the sweet spot appear in tool-definition-schemas but not on the reference page's core path. This is flagged as Task 2.3 territory, but the vocabulary is needed here to understand when expanding descriptions is and is not the remedy.
Action-perception loop and result design. Lesson agentic-eng-action-tools frames tools as the agent's action space and tool results as perception that drives the next reasoning cycle. The signal-to-noise principle there, returning relevance-scored paths and pass/fail summaries rather than raw dumps, is the production counterpart to the description guidance. A well-described tool that returns 500 lines of raw output still leaves the loop open and wastes tokens.
Risk-tiered permissions. The same lesson classifies tools into read-only, write, destructive, and external, with progressively stricter approval expectations. Schema design interacts with this classification because a tool's side effects must be documented in its description so the model, and a human reviewer, know that calling it mutates state.
Granularity guidance. Read operations should be fine-grained for context control, writes should be coarse-grained for safety, and search should be medium-grained. This is omitted from the reference page but sets the default for whether splitting is appropriate in a given domain.
Worked production examples
Two end-to-end walkthroughs show how the mechanisms combine in a system that could ship. Each names the failure mode it avoids, the reasoning chain that leads to the remedy, and the observable outcome.
Worked production examples: Example 1: Customer support copilot that stopped misrouting order queries
Context. A retail copilot exposes two tools to handle the most frequent intents. Observability showed that roughly one in three order-tracking utterances such as "check my order #12345" was routed to get_customer instead of lookup_order, while customer-identity utterances such as "is my account still active under [email protected]" were rarely misrouted the other way. The tools array at the time carried minimal descriptions.
Step 1: Capture the failure. Production logs sampled ten real user utterances covering both intents and recorded the selected tool per turn. Three of ten order queries hit get_customer, confirming an ambiguity problem rather than random noise. The sampling excluded any prompt-forced or routing-layer behavior so the measurement reflected pure description-driven selection.
Step 2: Diagnose the root cause. The definitions at the time looked like this, stripped to the load-bearing lines:
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic();
const weakTools: Anthropic.Tool[] = [
{
name: "get_customer",
description: "Retrieves customer information",
input_schema: {
type: "object",
properties: {
identifier: {
type: "string",
description: "Customer identifier",
},
},
required: ["identifier"],
},
},
{
name: "lookup_order",
description: "Retrieves order details",
input_schema: {
type: "object",
properties: {
identifier: {
type: "string",
description: "Order identifier",
},
},
required: ["identifier"],
},
},
];Both tools share the same identifier property name, the same type, and near identical descriptions. No format tells the model that get_customer expects email, phone, or customer_id while lookup_order expects #NNNNN or a tracking ID. No boundary tells it what not to use each tool for. The model therefore treats the two as synonymous and picks on surface keyword overlap.
Step 3: Expand descriptions and tighten schemas. The remedy is low effort and high leverage. Each tool keeps its name but receives a production-grade description that states purpose, accepted formats, returned fields, and an explicit do-not-use clause. Each input_schema gains per-property description with format examples, and additionalProperties: false closes the object.
const strongTools: Anthropic.Tool[] = [
{
name: "get_customer",
description:
"Looks up a customer account by email address, phone number, or customer ID. "
+ "Returns customer profile fields: name, contact details, account status, and loyalty tier. "
+ "Use this when you need to verify who the customer is. "
+ "Do NOT use for order-specific queries such as status or shipping, use lookup_order for those.",
input_schema: {
type: "object",
properties: {
identifier: {
type: "string",
description:
"Customer identifier. One of: email address (e.g. '[email protected]'), "
+ "phone number in E.164 format (e.g. '+14155552671'), or customer ID in CUST-XXXX format (e.g. 'CUST-4831').",
},
include_loyalty: {
type: "boolean",
description:
"When true, include loyalty tier and points balance. Defaults to false if omitted.",
},
},
required: ["identifier"],
additionalProperties: false,
},
},
{
name: "lookup_order",
description:
"Retrieves order details by order number in #NNNNN format or by tracking ID. "
+ "Returns order status, item lines, shipping details, and refund eligibility. "
+ "Use this when a customer asks about a specific order. "
+ "Do NOT use for customer identity verification or account lookups, use get_customer for those.",
input_schema: {
type: "object",
properties: {
identifier: {
type: "string",
description:
"Order identifier. One of: order number in #NNNNN format (e.g. '#12345') "
+ "or carrier tracking ID (e.g. '1Z999AA10123456784').",
},
include_items: {
type: "boolean",
description:
"When true, include full item lines with SKU, quantity, and price. Defaults to true if omitted.",
},
},
required: ["identifier"],
additionalProperties: false,
},
},
];What this block proves: the name, description, and input_schema are the full selection surface, and each property description teaches format in a way that enum alone cannot for open-ended identifiers. The failure boundary it addresses is ambiguous overlap between tools that accept similar-shaped strings. Its observable output is that the same ten sampled utterances, when replayed against strongTools, route correctly on nine or ten of ten turns without any change to model, prompt, or infrastructure.
Step 4: Audit the system prompt. The team found the prompt contained "Always check customer details before handling any request." That sentence creates a keyword association between any mention of a customer and get_customer, even for order flows. It was rewritten to "Verify customer identity only when the request requires identity-sensitive action such as profile changes or payment updates. For order tracking, use the order identifier directly." After the rewrite the last remaining misroutes disappeared.
Step 5: Guard with strict and server-side checks. The team set strict: false during iteration and then enabled strict: true once the schema stabilized, which guarantees that tool_use.input matches input_schema in shape, not in business authority. A server-side check still verifies that the caller is entitled to view the returned customer or order, because a well-shaped identifier can still reference a record outside the session's scope.
Step 6: Verify the loop contract. A minimal harness shows the tool_use and tool_result shapes and the required ordering:
async function handleTurn(userText: string) {
const response = await client.messages.create({
model: "claude-sonnet-4-6",
max_tokens: 1024,
tools: strongTools,
tool_choice: { type: "auto" },
messages: [{ role: "user", content: userText }],
});
if (response.stop_reason === "tool_use") {
const toolUses = response.content.filter((b) => b.type === "tool_use");
const toolResults = await Promise.all(
toolUses.map(async (block) => {
if (block.type !== "tool_use") throw new Error("unexpected block type");
const result = await dispatchTool(block.name, block.input);
return {
type: "tool_result" as const,
tool_use_id: block.id,
content: result,
};
})
);
const second = await client.messages.create({
model: "claude-sonnet-4-6",
max_tokens: 1024,
tools: strongTools,
messages: [
{ role: "user", content: userText },
{ role: "assistant", content: response.content },
{ role: "user", content: toolResults },
],
});
return second;
}
return response;
}
async function dispatchTool(name: string, input: unknown): Promise<string> {
if (name === "get_customer") return JSON.stringify(await lookupCustomer(input));
if (name === "lookup_order") return JSON.stringify(await lookupOrder(input));
throw new Error(`unknown tool ${name}`);
}What this block proves: the id to tool_use_id correlation, the requirement to preserve the assistant message before appending tool_result blocks, and the handling of parallel tool_use blocks in one turn. Its failure boundary is a single missing or mismatched tool_use_id or an omitted assistant message, either of which causes the follow-on POST /v1/messages to fail validation. Its observable output is a second response with stop_reason: "end_turn" containing the final answer that synthesizes the tool output.
Worked production examples: Example 2: Splitting a monolithic document tool and adding a batch variant
Context. An operations agent started with a single tool called analyze_document: "Analyses a document and returns results" whose input_schema accepted a document_id and a task string. The agent was asked to extract invoice amounts one turn, summarize a contract the next, and verify a compliance claim the turn after. The single tool forced the model to encode intent in the task parameter free text, which the backend then parsed heuristically. Accuracy was low because two distinct capabilities hid behind one name and one vague description.
Step 1: Split by intent. The team replaced the monolith with three narrowly scoped tools, each with its own description and narrow input_schema. A batch variant was added for the extraction case where runs are long and repeated:
const documentTools: Anthropic.Tool[] = [
{
name: "extract_data_points",
description:
"Extracts structured data fields (dates, amounts, names, SKUs) from a document. "
+ "Returns an array of field objects with value, page, and confidence. "
+ "Use for extraction only. Do NOT use for summarization or claim verification.",
input_schema: {
type: "object",
properties: {
document_id: {
type: "string",
description: "Document ID in DOC-XXXXXXXX format, e.g. 'DOC-9F3A21C0'",
},
fields: {
type: "array",
items: { type: "string", enum: ["date", "amount", "name", "sku", "total"] },
description: "Field types to extract. At least one required.",
},
page_range: {
type: "string",
description: "Optional page range, e.g. '1-3' or '2,5'. Defaults to all pages if omitted.",
},
},
required: ["document_id", "fields"],
additionalProperties: false,
},
},
{
name: "extract_data_points_batch",
description:
"Batch variant of extract_data_points. Accepts an array of document field requests "
+ "and returns an array of results in the same order. Use when extracting the same fields "
+ "from multiple documents or multiple field sets from one document. One batch call replaces N sequential calls.",
input_schema: {
type: "object",
properties: {
requests: {
type: "array",
description: "Array of extraction requests, 1 to 20 items per batch",
items: {
type: "object",
properties: {
document_id: { type: "string", description: "Document ID in DOC-XXXXXXXX format" },
fields: {
type: "array",
items: { type: "string", enum: ["date", "amount", "name", "sku", "total"] },
description: "Fields to extract for this request",
},
},
required: ["document_id", "fields"],
additionalProperties: false,
},
},
},
required: ["requests"],
additionalProperties: false,
},
},
{
name: "summarize_content",
description:
"Produces a concise summary of a document's key arguments and conclusions. "
+ "Returns a summary string plus the three most cited sections. "
+ "Use only when the user asks for a summary or overview. Do NOT use for field extraction.",
input_schema: {
type: "object",
properties: {
document_id: {
type: "string",
description: "Document ID in DOC-XXXXXXXX format",
},
max_sentences: {
type: "integer",
description: "Maximum sentences in the summary. Defaults to 8 if omitted. Range 3 to 12.",
},
},
required: ["document_id"],
additionalProperties: false,
},
},
{
name: "verify_claim_against_source",
description:
"Checks whether a specific claim is supported by the source document. "
+ "Returns an object with verdict (supported, contradicted, not_enough_evidence) and cited evidence spans. "
+ "Use only for claim verification against a named source document.",
input_schema: {
type: "object",
properties: {
document_id: {
type: "string",
description: "Document ID in DOC-XXXXXXXX format",
},
claim: {
type: "string",
description: "The claim to verify, as a single declarative sentence, e.g. 'Invoice total is $4,200'",
},
},
required: ["document_id", "claim"],
additionalProperties: false,
},
},
];Step 2: Prefer parallel variants over boolean switches. The extraction pair follows the lesson guidance to add a new tool rather than a boolean is_batch switch on the existing one. Each tool has its own description and schema, the batch variant constrains batch size at the schema layer, and existing callers of extract_data_points require no migration. The summary tool uses integer for max_sentences rather than number because the backend expects a whole number.
Step 3: Wire the batch path to real parallelism. The backend executes extract_data_points_batch concurrently across its requests and returns an array in matching order, so one round trip replaces N serialized turns. Input size limits are enforced at the API layer so that an oversized batch is rejected with a structured error rather than timing out. The model's ability to call extract_data_points_batch with an array is governed entirely by its input_schema, with items and enum constraining each entry.
Step 4: Design for caching and loading. Because four detailed tool definitions raise the per-turn token cost, the application marks the stable prefix with cache_control: { type: "ephemeral" } so repeated turns reuse the cached prefix. If the agent's repertoire grows further, the team can mark the batch tool with defer_loading: true and keep only the single-document extractor eager, so the model searches for the batch capability only when it needs it. For code-execution orchestration, allowed_callers: ["code_execution_20250825"] can be added to the extraction tool so that bulk filtering happens inside sandbox code without returning bulky intermediates to the context.
Observable outcome. After the split, intent accuracy on the same utterance set moved from frequent conflation of extraction and summarization to reliable selection on each variant, because each name and description now corresponds to exactly one user intent. The batch variant reduced wall-clock time for three-document extraction jobs from the sum of three serialized calls to roughly the duration of one, observable as fewer tool_use round trips and lower output token cost per job.
Worked production examples: Validation code that catches what strict cannot
Even with strict: true, application code must validate business shape and handle the truncation case. The block below sits in the dispatch layer, between the model output and the domain handler, and is deliberately language tagged and complete so its failure modes can be named.
type ToolUseBlock = {
type: "tool_use";
id: string;
name: string;
input: unknown;
};
function validateToolUse(block: ToolUseBlock, stopReason: string | null): void {
if (!block.id.startsWith("toolu_")) {
throw new Error(`invalid tool_use id prefix: ${block.id}`);
}
if (!["get_customer", "lookup_order"].includes(block.name)) {
throw new Error(`unknown tool name ${block.name}`);
}
if (stopReason === "max_tokens") {
throw new Error(
"response truncated at max_tokens, input may be incomplete, do not execute"
);
}
const input = block.input as Record<string, unknown>;
if (typeof input.identifier !== "string" || input.identifier.length === 0) {
throw new Error("identifier must be a non-empty string");
}
if (!/^CUST-|@|\+1/.test(input.identifier) && block.name === "get_customer") {
console.warn(`get_customer received non-customer identifier ${input.identifier}`);
}
if (!/^#\d{5}$/.test(input.identifier as string) && block.name === "lookup_order") {
const looksLikeEmail = (input.identifier as string).includes("@");
if (looksLikeEmail) {
throw new Error(`lookup_order received an email ${input.identifier}, likely misroute`);
}
}
}What this block proves: strict guarantees schema shape but not business correctness, max_tokens truncation must be treated as incomplete and not executed, and heuristic format checks in the handler catch the residual case where the model's best-effort input leaks a category mismatch. Its failure boundary is intentionally visible. The get_customer branch warns rather than throws on a loose pattern because legitimate phone formats vary, while the lookup_order branch throws on an email because that always signals a misroute worth surfacing before a backend round trip. Its observable output is either a dispatched call with validated arguments or a logged error that returns a tool_result with is_error: true and a human readable message the model can act on.
Build exercise material
Each step below is verifiable. No step asks the learner to guess at success. The observable outcome at the end of each step proves that the step worked.
Step 1: Register two ambiguous tools. In a throwaway MCP server or a local Messages API harness, define get_customer and lookup_order with single-sentence descriptions and a shared identifier string property, as shown in the weak-tools block above. Start the server and list the registered tools. You should see exactly two tools, each with a description under ten words and no boundary clause, and a single call to list tools returns both names.
Step 2: Replay ten queries and log selection. Send the same ten user utterances used in the walkthrough, five that mention customer identity (email or phone or customer ID) and five that mention orders (order number or tracking ID). For each turn log the emitted tool_use.name and tool_use.input. You should see at least two to three cases where an order-bearing utterance is routed to get_customer or vice versa, with stop_reason: "tool_use" on every turn where the model chose a tool.
Step 3: Rewrite descriptions to production grade and tighten schemas. Replace each one-line description with a three to five sentence description that states purpose, accepted identifier formats with examples, what the tool returns, and an explicit do-not-use boundary against the sibling tool. In each input_schema, expand the identifier description to name formats such as CUST-XXXX or #NNNNN, and add additionalProperties: false. Keep the name identifiers unchanged. Reload the tools and verify that get_tools or the equivalent listing shows the longer strings and the stricter schema.
Step 4: Re-run the same ten queries. Send the identical utterances in the same order without changing the system prompt. You should see selection correct on nine or ten of ten cases, with the previously misrouted order utterances now hitting lookup_order and carrying a correctly formatted #NNNNN identifier in tool_use.input.
Step 5: Audit the system prompt. Open the system prompt and search for keyword-sensitive phrases such as "always check customer details", "always use customer data first", or any instruction that links any mention of a customer to a single tool. Replace each found phrase with a scoped version such as "Verify customer identity only when the task requires identity-sensitive action, otherwise use the identifier the user provided directly." Re-send any order query that still contained the word "customer" and confirm that tool_use.name no longer flips to get_customer.
Step 6: Add the loop guards. In the loop harness, add handling for stop_reason: "max_tokens" that treats any trailing tool_use as incomplete and does not execute it, handling for tool_result with is_error: true that surfaces a readable message, and a branch that sends an assistant message's content first before appending the tool_result user message on the next turn. You should see that a deliberately low max_tokens: 10 request produces the truncation path without a dispatch, and that swapping the assistant and user message order produces a validation error from the API that disappears when the correct ordering is restored.
Step 7: Observe caching and cost. With strongTools installed and a system prompt of at least a few hundred tokens, set cache_control: { type: "ephemeral" } on the prompt and on the tools prefix, then make two consecutive calls with the same tool set. You should see the second response report cache reuse in the usage or billing metadata and a lower input token bill for the repeated prefix, even though the description content and selection behavior are unchanged.
Mechanism and API surface
- Messages API tools array
- Transmit tools as an array of name, description, and input_schema on every request. The model receives this list alongside system and history and emits tool_use blocks with type tool_use, id, name, and input when it chooses to act.
- JSON Schema keywords for input_schema
- Use type object, properties with per property type and description, enum for fixed sets, pattern and format for identifier shapes, minimum and maximum for ranges, additionalProperties false to close stray keys, and required listing only truly required fields.
- Strict mode as sampling constraint
- Set strict true alongside name, description, and input_schema to guarantee the emitted input matches the schema shape, including enum and type correctness. Shape is guaranteed, business validity remains host side.
- Five part description template
- State what the tool does, what inputs it expects with formats and required versus optional, concrete example queries it handles well, edge cases and what it does not do, and explicit when to use this tool versus its neighbor.
- Property level guidance and examples
- Every property carries its own description with format and example such as USR dash XXXXX with USR dash 48721 as example. Fixed sets use enum so the model does not invent values outside the allowed set.
- Splitting and renaming as interface repair
- Replace one generic umbrella with narrow typed tools such as extract_data_points, summarize_content, and verify_claim_against_source. Where names confuse, rename to a behaviour anchored form such as extract_web_results and rewrite the description to be scope specific.
The decision rules in play
Each rule below is a distinct decision the exam tests, stated with its mechanism, the boundary where the answer flips, and the wrong answers written against it.
Tool descriptions are the primary selection signal
When a model receives a tool list, it reads each tool's name, description, and input_schema. The description is the dominant signal for matching user intent to tool capability.
Descriptions are the only place where purpose, scope, input expectations, and alternative-tool guidance can be expressed in natural language the model can reason over before acting. Schemas are structured constraints for validation, names are short labels without conditional logic.
Boundary. This rule dominates when the agent has a workable number of tools and the failure is discrimination between two or three similar options. The opposite case is when overload itself is the problem.
Recurring specifics. Field names that recur include customer_id, email, phone, order_id with format #NNNNN, tracking_id, parcel_id, mls_id, account_number, file_id, project_key, severity. Thresholds that recur include error rates such as 30 to 45 percent misrouting on minimal descriptions dropping to low single digits after rewrite, and near-random 50/50 alternation when descriptions are identical. The five-element definition recurs verbatim as purpose, expected inputs with formats, example queries, edge cases, and explicit boundaries against similar tools.
Proposal. Fix misrouting with longer tool names or alphabetical reordering.
Why it attracts. Names are visible and cheap to change, and ordering feels like a priority control.
Why it fails. Names contribute only weak signal without the conditional prose that explains when to use the tool versus the alternative. Order has no documented priority effect.
When it would be right. Never as a primary fix. Renaming helps only when paired with a full description rewrite that carries the actual boundary logic.
Proposal. Fix with tool_choice forced to a single tool.
Why it attracts. Guarantees a call to the preferred tool.
Why it fails. It removes selection rather than improving it, disabling the other workflow entirely and masking the overlap instead of eliminating it.
When it would be right. Only for a workflow where only one tool should ever be callable in that turn, not for curing ambiguous contracts.
Proposal. Fix with a validation layer that rejects malformed identifiers before the backend.
Why it attracts. Feels defensive and catches errors.
Why it fails. Validation fires after wrong selection, it does not prevent the wrong tool from being chosen in the first place.
When it would be right. As a second-layer safety net after descriptions are fixed, never as the first step.
- Replacing one identifier format with another such as
emailversusphoneversuscustomer_idwhile keeping descriptions minimal leaves misrouting rate unchanged. Adding input format detail to only one description partially helps but leaves the unedited sibling still matching broadly and reintroduces misrouting in the opposite direction.
Minimal one-sentence descriptions cause systematic misrouting
A minimal description such as Retrieves customer information or Analyzes content and returns insights states a capability without stating its scope, inputs, outputs, or alternatives. When two or more tools share that pattern, the model cannot form a decision boundary.
Minimal descriptions carry high generality and low discriminative power. Every phrasing about retrieving data matches every minimal description equally well.
Boundary. Short descriptions are not automatically wrong when tools are semantically distant. search_flights and search_hotels would still be distinguishable with one sentence each if each names its entity, but even there the examined evidence rewards a full rewrite that adds trigger conditions and example inputs.
Recurring specifics. Minimal patterns that recur include Gets data, Does the customer thing, Handles all customer-related operations, Processes a file, Searches the codebase, and Retrieves customer information paired with Retrieves order details. Vague variants that bundle lookup, update, and deletion into one scope statement also recur as a signal of overload. the tested material repeatedly marks minimalism as trading a marginal imagined performance benefit for a large accuracy loss that does not actually exist.
Proposal. Keep descriptions concise for faster agent processing.
Why it attracts. Brevity feels efficient and token-aware.
Why it fails. Selection accuracy dominates latency, and sparse text gives the model less to reason with, not less latency in any meaningful sense.
When it would be right. Never when overlap exists. Only when tools are already distinct by name and schema and no selection confusion has been observed.
Proposal. Minimal descriptions are fine because the agent will learn through trial and error.
Why it attracts. Assumes in-context adaptation.
Why it fails. Tool definitions are stateless per turn, there is no learning loop that corrects a permanently ambiguous contract.
When it would be right. Never.
- Changing
Retrieves customer informationtoRetrieves customer information by identifierwithout enumerating formats or examples leaves misrouting largely intact. Adding one example query but no boundary improves one direction of the confusion while leaving the reverse misroute.
Production-grade descriptions require five elements
A production description answers five questions before the model acts: what the tool does, what inputs it expects with formats and constraints, what example queries it handles well, what edge cases and limitations exist, and when to use this tool versus its closest sibling. The model uses the first to match intent, the second to validate argument formation, the third to anchor phrasing, the fourth to avoid invalid calls, and the fifth to suppress the competitor.
{
"name": "get_customer",
"description": "Looks up a customer account by email address, phone number, or customer ID. Returns customer profile including name, contact details, account status, and loyalty tier. Use when verifying customer identity, profile, or preferences. Do NOT use for order-specific queries such as status or shipping, use lookup_order for those.",
"input_schema": {
"type": "object",
"properties": {
"identifier": {
"type": "string",
"description": "Email address, phone number in E.164 format, or customer ID. Examples: '[email protected]', '+14155552671', 'CUS-88412'."
}
},
"required": ["identifier"],
"additionalProperties": false
}
}{
"name": "lookup_order",
"description": "Retrieves order details by order number formatted as #NNNNN or by tracking ID. Returns order status, items, shipping details, and refund eligibility. Use when the customer asks about a specific purchase, shipment, or transaction. Do NOT use for customer identity verification, use get_customer for that.",
"input_schema": {
"type": "object",
"properties": {
"order_id": {
"type": "string",
"description": "Order number in format #NNNNN or carrier tracking ID. Examples: '#55821', '1Z999AA10123456784'."
}
},
"required": ["order_id"],
"additionalProperties": false
}
}Each element addresses a distinct failure mode. Purpose without inputs leads to correctly chosen but malformed calls.
Boundary. When ambiguity is concentrated in one element, a targeted enhancement to that element suffices. If inputs are already well typed and examples are present but boundaries are missing, adding only the boundary clause resolves the remaining 2 percent tail.
Recurring specifics. Purpose phrasing that recurs includes Looks up a customer account by, Retrieves order details by order number with format #NNNNN, and Returns sellable nightly rate for a date. Input format details that recur include date plus room_type for rates, parcel_id versus mls_id for property versus listing, email, phone, customer_id variants for identity. Output shape details that recur include name, contact details, account status, loyalty tier versus order status, items, shipping details, refund eligibility. Boundary phrasing that recurs includes Do NOT use for order-specific queries, use lookup_order for those and its mirror.
Proposal. Adding a formal input_schema or outputSchema alone is sufficient without prose.
Why it attracts. Schemas feel rigorous and machine validated.
Why it fails. Schemas constrain how arguments are shaped after selection, they do not supply the conditional natural language the model needs to choose between siblings.
When it would be right. As a complement for argument validation, never as a substitute for boundary prose.
Proposal. Adding version or performance metadata to the description.
Why it attracts. Feels like richer metadata.
Why it fails. Version numbers and timing numbers do not map to any selection decision.
When it would be right. Never for selection accuracy.
- Swapping
format: #NNNNNfor another identifier pattern changes the input element but not the rule. Replacing example queries with edge case notes shifts which failure mode is addressed first. - Examples below illustrate the transformation from minimal to production grade.
- The block above moves the model from a generic retrieval match to a precise identity-verification decision. The identifier enumeration tells the model which surface forms are valid, the return shape tells it what to expect, and the negative boundary suppresses order queries.
- This sibling description is mirrored. Each states when to use itself and when to defer to the other, forming a bidirectional decision boundary.
Explicit negative boundaries with named alternatives
A negative boundary states when not to use the current tool and names the correct alternative. The pattern Use this for X, do NOT use for Y, use toolY for Y directly suppresses the competitor at selection time.
Positive descriptions alone leave overlap. Two tools that both plausibly handle cancellation will both score highly on that term unless one explicitly declines it.
Boundary. Negative boundaries are most critical when keyword overlap is high, such as account balance mapping to both check_balance and account_summary, or process_action and search_knowledge_base both containing cancellation. When tools are in distinct semantic zones, negative boundaries add little value and positive purpose alone suffices.
Recurring specifics. Boundary phrasings that recur include Use when the customer wants to execute a cancellation, use search_knowledge_base when they want to learn about the process, Use for pricing questions not allotment counts, Executing an action versus seeking information, and returns a single numeric balance figure versus returns the full account statement including balance, recent transactions, and status. the tested material repeatedly rewards Do NOT use for plus use toolX instead as a unit.
Proposal. Remove the shared keyword from both descriptions to eliminate conflict.
Why it attracts. Feels like it removes the trigger.
Why it fails. It makes both tools less discoverable for relevant queries and leaves the model with no signal about which tool handles the keyword.
When it would be right. Never as a fix for overlap. Only when the keyword was incorrectly added to a tool that genuinely should not handle that concept.
Proposal. Fix with tool_choice forcing.
Why it attracts. Guarantees routing for that keyword.
Why it fails. It is brittle to paraphrase and pushes a description problem into call-time forcing.
When it would be right. Never as a first fix. Only when workflow policy requires a single tool per turn regardless of description quality.
- Changing
subscription cancellationstoplan changesshifts which workflow triggers the boundary but not the mechanism. Reversing which tool gets the positive versus negative clause swaps the misrouting direction without fixing it unless both are mirrored..
Front-loaded differentiation versus identical openings
When two descriptions open with the same phrase such as Analyzes content and returns insights or Analyzes content and extracts key information, the model cannot distinguish them early in reading. Disambiguating purpose and accepted inputs should lead the first sentence.
Selection scoring is front-weighted. If the opening clause matches both tools equally, the model treats them as equivalent until late in the sentence, by which point its selection may already be anchored.
Boundary. Length itself is not the cause. A long description that front-loads differentiation early succeeds.
Recurring specifics. Identical openings that recur include Analyzes content and returns, Searches the codebase, and Gets data. Front-loaded rewrites that recur include extract_web_results described as Returns information retrieved from web searches and URLs versus document tools described with uploaded files such as PDFs and Word documents.
Proposal. The model stops reading after the first sentence, so descriptions must be short.
Why it attracts. Misreads truncation as intentional brevity.
Why it fails. The fix is not shortening but ordering the differentiating information first.
When it would be right. Never. Clarity of ordering dominates length.
Proposal. Renaming with numbers such as tool_1 and tool_2.
Why it attracts. Feels like it creates order.
Why it fails. Numbers carry no semantic signal.
When it would be right. Never.
- Swapping which tool gets the more specific first clause does not fix overlap unless both diverge. Renaming plus front-loaded description such as
extract_web_resultsversusmerge_subagent_findingsis shown as the most reliable combination for coordinator routing..
Renaming without rewriting descriptions does not cure misrouting
Renaming changes the name token the model sees but leaves the selection prose ambiguous. Evidence shows a tool renamed from analyze_content to analyze_web_page while leaving the description unchanged still misroutes because the description remains the vague Analyzes content.
Selection is description-driven. A name is one token among many, while the description is multiple sentences of conditional prose.
Boundary. Renaming becomes effective when paired with a full description rewrite and schema narrowing. The combination extract_web_results plus Returns information retrieved from web searches and URLs plus a sibling merge_subagent_findings description succeeds where either alone fails.
Recurring specifics. Renames that recur include resize_image and detect_objects versus handles images and works with pictures, extract_search_highlights versus merge_subagent_findings, get_room_allotment versus query_inventory, and search_issues with enriched scope.
Proposal. Rename to alphabetic advantage or with an ordering prefix.
Why it attracts. Feels like a priority hack.
Why it fails. Order and prefix tricks have no documented priority effect.
When it would be right. Never as a standalone fix.
Proposal. Rename only the more confused tool and leave the sibling unchanged.
Why it attracts. Feels like minimal effort.
Why it fails. Bidirectional overlap requires both descriptions to draw boundaries. Leaving one unchanged leaves one direction of misrouting intact.
When it would be right. Only when telemetry shows confusion is strictly one-way, which the tested material shows is rare.
- High-collision-only renames such as fixing only
search_candidatesandfind_talentwhile leavingsearch_employeesunchanged are tested and marked insufficient. Full set rewrites where every overlapping sibling names its boundary are marked correct. - .
Overlapping output shapes require input-type and trigger-condition disambiguation
Two tools can return the same fields such as get_user_by_id and get_user_by_email both returning identical user records yet still require different selection logic based on what the caller already knows. When output shape is identical, the model cannot use output expectation to choose.
Output overlap increases confusion because the model infers that either tool satisfies the goal. Only an input contract that names the required identifier format resolves the tie.
Boundary. When outputs differ meaningfully such as get_customer_profile returning name and preferences versus get_customer_orders returning order history, boundary guidance can be phrased in terms of return shape and default routing such as profile is default for tell me about queries, orders only when order-specific information is requested. When outputs are identical, return-shape language does not help and input-type language is required.
Recurring specifics. Identifier pairs that recur include candidate_id versus criteria, parcel_id for ownership and tax versus mls_id for sale price and broker, order number #NNNNN versus email or phone versus account_number, and file path for repository PDFs versus URL for web pages. The symptom of passing claim number to get_policyholder or policyholder name to lookup_claim alternates unpredictably when this rule is violated.
Proposal. Remove one tool and add both lookup methods as parameters on a single tool.
Why it attracts. Feels like consolidation solves overlap.
Why it fails. Here outputs are identical but the inputs signal which lookup path is appropriate, merging would require the single tool to handle both identifier types correctly and the tested material marks expanding descriptions as the lower-effort correct first step before consolidation.
When it would be right. When the tools truly represent the same action varying only by identifier field and the catalog is overloaded with eight near-identical variants, consolidation into lookup_user with identifier plus type becomes the architecturally correct fix per Rule 9. For the two-tool case, description expansion remains first.
Proposal. Add a routing layer that detects input type automatically.
Why it attracts. Automates the type check.
Why it fails. Bypasses the model's language understanding and adds brittle keyword logic when a description sentence solves it.
When it would be right. Never as a first step. Only as post-selection validation.
- Swapping which tool is described as
active for-sale priceversusownership and tax assessmentmoves the error from one entity domain to the other without changing the mechanism. Changing identifier format fromcandidate_idtoemailorphonepreserves the same rule, the description must simply name the accepted format of each sibling..
Splitting a generic overloaded tool into purpose-specific narrow tools
A generic tool such as analyze_document or process_file or analyze_content or query_features that handles multiple behaviors behind one description forces the model to infer intent from free text. The agent then picks the wrong internal operation in 22 to 33 percent of runs, validates files that were never parsed, or re-converts already converted files.
// Before: one generic tool with free-text instruction, ambiguous selection
{
name: "analyze_document",
description: "Analyzes a document and returns results",
input_schema: {
type: "object",
properties: {
document: { type: "string", description: "Document content or path" },
instruction: { type: "string", description: "What to do with the document" }
},
required: ["document", "instruction"]
}
}
// After: three narrow tools, each with a typed contract and boundary
{
name: "extract_data_points",
description: "Extracts structured data fields such as dates, amounts, names, and statistics from a document. Returns an array of discrete values with units and location. Do NOT use for prose summaries or claim verification, use summarize_content or verify_claim_against_source for those.",
input_schema: {
type: "object",
properties: {
document: { type: "string", description: "Document path or raw text. Example: '/docs/earnings-q3.pdf'" },
data_types: { type: "array", items: { type: "string", enum: ["dates", "amounts", "names", "financial_metrics"] }, description: "Which field types to extract" }
},
required: ["document", "data_types"]
}
}{
name: "summarize_content",
description: "Produces a concise summary of a document's key arguments and conclusions. Returns prose summary plus key point list. Use when the user asks for overview or gist. Do NOT use when discrete field values are needed for a table, use extract_data_points for that.",
input_schema: {
type: "object",
properties: {
document: { type: "string", description: "Document path or raw text" },
focus_area: { type: "string", description: "Section to emphasize. Example: 'methodology'" },
summary_length: { type: "string", enum: ["brief", "concise", "detailed"], description: "Desired summary length" }
},
required: ["document"]
}
}Tool identity is a discrete choice the model can branch on reliably, while a mode argument is a continuous inference inside one tool that shares a description with all modes. Splitting gives each behavior a distinct name and a distinct description that separately encodes its trigger.
Boundary. Splitting is the correct remedy when the overloaded tool covers semantically broad behaviors that need different parameter subsets and produce different output shapes. Examples include extracting data points versus summarizing content versus verifying a claim, or convert_codec versus extract_thumbnail versus normalize_audio versus remux_container.
Recurring specifics. Split sets that recur include extract_data_points, summarize_content, verify_claim_against_source for document work, extract_web_results, parse_document, analyze_code for content analysis, lookup_point, search_radius, find_nearest, compute_route, export_bbox for geospatial query_features, and convert_codec, extract_thumbnail, normalize_audio, remux_container for media process_file. Mode counts that recur include three to four behaviors for documents, four for media, eight for geospatial. Misrouting rates for generic tools recur at 22 to 33 percent, dropping only slightly with prompt hints.
Proposal. Keep the generic tool and add few-shot examples demonstrating correct operation choices.
Why it attracts. Examples feel like they teach the mode mapping.
Why it fails. The examined evidence shows three few-shot examples cutting misrouting only from 24 to 19 percent or system hints lowering 33 to 26 percent, leaving the structural ambiguity unresolved across unfamiliar phrasings.
When it would be right. After splitting, targeted few-shots on the remaining ambiguous edge phrasings can help, but never as a substitute for splitting.
Proposal. Keep the generic tool and expand its description to enumerate the three jobs plus a keyword-to-job mapping in the system prompt.
Why it attracts. Feels like it adds clarity without changing the interface.
Why it fails. One description must still cover every behavior, so any request phrasing still plausibly matches the same description. The mapping drifts as new phrasings appear.
When it would be right. Never when behaviors need different parameter subsets and outputs.
Proposal. Add a confidence score output to the generic tool.
Why it attracts. Feels like it helps the agent self-correct.
Why it fails. It does not change selection at all, it only labels a wrong selection after the fact.
When it would be right. Never for misrouting.
- Replacing the domain from documents to media to geospatial preserves the same split logic. Whether the split yields three tools or four or eight changes only the number of narrow descriptions, not the principle.
- Concrete split example for document extraction:
- The pair shows why splitting works. Each description now names a distinct output shape and states when to defer to the sibling.
Consolidating tools that differ only by input field
When tools represent the same action varying only by the identifier field such as lookup_user_by_email, lookup_user_by_phone, lookup_user_by_id across eight variants, or 19 transformation tools sharing input data, transformation type, and output format, selection reliability degrades on choice count alone. Consolidating into one parameterized tool such as lookup_user accepting identifier plus type or transform_data with transform_type reduces the decision space from many overlapping options to one typed choice with an enum.
{
"name": "lookup_user",
"description": "Looks up a user account by any supported identifier. Returns profile fields. Use for any user lookup by providing the identifier and its type. One tool covers all identifier variants.",
"input_schema": {
"type": "object",
"properties": {
"identifier": { "type": "string", "description": "The identifier value. Examples: '[email protected]', '+14155552671', 'USR-9912'" },
"identifier_type": { "type": "string", "enum": ["email", "phone", "customer_id", "account_number"], "description": "The type of the provided identifier" }
},
"required": ["identifier", "identifier_type"],
"additionalProperties": false
}
}The tools are functionally identical aside from which field is populated. Maintaining separate tools for each field creates artificial overlap where every phrasing about looking up a user matches multiple tools.
Boundary. Consolidation is correct when tools share action, output shape, and differ only by input field value. Splitting is correct when tools share a name but handle semantically distinct actions with different outputs and parameter subsets.
Recurring specifics. Counts that recur include eight near-identical lookup variants, 19 transformation tools consolidated to one, catalog sizes of 22 tools reduced to four which brings the count back within the four to five reliable range. The anti-pattern of splitting further by adding region-specific variants is flagged as worsening the overload.
Proposal. Keep all eight but write longer descriptions for each.
Why it attracts. Feels like it preserves granularity with better documentation.
Why it fails. Eight overlapping choices remain eight choices regardless of description length, selection complexity is not reduced.
When it would be right. When overlap is low and each tool serves a genuinely distinct action, not when they are field variants.
Proposal. Add few-shot examples distinguishing all eight.
Why it attracts. Examples seem to teach the difference.
Why it fails. Examples add token overhead on every turn and do not reduce the number of indistinguishable options the model must choose among.
When it would be right. After consolidation, targeted examples for the remaining ambiguous phrasings can help as a refinement.
Proposal. Split each further by region-specific variants for finer routing.
Why it attracts. Feels like finer granularity improves precision.
Why it fails. It multiplies overlap and exacerbates the threshold degradation.
When it would be right. Never in this pattern.
- Changing the number of field variants from five to eight to 19 does not change the logic. Replacing
lookup_userwithtransform_datapreserves the same consolidation mechanism. - Consolidation schema example:
- The enum in
identifier_typereplaces eight separate tools with one typed argument. Replacinglookup_userwithtransform_datapreserves the same consolidation mechanism.
Tool count threshold and distribution to subagents
Selection reliability degrades as tool count grows, independent of description quality. Evidence places the practical reliable limit at roughly four to five tools per agent.
Each additional tool adds a pairwise comparison the model must perform. With 18 to 30 tools the comparison space grows large enough that even well-written descriptions blur together.
Boundary. Below the threshold, description quality is the correct first lever. With two to five tools that are simply poorly described, rewriting descriptions is low effort and high leverage.
Recurring specifics. Threshold counts that recur include four to five per agent as optimal, 18 tools degrading on multi-part requests, 22 tools taking three to four turns to select, 30 tools calling wrong tools on billing versus refund versus insurance phrasing, and the 94 to 81 percent drop after adding one tool. Dynamically loading only the tools relevant to the current task is the most scalable variant for large catalogs.
Proposal. Improve all 22 tool descriptions with detailed examples.
Why it attracts. Feels like it addresses quality.
Why it fails. Quality does not overcome count. 22 options remain 22 options and the model still faces ambiguous choice among many similar candidates.
When it would be right. As a complement after reducing count, not as a replacement for reduction.
Proposal. Use tool_choice: any to force a tool call.
Why it attracts. Guarantees activity.
Why it fails. It ensures a tool is called but not the right one, it does not improve selection accuracy.
When it would be right. Only when the risk is tool avoidance rather than misrouting, which is not the overloaded-catalog symptom.
Proposal. Combine everything into one large composite multi-purpose tool.
Why it attracts. Feels like it reduces count to one.
Why it fails. It overloads the single tool with every behavior behind one description, reproducing the split problem.
When it would be right. Never for misrouting. Composite tools belong only to Rule 8 splitting contexts where distinct outputs are needed.
- Replacing the domain from calendar and database to billing and shipping preserves the threshold logic. Whether the catalog is 18 or 22 or 30 changes only the severity, not the remedy.
Keyword-sensitive system prompt instructions that override clean descriptions
System prompts are heavily weighted. A sentence containing a keyword that overlaps a tool name can create an unintended association that fires on that keyword even when descriptions are clean.
System instructions function as standing objectives that prime the model before it reads tool definitions. A well-intentioned rule like when customers ask about their account, always start by retrieving their customer profile creates a keyword trigger that activates get_customer whenever account appears, even on order queries that mention account incidentally.
Boundary. This rule applies only after descriptions have been verified as clean. Most examined items show minimal descriptions as the root cause, and descriptions should be checked first.
Recurring specifics. Trigger phrasings that recur include always check customer details before proceeding, always investigate thoroughly, always prioritize resolving billing issues directly, when a customer mentions a refund, always ensure a human is involved, always analyse content before responding, and for any billing-related concerns, prioritise resolving them quickly using available billing tools. The bias rate that recurs is 78 to 80 percent calling the primed tool when the keyword is present versus 90 to 93 percent calling the alternative when it is absent. The mandated tool names that recur include get_customer on account, deep_investigate on thoroughness, handoff_to_human on refund, and adjust_billing on billing.
Proposal. Base model training associates the word with that domain strongly enough to override descriptions.
Why it attracts. Training associations are real.
Why it fails. Training bias is not the fixable in-scope cause the exam targets. When descriptions are clean yet selection is sharply keyword-correlated, the controllable lever is prompt wording, not training.
When it would be right. Never as the actionable diagnosis. Training is the background prior that the prompt layer amplifies.
Proposal. The tool descriptions need more negative examples.
Why it attracts. Feels like more examples could counter bias.
Why it fails. Descriptions cannot reliably override an explicit prompt instruction, refining them further leaves the override intact.
When it would be right. Only when descriptions are still weak, which the stem rules out.
Proposal. Fine-tune the model on mixed phrasing.
Why it attracts. Feels like it teaches disambiguation.
Why it fails. It treats a prompt artifact as a training gap.
When it would be right. Never as the first fix.
- Replacing
accountwithrefundorbillingorcontentpreserves the same mechanism, only the trigger word and the primed tool change. Sample variations that keep descriptions clean while shifting the bias rate within the 78 to 93 percent range are all treated identically: review the system prompt for keyword overlap with tool names and remove or precision-scope the instruction..
Contradictory prompt constraints versus permissive tool descriptions
A system prompt may impose a gate such as Only call send_notification for severity CRITICAL or Use process_payment only for amounts under $500 while the tool description invites unrestricted use such as Sends a notification for any event severity or Processes payments of any amount. The model then follows the broader description framing when selecting and invoking the tool, violating the prompt constraint such as sending HIGH severity notifications or processing $1,200 transactions.
Descriptions and prompts are both selection signals that must be consistent. When they contradict, the description's permissive scope lowers the perceived pre-condition for the call.
Boundary. This rule covers logical contradiction, while Rule 11 covers keyword association priming. Both are prompt interactions but with different fixes.
Recurring specifics. Severity values that recur include CRITICAL versus HIGH, payment amounts under and over $500 with $1,200 as the violating test amount. The examined correct fix wording recurs as Update the tool description to specify the exact trigger condition so the selection signal and tool description are consistent.
Proposal. Increase repetition of the constraint in three separate prompt locations.
Why it attracts. Feels like reinforcement.
Why it fails. Repetition does not cure a contradictory description that still invites unrestricted use.
When it would be right. Never as the primary fix. Consistency across layers is required.
Proposal. Rely on tool_choice: auto and model judgment.
Why it attracts. Trusts the model to apply the filter.
Why it fails. It leaves the contradictory framing in place that causes the misjudgment.
When it would be right. Only after consistency is restored.
Proposal. Rename the tool to encode the constraint such as send_critical_notification.
Why it attracts. Name carries the gate.
Why it fails. Name alone is weaker than a description sentence that names the exact trigger condition.
When it would be right. As a supplement to the description fix, not a replacement.
- Changing the gated field from severity to amount to customer scope preserves the contradiction pattern. The violation always involves a test value that lies outside the prompt gate but inside the description's broad framing..
Sparse MCP descriptions lose to built-in tools with richer context
MCP tools often ship with minimal descriptions such as Searches code while built-ins like Grep have rich default context. The agent then defaults to the built-in even when the MCP tool would return richer cross-file results.
Selection is description quality weighted. A sparse MCP description such as Searches code reads as indistinguishable from Grep to the model, so the familiar built-in wins.
Boundary. Disabling the built-in such as removing Grep is marked incorrect as a first step because it sacrifices a genuinely useful capability for trivial lookups and docs explicitly preserve the built-in for those cases. Likewise, a system prompt line such as prefer semantic search when tracing references helps partially such as cutting fallback from 64 to 49 percent or 35 to 24 percent but does not stabilize selection because the underlying descriptions still read as equivalent.
Recurring specifics. Built-in names that recur include Grep, Read, Bash, Glob. MCP tool names that recur include search_symbols, semantic_code_search, semantic_search, query_database, find_symbol_references, search_issues, get_ticket_details, search_codebase, code_locator, search_internal_kb. Fallback rates that recur include 35 percent for Grep versus call hierarchy resolution and 64 percent for Grep versus semantic index lookup. The correct description phrasings that recur include returns ranked hits with call-graph context, structured results with column types and pagination, and handles edge cases like overly narrow searches.
Proposal. Remove the built-in tool entirely.
Why it attracts. Guarantees the MCP tool is used.
Why it fails. It forces every text search through the semantic tool, losing fast literal matching for trivial cases.
When it would be right. Never as a first fix. Only if policy genuinely requires no text search.
Proposal. Add a system prompt instruction always prefer MCP tools over built-in tools.
Why it attracts. Simple global preference.
Why it fails. It is brittle and does not scale, and it bypasses the natural disambiguation that a correct description provides per query type.
When it would be right. As a temporary patch while descriptions are improved, not as the durable fix.
Proposal. Reinstall the MCP server or restart the connection.
Why it attracts. Feels like a connectivity fix.
Why it fails. The tools are discovered and available, the issue is selection not availability.
When it would be right. Only when attempts to call the MCP tool fail rather than being deprioritized.
- Replacing the MCP server domain from code search to ticket lookup to database query preserves the same sparse description failure. the tested material shows both a bare
Searches codeand the more polished but still generica powerful, comprehensive code-intelligence capabilityare marked incorrect when they lack concrete output differentiation. - .
Input schemas guide argument formation after selection
The input_schema or inputSchema is a JSON Schema object inside each tool definition that declares type, properties, enum constraints, required fields, and per-field description values. After the model selects a tool based on its description, it uses the schema to shape arguments: which fields are valid, which are required, what formats are expected such as ISO 8601 date string e.g.
Schemas are typed contracts the runtime can validate. A description can say accepts SQL but only the schema and its field description can say which SQL dialect and which parameter holds it.
Boundary. This rule does not replace description improvements for selection failures. Adding strict: true or expanding a schema does not help the model choose between query_db and search_logs when both accept a query string, descriptions with explicit boundaries are required for that.
Recurring specifics. Schema elements that recur include name, description, input_schema as the essential triple, type: object with properties, required, additionalProperties: false, and $schema with value https://json-schema.org/draft/2020-12/schema indicating the JSON Schema draft. The correct fix phrasing recurs as state in each description the expected input format, when the identifier becomes valid, and the edge cases such as a build that has not started plus field descriptions that give format and constraints.
Proposal. Use strict: true to fix selection between two similar tools.
Why it attracts. Strictness feels like precision.
Why it fails. Strict mode validates parameters after selection but does not teach the model when to prefer one tool over the other.
When it would be right. Only for argument validation after descriptions have fixed selection.
Proposal. Encode all parameters as one freeform string to simplify the schema.
Why it attracts. Feels flexible.
Why it fails. It destroys structure and enables hallucinated arguments on complex multi-part requests.
When it would be right. Never.
- Whether the schema omission is a missing
requiredfield, a missing enum, or a missing field description, the symptom is the same: selection succeeds but the call fails validation. Adding a worked example call to the description plus tighteningrequiredtogether is consistently the prescribed fix. - The three essential fields appear as a unit wherever tool metadata quality is questioned:
- json { "name": "search_bugs", "description": "Searches the bug tracker for matching tickets by keyword, component, and status.
Few-shot examples cannot substitute for clear boundaries as a first fix
Few-shot examples add 5 to 8 demonstrations to the prompt showing correct tool selection. On minimal descriptions, examples add token overhead on every turn without fixing the information poverty that causes misrouting.
Examples teach by similarity to demonstrated phrasings. Descriptions teach by stating an abstract rule that generalizes to novel phrasings.
Boundary. This rule applies when the question asks for the most effective first step on minimal descriptions. The opposite case is genuinely ambiguous inputs such as help me with my recent purchase where both tools could plausibly apply even after clean descriptions.
Recurring specifics. Example counts that recur include 5 to 8 and 6 to 8 as the typical proposed example block, with 4 to 6 prescribed for the refinement case. Token overhead and infrastructure cost are the recurring costs cited for premature example use. The comparison low effort high leverage single edit to each tool definition versus extra machinery every turn is the recurring tradeoff language.
Proposal. Add 6 to 8 few-shot examples routing order queries to lookup_order.
Why it attracts. Concrete demonstrations feel pedagogical.
Why it fails. The model is confused because descriptions do not differentiate the tools, examples only demonstrate specific phrasings the model already confuses.
When it would be right. Only after descriptions are fixed and only for the remaining ambiguous tail, per Rule 16.
Proposal. Group examples by tool.
Why it attracts. Feels organized.
Why it fails. Grouping does not address phrasing diversity and may reinforce separate patterns rather than teaching discrimination.
When it would be right. Never as the first fix. Even for refinement, ambiguous-case examples grouped by decision with reasoning are preferred.
- Replacing the order query phrasing with any ambiguous phrasing such as
help me with my recent purchasepreserves the same judgment: descriptions first, then targeted ambiguous-case examples..
Few-shot examples are highest leverage when targeted at ambiguous cases with reasoning
When descriptions are clean yet a small set of inputs remains ambiguous, 4 to 6 examples that each show the reasoning for choosing one tool over the plausible alternative provide the highest refinement leverage. The examples should target the failure cases, not the easy cases the model already handles, and should include the decision rationale so the model can generalize beyond surface similarity.
Reasoning traces teach the decision logic rather than the outcome alone. The model learns which disambiguating detail mattered such as recent purchase with order number versus account balance with no order reference and can apply that logic to novel ambiguous phrasings not seen in the examples.
Boundary. This rule is a refinement step that assumes descriptions have already been fixed. If descriptions are still minimal, examples are premature and marked incorrect.
Recurring specifics. Quantities that recur include 4 to 6 examples with reasoning, 2 to 4 targeted examples in the shallow reference page, and 10 to 15 clear-case examples marked as wasteful. The phrasing each showing the reasoning for choosing one tool over the plausible alternative is the recurring correctness signal.
Proposal. Add 10 to 15 examples of clear, unambiguous requests.
Why it attracts. Feels comprehensive.
Why it fails. It teaches the easy cases the model already gets right and does not address the ambiguous tail.
When it would be right. Never for refinement. Only as dataset building for evaluation.
Proposal. Add examples grouped by tool with all get_customer cases together.
Why it attracts. Feels organized.
Why it fails. It does not demonstrate discrimination at the boundary, it demonstrates each tool in isolation.
When it would be right. Never. Ambiguous-boundary examples are always preferred.
Proposal. Use use when and do not use when clauses instead of examples.
Why it attracts. Those clauses belong in descriptions.
Why it fails. The question stem explicitly asks which example set is most effective when examples will be added. Substituting a description improvement does not answer the asked refinement choice.
When it would be right. As the first fix before examples are considered, per Rule 15, but not as the answer to which example set is best.
- Swapping the ambiguous anchor from
recent purchasetoaccount balancepreserves the same need: reasoning examples that resolve thecheck_balanceversusaccount_summaryboundary. - Few-shot refinement example for the ambiguous tail:
- The first case shows order tool winning despite ambiguous
my recent purchasephrasing because delivery semantics dominate. The second shows customer tool winning on the same opening phrasing because the downstream intent is profile mutation.
Conditional or hedged boundary language creates ambiguity
Boundary statements that read May also be used for order lookups in some cases or similar conditional phrasing create scope uncertainty. The model cannot determine when the condition holds and treats the tool as optionally applicable to the sibling's domain, reintroducing misrouting.
Conditional language preserves overlap rather than eliminating it. A permissive hedge invites the model to test the sibling domain whenever phrasing is even vaguely order related.
Boundary. Hedged language is sometimes tempting for genuine edge overlaps such as when a single identifier plausibly exists in both systems. Even there the description should still state the definitive preference and the disambiguating test such as if the request contains an order number in format #NNNNN, use lookup_order, otherwise verify whether it is an account identifier.
Recurring specifics. Hedged phrasings that recur include may also be used for, can sometimes handle, and does stuff with data. The correct counterpart phrasing that recurs is USE this for X. DO NOT USE for Y, use [other_tool] for Y plus explicit boundary explanations for ambiguous terms such as laws.
Proposal. Length is the issue, shorten the description.
Why it attracts. Brevity bias.
Why it fails. Clarity of boundaries matters, not length.
When it would be right. Never for hedged boundaries.
Proposal. Specify the database being searched.
Why it attracts. Feels concrete.
Why it fails. Implementation detail does not help selection between two tools that search different entities.
When it would be right. Only as an additional input detail, never as the boundary fix.
- Replacing
order lookupswith any sibling domain such aslegislationversuscourt rulingspreserves the same need for definitive redirection. The definitive redirect template is the controlled variable, not the specific sibling name..
Lifecycle preconditions and validity windows belong in the description
Some tools have preconditions that change what constitutes a valid call. Examples include get_build_log where the build_id becomes valid only after the build has started, and query_features or query_snowflake where format or dialect constraints affect success.
Preconditions are selection-adjacent information. The model optimizes sequencing based on what the description tells it is safe to do next.
Boundary. Shortening descriptions removes preconditions and is marked incorrect. Merging the lifecycle tools into one generic build action parameter is marked incorrect as it obscures the distinct contracts.
Recurring specifics. Preconditions that recur include build identifier becomes valid and edge case build that has not started, dialect differences such as Snowflake SQL with LISTAGG versus string_agg and ILIKE handling, and format validity such as CSV versus XML versus JSON conversion preconditions.
Proposal. Shorten descriptions so the model has less to reason about.
Why it attracts. Minimalism bias.
Why it fails. It removes the very information needed to avoid the edge case.
When it would be right. Never.
Proposal. Instruct the agent to wait a fixed duration.
Why it attracts. Seems to handle timing.
Why it fails. It is a guess that does not generalize to variable build times and leaves the contract ambiguous.
When it would be right. Only as a runtime backoff inside the tool, never as the documented contract.
- Replacing the build domain with any sequenced resource lifecycle preserves the same fix: name the input format, the validity window, and the edge case behavior..
Irreversible or destructive operations must declare permanence
A tool that deletes, archives, or permanently removes records must state in its description that the operation is permanent and irreversible. The examined failure is removes records matching criteria with no mention of permanence, where the agent uses it not knowing deletions are irreversible.
Models cannot infer destructive consequences from a generic capability phrase. Only an explicit scope and consequence statement such as irreversible, do not use for backup files, use archive_file for those gives the model the risk signal needed to avoid catastrophic selection or to prompt for confirmation downstream.
Boundary. This rule does not argue that deletion tools should be excluded from agent access or always require admin credentials as the first fix. Description clarity is the first fix that improves selection at the decision point.
Recurring specifics. Tool names that recur include delete_file versus archive_file on file_id, delete_record style tooling, and the edge phrasing Do not use for backup files on delete. The policy example is that company policy requires archiving backup files, and the backup-tag misrouting is the examined signal.
Proposal. Require admin credentials to prevent casual use.
Why it attracts. Feels protective.
Why it fails. It restricts execution but does not help the agent make the right selection initially.
When it would be right. As an execution guard after description clarity, not as the selection fix.
Proposal. Always confirm with the user before any deletion.
Why it attracts. Safe.
Why it fails. It is a process fix that does not address the description deficiency causing misuse.
When it would be right. As a complementary safeguard, not the highest-leverage improvement.
- Replacing the domain from files to customer records preserves the same rule: generic handling descriptions for deletion operations must include the permanence qualifier..
Typed schemas with enums and field descriptions reduce hallucination
Hallucinated tool arguments on complex multi-part requests correlate with weak schemas: missing field descriptions, missing enums, and hallucination-friendly freeform strings. Strengthening schemas with typed fields, enum constraints, and per-field description text such as ISO 8601 date string e.g.
Schemas are the only machine-enforced shape on arguments. A freeform query string or an underspecified identifier field leaves the model to guess formatting.
Boundary. Schema strengthening is not a substitute for description quality on selection problems, and description quality alone does not fix argument hallucination on large catalogs. For single-tool argument errors such as placing a summary in the wrong field, schema plus description example calls together are the fix.
Recurring specifics. Schema details that recur include per-field description with format hints, type: string, enum with values like ["lookup", "update", "delete"] or ["CRITICAL"] constraints, required, and additionalProperties. Hallucination signals that recur include wrong field for summary and missing project_key on create_ticket.
Proposal. Add more tools to provide finer-grained control.
Why it attracts. Feels like precision.
Why it fails. More tools increase confusion and hallucination on an already overloaded agent.
When it would be right. Never for hallucination caused by overload.
Proposal. Set tool_choice: any or auto tweaks.
Why it attracts. Feels like it changes evaluation strictness.
Why it fails. It does not address shape guidance for arguments.
When it would be right. Only when the risk is avoidance of tool use, not hallucinated arguments.
- Replacing the enum domain from operation to severity to identifier type preserves the same mechanism: typed enum plus field description reduces hallucination.
- Typed guard example for notification gating that complements Rule 12:
- json { "name": "send_notification", "description": "Sends a notification when severity is CRITICAL only. Trigger condition: severity must equal CRITICAL.
Decision boundary between consolidation and splitting
Consolidation and splitting are opposite architectural moves and the exam tests the ability to choose correctly. Consolidate when tools share the same action and output but differ only by input field value or repetitively by data shape such as eight lookup_user_by_* variants or 19 transformations handling pivot versus percentile versus currency.
Each move addresses a different structural cause. Consolidation reduces count where tools are artificially multiplied on a single dimension.
Boundary. The near case is a generic analyze_content that could either be enumerated with a mode parameter or split. the tested material marks the mode parameter as incorrect because the generic description remains.
Recurring specifics. Consolidation targets that recur include 19 transformations into transform_data, eight lookups into lookup_user, and creation plus update pairs that should remain split as get_order_status versus modify_order where reads and writes differ. Splitting sources that recur include analyze_document into three tools and query_features into eight geospatial tools.
Proposal. Consolidate broad tools such as analyze_document behind a single analyze_any_source with auto-dispatch.
Why it attracts. Feels like it eliminates choice.
Why it fails. It pushes routing inside the tool where the model loses visibility and each behavior still needs a distinct description.
When it would be right. Never for broad semantic breadth.
Proposal. Split field variants into region-specific sub variants.
Why it attracts. Finer routing.
Why it fails. It multiplies the same problem.
When it would be right. Never for field variants.
- Replacing the domain from user lookup to billing versus shipping preserves the choice logic. The exam repeats the same decision across many surfaces to ensure the pattern is recognized, not memorized on one example..
Pre-routing classifiers and keyword routers are over-engineered as first steps
Pre-routing classifiers that parse the user message for keywords or identifier patterns and pre-select the tool before the agent runs, or that detect MIME type or URL pattern to route analysis versus synthesis, are repeatedly offered as answer options. Evidence consistently marks them incorrect as the first step because they bypass the model's natural language understanding, are brittle to paraphrase, add infrastructure complexity and failure points, and do not fix the description contract for other callers.
Descriptions are the low effort, high leverage fix of a single edit to each tool definition that applies to every subsequent decision. A classifier is a larger engineering commitment that replicates the same disambiguation in code but without addressing the standing contract other agents and all future phrasings will rely on.
Boundary. Classifiers become relevant only where the problem is outside Task 2.1 scope such as Task 2.3 distribution and loading strategy. For example, dynamically loading only the tools relevant to the current task is a scaling technique for large catalogs, not a substitute for fixing two-tool overlap.
Recurring specifics. Proposals that recur include routing layer that parses input each turn and pre-selects the tool from detected keywords and identifier patterns, MIME type or URL pattern routing, keyword-sensitive system prompt routing, pre-routing before coordinator delegation, and detect file extension or URL pattern. Each is marked over-engineered as a first response for minimal-description misrouting.
Proposal. It is needed because few-shots do not generalize.
Why it attracts. Correct that few-shots generalize poorly, but the correct generalization lever is description quality, not a classifier.
Why it fails. It treats a description poverty problem as an infrastructure problem.
When it would be right. Only when description clarity and threshold distribution have already been addressed and a narrow cross-tool routing policy must be enforced for compliance reasons outside the examined scope.
- Replacing the detection signal from
emailversusphoneto file extension versus URL preserves the same over-engineering judgment. The remediation hierarchy is invariant: descriptions first, then targeted ambiguous-case examples, then architectural distribution if needed..
Tool ordering and positional tricks have no reliable effect
Suggestions such as moving a tool to position 1 in the tools array, renaming with alphabetical advantage, or encoding priority in install order are consistently marked incorrect. Tool array position does not reliably influence selection.
Array order is an implementation detail of the request payload, not a documented selection signal. Evidence shows sparse descriptions remain sparse regardless of position.
Boundary. There is no nearby opposite where ordering is the correct fix. The only place where limiting visibility matters is distribution: specialized subagents and dynamic loading that reduce the visible set per agent are architectural fixes, but they operate by removing tools from consideration, not by reordering them.
Recurring specifics. Proposals that recur include move to position 1, rename to earlier letter, and rely on install order. MCP versus built-in ordering such as mcp__ prefix signaling lower priority is also marked irrelevant.
Proposal. Tool names matter more than descriptions so renaming alone fixes selection.
Why it attracts. Confuses the weak name signal with the primary description signal.
Why it fails. Both matter, but description dominates, renaming without rewriting leaves the primary signal unchanged.
When it would be right. Only as a paired rename plus rewrite, per Rule 6.
- Position, alphabetical trick, and install order are all variants of the same ordering fallacy and are treated identically..
Field mapping failures require example calls and required-field documentation
Some tools fail not on which tool is chosen but on how arguments are formed. Evidence shows create_ticket where agents frequently place the summary in the wrong field and omit the required project_key so most calls fail validation, and fetch_record style cases where the model invents fields.
# Correct invocation pattern the description should illustrate
create_ticket(
project_key="SUPPORT",
summary="Login timeout on checkout for customer CUS-88412",
component="backend",
status="open"
)Required-field and field-mapping knowledge has no other home. Built-in tools for normal schemas do not teach the agent which field holds the summary or when project_key is required.
Boundary. Retry loops, making every field optional, or moving field requirements into the system prompt as a checklist are each marked incorrect. Retrying repeats the same malformed call, relaxing required fields yields unusable records, and duplicating the contract in the prompt patches a single consumer while the tool contract stays ambiguous.
Recurring specifics. Field names that recur include summary, project_key, component, status, customer, identifier. The validation failure that recurs is omits required project key, most calls fail validation.
Proposal. Add a retry loop so failed calls are attempted again.
Why it attracts. Handles failure.
Why it fails. Repeats the same malformed shape.
When it would be right. Only when transport is flaky, not when field mapping is wrong.
Proposal. Make every field optional so calls always validate.
Why it attracts. Eliminates validation errors.
Why it fails. Produces unusable records that are missing mandatory data.
When it would be right. Never.
- Replacing
create_ticketwith any structured creation tool where summary placement is the ambiguity preserves the same fix: per-field prose plus a worked example call plus correctrequired. - Worked example call:
- The example belongs in the description prose so the model has both the abstract schema and a concrete field mapping it can replicate..
Mode-parameter overload should be replaced by tool identity
A single tool exposed with a mode parameter that spans many behaviors such as query_features with mode variants for point lookup, radius search, nearest-neighbor, route distance, and bounding-box export is an overloaded description that must cover every behavior. The agent picks the wrong mode or omits required parameters in 33 percent of radius and nearest-neighbor requests.
A mode parameter collapses many discrete decisions into one description. The model must infer both which tool to use and which mode it is in from the same text, with one shared prose that plausibly matches every request type.
Boundary. A single mode enum can be appropriate when behaviors are truly minor variants of the same operation with identical parameter sets and outputs. Evidence never shows that case succeeding.
Recurring specifics. Mode counts that recur include eight behaviors on query_features, three on process_file with parse, convert, validate, four on media transcoding. Error contracts that recur alongside include returning {"features": []} for both empty and unreachable layers or {"isError": true} style generic failures that do not distinguish retryable from permanent, which Rule 8 and splitting still drive as the primary selection remedy.
Proposal. Keep one tool but add mode usage hints and retry guidance to the description.
Why it attracts. Enumerates modes in prose.
Why it fails. One description must still cover every behavior, and when parameters or service state are ambiguous the hints lower misrouting only from 33 to 26 percent without stabilizing selection.
When it would be right. Never when modes need different parameter subsets.
Proposal. Split but keep short descriptions such as Parses a file with a prompt routing block.
Why it attracts. Implements identity but defers boundaries to the prompt.
Why it fails. Short descriptions remain ambiguous and prompt routing reintroduces keyword brittleness.
When it would be right. Never. Each split tool needs its own boundary prose.
- Replacing
query_featureswith any multi-mode tool such asmanage_orderorprocess_filepreserves the same judgment. The controlled lever is not the domain but whether modes share or diverge on required parameters.
Distinctions that decide answers
| This | Not this | How to tell them apart |
|---|---|---|
| Production grade description with boundaries and examples | Minimal single sentence description | The production form carries purpose, expected inputs with formats, example queries, edge cases, and explicit when not to use guidance, while the minimal form supplies only a headline that two plausible tools share. |
| Tool splitting for overlapping single tool | Prompt level clarification for overlap | Splitting replaces one generic tool with several narrow, well named tools each with its own contract. Prompt clarification keeps the generic tool and tries to steer selection in text, which fails when the single entry point must cover distinct jobs. |
| Description driven fix | Routing classifier as first step | The description fix addresses why the model is confused between two tools in a small set. A classifier bypasses model judgement and adds infrastructure, and is over engineered as the first response when sentence level clarity would suffice. |
| Renaming with description rewrite | Keeping confusing names with longer prose | Renaming changes the lexical trigger the model pattern matches on. Keeping the original name and lengthening the description leaves the name ambiguity in place. |
| Interface scope for selection | Enforcement logic for compliance | Interfaces guide the model toward the right retrieval. Compliance such as refunds is enforced programmatically in executor code and not in description wording, so no description variant can replace a gate. |
Traps
The tempting answer. Add a few shot example block to patch selection when descriptions are the root cause because examples feel tangible.
Why it fails. Examples improve parameter ambiguity such as date formats or prefix conventions, but they impose token overhead and do not address the confusion between two tools on the selection axis, which is driven by what the descriptions say about when each tool applies.
What is correct. Rewrite both descriptions to the five part template with formats, example queries, and explicit boundaries, then remeasure routing before adding examples.
The tempting answer. Introduce a deterministic classifier that picks the tool before the model sees the prompt, guaranteeing selection between two similarly described tools.
Why it fails. That guarantee is premature when the underlying problem is sentence level vagueness and it adds a system dependency the exam treats as disproportionate early compared with a low effort description rewrite.
What is correct. Revise descriptions and system prompt coupling first, then evaluate whether a classifier is still needed under distribution pressure.
The tempting answer. Merge get_customer and lookup_order into one lookup with polymorphic identifier to remove the choice being made incorrectly.
Why it fails. Consolidation removes the choice but collapses distinct return shapes and business boundaries, leaving the model with one large schema where two focused ones would have been easier to choose between, with larger implementation cost than revising descriptions.
What is correct. Keep two focused tools with explicit format and boundary guidance and review prompt wording, using consolidation only for homogeneous overloads such as nineteen transform variants.
The tempting answer. Ship the description fix alone even where the prompt contains instruction templates such as always check customer details before proceeding.
Why it fails. Keyword sensitive prompt instructions can create unintended tool associations that override even well crafted descriptions, so prompt and interface must convey the same routing policy and the fix is incomplete without that alignment.
What is correct. Inspect the system prompt for triggers, remove or narrow the conflicting phrasing, and remeasure stability on the same benchmark.
The tempting answer. Keep analyze_document as one tool and lengthen its umbrella description rather than splitting into bounded tools because one tool feels simpler to maintain.
Why it fails. A generic analyser that returns results cannot simultaneously specify extraction, summary, and verification contracts crisply, and selection error persists until each job has its own typed entry point with narrow description and schema.
What is correct. Split the generic tool into extract_data_points, summarize_content, and verify_claim_against_source each with a focused description and typed contract.
Structured Error Responses
Return isError plus errorCategory and isRetryable so the agent can distinguish access failure from valid emptiness and retry only what is retryable.
What you need to know
When an MCP tool fails, the error response it returns to the agent carries more weight than the failure itself, because the agent's next decision is driven by that response, not by the exception you caught on the server. A generic message such as Operation failed contains no signal about what went wrong, whether the input was malformed, whether a policy blocked the request, whether credentials are missing, or whether a retry might succeed, and the model given no structured field to route on either retries blindly, escalates prematurely, or treats an error as a successful empty result. The MCP protocol provides an isError flag for exactly this purpose on tool_result like responses, signalling that execution failed so the model can reason about recovery rather than interpreting the error text as data. The flag alone is coarse, though, and production systems extend it with a small taxonomy that lets the model branch to the right recovery action.
Four categories describe the space of tool failures the exam tests, each demanding a distinct recovery strategy that only works when the agent can distinguish the category from structured metadata rather than from free text. Transient errors such as timeouts, overload, and rate limits are the one category where the underlying system is temporarily unreachable yet the request itself is valid, so the correct action is to retry the same call after a delay, ideally with exponential backoff and jitter at the executor layer before the model retries itself. Validation errors such as invalid formats, missing required fields, or out of range enum values reflect a malformed request, and the correct recovery is to reformulate the input, not to resend the same payload, so the error must include the accepted format and an example. Business errors such as refund exceeds policy limit represent requests that are technically correct but violate business constraints, and the correct action is to take an alternative path or escalate rather than retry. Permission errors such as access denied reflect missing credentials or insufficient scope on the caller, and again the correct action is to retry only as a different principal or to escalate, never to resend the same call and expect a different outcome.
A narrow convention clarifies how two fields interact. isRetryable answers one question only, will resending this exact request with no change in input, authentication, or business context succeed. Only transient failures get true on that field, because the request was valid and the system was briefly unavailable. Validation, business, and permission are false on that field, because something must change first, the input for validation, the request strategy or an escalation decision for business, and the caller for permission. The surrounding errorCategory field is where the recovery instruction lives when the retryable answer is false, telling the agent to correct an identifier, restructure a call, or seek an alternative workflow, and the distinction inside the false rows matters most, with validation recoverable by the agent alone and business and permission requiring a different path or a different principal. Reading isRetryable to decide whether to resend as is, and then reading errorCategory to decide what to do when retry is not the answer, is the token budget safe branching the exam expects.
A closely related distinction the exam tests directly is between an access failure and a valid empty result, which share no recovery logic yet are trivially confused when a tool returns an empty array for both cases. An access failure means the tool could not reach the data source because of a timeout, authentication failure, or service outage, so whether the data exists remains unknown and a retry may help. A valid empty result means the tool reached the source, ran the query, and found no matches, so the request succeeded, the absence is meaningful, and a retry will return the same absence. A scenario tested repeatedly illustrates the failure mode. A lookup tool returns an empty array for a customer that does not exist, and because the envelope cannot distinguish could not fetch from found nothing, the agent retries three times and then escalates, while a reviewer shows the account never existed and the data shape misled the model into treating a successful zero match as a retriable failure. The fix is to structure successful empties so the empty payload is materially distinct from an access failure, with isError false and resultCount zero plus descriptive text rather than an undifferentiated empty array on isError true.
In multi agent orchestration, the same taxonomy gains a local recovery dimension. Subagents handle transient failures themselves before involving the coordinator, propagating only unresolved failures upward with partial results and a record of what was tried, so the coordinator can distinguish found nothing from could not search and decide whether synthesis should proceed, retry, or escalate. That layering keeps retries cheap and local while preserving the signal the coordinator needs for global decisions without silently suppressing errors as empty successes.
The envelope and how isRetryable narrows to one meaning
At the protocol level, the standard MCP error representation for an unsuccessful tool call uses isError true in the tool response alongside content as a human readable message, while a successful call with no data returns isError false with a descriptive text payload and a typed field such as resultCount. The structured envelope that extends isError includes at minimum errorCategory as an enum string and isRetryable as a boolean, plus a message and an optional technicalDetail whose content is server side rather than model facing, and a context object that may carry resource, sanitised input, attemptNumber, retryAfterMs, or a suggestion the model can act on.
Category values may vary by codebase, references include transient, validation, business, permission, and in richer implementations rate-limit, auth, not-found, and permanent as distinct codes with matching retry semantics, where transient and rate-limit are retryable after delay and the rest are not. HTTP idioms align loosely but not perfectly with this taxonomy, which is why the envelope matters more than status code alone. A 503 behind a transient error suggests retry after backoff, a 422 on a validation error says correct the input, a 403 on a permission error says elevate or escalate rather than retry, and business errors rarely map to a single status code at all, often returning 200 with a domain reason.
Why valid emptiness must not look like failure
A lookup tool that returns an empty array for a customer that does not exist and also for a store timeout shares a data shape across two recovery branches that must not share. The agent parsing the phrase not found versus unavailable is fragile. The robust fix is to put valid emptiness on the success side of the envelope, with isError false, a text field that says the query executed successfully but found no matches, and a numeric resultCount of zero, so the empty payload is materially distinct from an access failure that carries isError true, errorCategory transient, and isRetryable true.
That split also prevents wasteful retry counts. A successful zero match will return the same zero on every retry, so branching on the envelope suppresses retries that string parsing would have triggered. The pattern extends to graceful degradation where a tool backed by several services returns what it can with per section degraded versus unavailable statuses, exposing degradedSections and availableSections to the agent so a partial dashboard still renders rather than failing the whole workflow.
Retry timing and where it belongs
In tool level retry wrappers, transient detection is encoded explicitly, for example checking status greater than or equal to 500, code equals TIMEOUT, or code equals RATE_LIMITED before choosing to retry, and distinguishing that detection from the outer isRetryable flag that the model reads in the tool result block. The executor layer owns exponential backoff with jitter, caps attempts, enforces maxDelayMs, and only then surfaces a structured failure to the model, rather than letting the model retry immediately and hammer a failing backend many times in rapid succession.
A backoff configuration object with maxAttempts, baseDelayMs, maxDelayMs, and useJitter, with a full jitter algorithm that scales baseDelayMs exponentially and caps at maxDelayMs, matches the executor timing principles in the loop harness rather than leaving cadence to the model. That separation is why transient is the only isRetryable true case. The request was valid, the system was briefly unavailable, and the same request after a delay has a chance to succeed without changing inputs or principals.
Branching table the agent follows after the error
Transient and rate-limit are retryable after delay, with the agent either retrying locally a few times or the executor handling the first retries before the model sees the result. Validation is not retryable as is but agent recoverable via correction, where the response names the expected format such as USR dash XXXXX and provides a suggestion to correct the prefix style. Business and permission are not retryable and not agent recoverable alone, requiring an alternative path or escalation because the policy or credential constraint will block every resend of the same payload.
Interpreting isRetryable false as abandon the task conflates three distinct non retryable branches. The correct reading is cannot retry this exact call as written, then consult errorCategory for what to do next, whether to correct the identifier, to restructure the call, to seek approval, or to escalate. That preserves budget and avoids both infinite retry loops on business policy errors and silent suppression that hides failure from the coordinator.
Authoritative mechanism reference
The sections below expand the tested rules into the complete documented surface: ownership, lifecycle, version differences, production boundaries, and adjacent mechanisms.
Mechanism reference: The protocol-level tool_result block
The Messages API defines tool_result as a content block type that appears in a user message following an assistant tool_use block. The protocol fields are, in full:
type- the literal string"tool_result".tool_use_id- a string that must match theidof thetool_useblock this result answers. Mismatches break the conversation.content- optional. The result of the tool, as a string, as a list of nested content blocks (text, image, document, search_result), or as a list of document blocks. Most tool results serialize a JSON string here.is_error- optional boolean. Set totruewhen the tool execution resulted in an error. When omitted it defaults tofalse.
Two structural ordering rules are protocol-enforced and produce a hard HTTP 400, not a soft failure. First, every tool_result block must immediately follow the assistant's tool_use message; you cannot insert any other message between them. Second, inside the user message that carries results, every tool_result block must come first in the content array; any plain text block must come after all tool results. A user message that leads with text before a tool_result is rejected.
The is_error boolean is the only failure signal the protocol gives Claude. It is deliberately minimal. It tells Claude "this tool call did not succeed" but carries no category, no retry hint, and no recovery instruction. That is by design: Anthropic leaves the content shape open so you can tailor it to your domain. The lesson is explicit that is_error is the one part the API standardizes, and "everything else in this lesson, the errorCategory taxonomy, the isRetryable flag, the backoff parameters, is application-layer convention that you design, not an Anthropic-mandated schema."
Mechanism reference: Full value space of the content field and ordering boundary conditions
The content field is the most misunderstood part of the protocol because its accepted type is unusually permissive. The Messages API accepts four distinct shapes for content on a tool_result block, and the chosen shape changes how Claude parses the result. First, a plain string, which is the common case where your application serializes a JSON object or a human-readable message. Second, a list of nested content blocks, where each element is one of text, image, or document. Third, a list of document blocks only. Fourth, when the result is an error, the same string or block list carries the error envelope; is_error: true is orthogonal to whether content is a string or a block list. A tool that returns a screenshot of a failed render can therefore set is_error: true and still pass an image block as visual evidence, and the model reads both the failure flag and the picture.
The boundary condition that breaks real integrations is the request-size ceiling. The documented hard limit is a 32 MB total request size for the Messages API, enforced as a 413 request_too_large error, not a 400. This is a request-wide limit, not a per-block cap, so a single enormous tool_result can fail the whole call even when every individual block is well formed. In practice the model context window is the tighter bound: large results degrade reasoning long before they approach 32 MB, so the production guidance is to summarize, paginate, or truncate well under any byte ceiling. The 413 is the protocol's last-resort guard; the application layer owns the discipline of never reaching it.
The two ordering rules are protocol-enforced and fail hard, but their exact boundaries matter. Rule one: a tool_result block must appear in a user message that directly follows the assistant tool_use message, with no intervening assistant or user turn. You cannot, for example, place a thinking block or a plain text message between the tool_use and its result. Rule two: inside one user message, every tool_result block must precede any text block. The sharp boundary is that a text block placed before even a single tool_result invalidates the entire message with a 400, even if every tool_result is otherwise correct. The SDK agent loop handles this automatically when results are returned as one user turn, but hand-built message arrays in raw API calls are where the 400 surfaces.
The wire example below shows the protocol surface at full extent: two tool results in one user turn, one a successful lookup rendered as a nested document block, the other a failure rendered as a text envelope plus an image block. Both respect the ordering rules, and the failure sets is_error at the block level rather than burying it inside the content.
{
"role": "user",
"content": [
{
"type": "tool_result",
"tool_use_id": "toolu_01renderOk",
"content": [
{
"type": "document",
"source": {
"type": "text",
"media_type": "application/json",
"data": "{\"customerId\":\"CUST-48721\",\"tier\":\"premium\",\"status\":\"active\"}"
}
}
]
},
{
"type": "tool_result",
"tool_use_id": "toolu_01renderFail",
"is_error": true,
"content": [
{
"type": "text",
"text": "{\"isError\":true,\"errorCategory\":\"transient\",\"isRetryable\":true,\"message\":\"Render service unreachable.\"}"
},
{
"type": "image",
"source": {
"type": "base64",
"media_type": "image/png",
"data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+M8AAAMBAQDJ/pLvAAAAAElFTkSuQmCC"
}
}
]
}
]
}What it proves: a failure is a block-level is_error: true regardless of whether content is a string, a nested block list, or a mix of text and image. The protocol failure signal travels independently of the result payload shape. The single user turn carries both results in the required order, tool results first, so no 400 is raised.
Mechanism reference: Backoff, idempotency, and the write-timeout boundary
The transient category with isRetryable: true is correct for read timeouts, but a timeout during a write is a different boundary. A read-only query that times out left no side effect, so a blind retry is safe. A write that times out halfway through may have partially committed on the upstream, so retrying the same call risks duplication: a double charge, a duplicate order, a second notification. The lesson material flags this read-versus-write distinction as exam-relevant, and the safe pattern is an idempotency key the upstream uses to detect and collapse repeats. The key must be supplied by your tool wrapper, never inferred by the model, because the model has no durable record of which calls it already issued.
The code below shows a tool wrapper that combines exponential backoff with jitter, the same shape the lessons recommend, and stamps an idempotency key on every mutating call. A read timeout retries under backoff. A write timeout retries only because the idempotency key lets the upstream treat the second attempt as a replay of the first, returning the original outcome instead of creating a duplicate. A non-transient error escapes the loop and becomes a structured tool_result error for the model.
interface IdempotentRetryConfig {
maxAttempts: number
baseDelayMs: number
maxDelayMs: number
useJitter: boolean
}
function backoffWithJitter(attempt: number, config: IdempotentRetryConfig): number {
const exponential = Math.min(config.baseDelayMs * Math.pow(2, attempt - 1), config.maxDelayMs)
return config.useJitter ? Math.random() * exponential : exponential
}
async function withIdempotentRetry<T>(
fn: (idempotencyKey: string) => Promise<T>,
isTransient: (error: unknown) => boolean,
config: IdempotentRetryConfig = { maxAttempts: 3, baseDelayMs: 300, maxDelayMs: 4000, useJitter: true }
): Promise<T> {
const idempotencyKey = crypto.randomUUID()
let lastError: unknown
for (let attempt = 1; attempt <= config.maxAttempts; attempt++) {
try {
return await fn(idempotencyKey)
} catch (error) {
lastError = error
if (!isTransient(error) || attempt === config.maxAttempts) break
await new Promise(r => setTimeout(r, backoffWithJitter(attempt, config)))
}
}
throw lastError
}
async function chargeCustomer(customerId: string, amountCents: number) {
return withIdempotentRetry(
(key) => paymentsApi.charge({ customerId, amountCents, idempotencyKey: key }),
(error) => error instanceof TransientError,
{ maxAttempts: 3, baseDelayMs: 300, maxDelayMs: 4000, useJitter: true }
)
}What it proves: backoff and idempotency are separate concerns that must be combined for mutating tools. Backoff alone prevents thundering-herd retries; idempotency alone prevents duplicate side effects when a retried write reaches an upstream that already acted. Together they make isRetryable: true safe for the write-timeout boundary the exam tests. The idempotencyKey is stable across the retries of one logical call but unique per call, which is exactly the contract an upstream needs.
Mechanism reference: Application-layer error envelope
Because is_error alone is insufficient for intelligent recovery, production tool code wraps structured metadata inside the content string. This is the "structured error envelope" the reference page shows. The envelope is a JSON object whose keys are your convention, not protocol fields. A representative envelope, matching the reference page and our tool-error-handling lesson, carries:
isError- a redundant in-content boolean mirroringis_errorso downstream parsers that read the content string (not the block) still see the failure.errorCategory- one of the taxonomy values (transient,validation,business,permission, and in the wider modelrate-limit,auth,not-found,permanent).isRetryable- boolean answering "will resending this exact request work?" Onlytransient(andrate-limitafter a wait) gettrue.message- a human-readable description Claude can include in its response to the user.description(reference page) orcontext/technicalDetail(our lesson) - optional fields carrying the recovery hint, the affected resource, the sanitized input, the attempt number, or a suggestion for what to try instead.
The critical rule: these envelope keys live inside content, which is a stringified JSON object. They are NOT keys of the tool_result block. The block itself still needs type, tool_use_id, content, and is_error. A correct structured error therefore sets is_error: true on the block AND embeds the envelope in content. Setting only the envelope inside content while leaving is_error absent (defaulting to false) is the silent-failure anti-pattern: Claude reads a successful block whose content describes an error, and reasons as if the call worked.
Mechanism reference: The four required categories and their value space
The reference page specifies four categories. Each maps to a recovery strategy and to a fixed isRetryable value. The full set, including the wider lesson taxonomy, is:
| Category | isRetryable | Recovery | Example |
|---|---|---|---|
transient | true | Resend the same call after a delay | DB connection timeout, 5xx, network blip |
rate-limit | true (after wait) | Wait for Retry-After, then resend | Upstream 429 with backoff window |
validation | false | Correct the input, send a new call | Wrong ID format, out-of-range value |
business | false | Take an alternative path or escalate | Refund exceeds policy limit |
permission | false | Escalate or retry as a principal with access | Insufficient role, missing scope |
auth | false | Report credential problem, escalate | Expired token, invalid key |
not-found | false | Report missing resource, try alternative | Record deleted, file absent |
permanent | false | Report failure, do not retry | Quota permanently exhausted, legal block |
The exam guide v1.0 frames the four reference categories. The wider taxonomy is the production reality and refines them: rate-limit is a transient variant that must include a wait time because Claude has no internal clock; auth overlaps permission (both need a different principal or credential, neither is retryable with the same call); not-found is the boundary case between error and empty result and is treated as an error distinct from a valid empty result.
The single most-tested nuance is the meaning of isRetryable: false. It does NOT mean "abandon the task." It means "resending this exact request is pointless because something must change first." For validation the change is the input (the agent fixes it unaided). For business and permission the change is a policy exception or a different principal (escalation). Three categories are non-retryable, but only two are dead ends.
Mechanism reference: Category deep dives
Each category has behaviour the exam probes beyond the boolean. The details below are the full value space.
Rate-limit errors are uniquely identifiable by a Retry-After header or a retryAfterSeconds field in the upstream response. They are retryable but only after waiting the indicated duration. Claude has no internal clock, so it cannot independently know when to retry; you must either include the wait time in the error description or handle the backoff inside your tool wrapper. Retrying before the window expires wastes a call and may extend the cooldown. In the wider taxonomy rate-limit is isRetryable: true but qualified "after wait"; in the four-category exam model it folds into transient. Either way the wait time must be communicated.
Timeout errors require a sub-distinction the lessons call out: client-side versus upstream. A client-side timeout means your tool imposed a deadline on an upstream call and gave up; the upstream may or may not have partially executed. An upstream timeout (for example an HTTP 504) means the service responded that it could not complete. A timeout on a read-only query is safe to retry; a timeout halfway through a write may have partially committed, so blind retry risks duplication. The category transient with isRetryable: true is correct for read timeouts, but idempotency keys or check-status calls are needed for writes. The exam tests the read-versus-write distinction.
Validation errors are never retryable with the same input. The recovery is a reformulated call with corrected parameters. A validation error that says only "invalid input" forces Claude to guess what failed; the envelope must state exactly which field violated which rule and what the expected format is. The lessons warn that marking a validation error isRetryable: true causes Claude to retry with the same invalid input indefinitely, the purest form of the retry anti-pattern. The context.input field should carry the sanitized bad value (never credentials) and context.suggestion should state the fix.
Authentication and permission errors are never retryable from Claude's perspective. A missing API key, expired token, or insufficient role will not be fixed by resending the same call. These errors must be logged server-side with full detail and returned sanitized: the message explains the issue without exposing the key, token, or internal scope string that an attacker could abuse. The permission category in the four-category model maps onto auth in the wider taxonomy; both require escalation or a different principal. Credential material must never appear in content or in context.
The three-layer error model is the conceptual frame that ties protocol, convention, and presentation together. Layer one is the tool implementation, where your code sees raw error objects, stack traces, and HTTP status codes as native exceptions. Layer two is the tool_result block, where Claude sees a sanitized, categorized error (the envelope). Layer three is Claude's natural-language response to the end user, which may include only as much of the error as is helpful. The rule is never to mix layers: raw stack traces and internal addresses stay in layer one and are logged, not returned. This is why the envelope separates message (shown to Claude and the user) from technicalDetail / context (logged, optionally shown to Claude for debugging but never to end users).
Graceful degradation is the pattern for tools that depend on multiple upstream services. When one service fails, the tool should still return what it can, marking each section ok, degraded, or unavailable. This is distinct from partial results in a batch (where some items fail), but both communicate mixed outcomes so Claude can report accurately rather than hiding failure. A degraded section tells Claude "here is stale or fallback data, label it as such"; an unavailable section tells Claude "this part genuinely failed." Serving fallback data without indicating its source is itself an anti-pattern: it creates false confidence.
Fallback chains operate at two levels. A tool-level fallback tries an alternative implementation (for example a premium weather API, then a free one, then an estimate). An agent-level fallback lets Claude choose a different tool. Either way the chosen source must be labelled in the result so Claude and the user know they are seeing fallback data. When all fallbacks are exhausted, the tool returns a structured permanent error rather than a silent empty result. The exam tests whether a candidate designs fallbacks that are transparent and orders them from best to worst quality.
Mechanism reference: Access failure versus valid empty result
This is the distinction the reference page flags as "the distinction to nail" and the exam tests directly. It deserves the deepest treatment because it is where protocol, convention, and reasoning interact.
A tool query has three outcomes, not two: success with data, success with no data, and failure to reach the data. The failure mode that breaks recovery is collapsing the last two into one shape.
A valid empty result means the tool successfully executed, reached the data source, ran the query, and found zero matches. The operation is a normal, expected outcome. The agent should report "no results found" and move on. There is nothing to retry because the query was correct and the answer is genuinely "nothing matches."
An access failure means the tool could not reach or query the data source at all. A timeout occurred, authentication failed, the service was down, or the network dropped. The data might exist, but the tool never got to check. The agent should decide whether to retry, escalate, or degrade, because the question "does anything match?" is genuinely unanswered.
The trap is structural. If a tool returns the same shape for both, an empty array [] whether it found nothing or reached nothing, then Claude cannot tell them apart. It will treat a successful empty result as a failure worth retrying, or treat an access failure as a benign empty result and confidently report "the customer does not exist" when the truth is "the database was unreachable."
The fix is to make the two outcomes look nothing alike at the protocol boundary. A valid empty result is a success block: is_error absent or false, with an explicit empty-state marker such as resultCount: 0 and a message stating the query ran and matched nothing. An access failure is an error block: is_error: true, errorCategory: "transient" (or auth/not-found as appropriate), isRetryable: true for transient, with a description stating the query did not execute.
The lesson material makes this the central anti-pattern. A queryDatabase function that catches an exception and returns [] silently is explicitly labelled DANGEROUS: Claude concludes there are no results and proceeds with "no orders found" reasoning when the real problem is a broken connection. The safe version returns a structured error object with isError: true, errorCategory: "transient", isRetryable: true.
The reference page's practice scenario grounds this precisely: a tool returns an empty array after a customer lookup, the agent retries three times, then escalates to a human, and analysis shows the customer's account simply does not exist. The root cause is that the tool does not distinguish access failures from valid empty results, so the agent treats no-matches as a retriable failure. The correct answer to that scenario is "the tool does not distinguish between access failures and valid empty results."
A subtler boundary is not-found as an error versus empty as a success. A search that returns zero rows for a valid query is a success with empty data. A lookup by a specific identifier that expects exactly one record, where that record does not exist, is usually modelled as a not-found error (or as an empty success, depending on the tool's contract). The distinguishing question is whether "no match" is a normal outcome of the query or a sign that the requested resource is absent. Our lessons handle both: tool-error-handling returns errorCategory: "not-found" from a database lookup that expects a row, while tool-result-handling returns items: [], count: 0, message: "No orders found for this customer." for a search where empty is expected. The contract must be explicit and consistent across a tool suite.
Mechanism reference: Error propagation in multi-agent systems
In multi-agent architectures the reference page states a principle of local recovery with selective propagation. This is consistent with our lessons and with the agent-sdk guidance on building effective agents. The mechanism has three rules.
Rule one: subagents implement local recovery for transient failures. If a web-search subagent times out, it retries internally before bothering the coordinator. The coordinator should not see a transient blip; it should see either a result or a genuinely unrecoverable failure.
Rule two: only propagate errors that cannot be resolved locally. If all internal retries fail, the subagent reports the failure upward. This avoids two anti-patterns the reference page names. The first is silently suppressing errors by returning empty results as success, which hides failure from the coordinator and leaves it making decisions blind. The second is terminating an entire workflow on a single failure, which throws away all the partial progress from sibling steps.
Rule three: include partial results and what was attempted. The coordinator needs context: "I searched 3 of 5 sources successfully. Sources 4 and 5 timed out. Here are partial results from the 3 successful sources." This is the multi-agent analogue of the partial-result pattern in tool-result-handling, where a batch that processes 8 of 10 records returns status: "partial" with both successes and errors rather than failing the whole batch or hiding the failures.
The ownership of this boundary is shared. The subagent's own code owns local retry and the decision to propagate. The coordinator's code owns interpreting the propagated error, choosing escalation or alternative workflow, and folding partial results into its synthesis. The protocol owns only that the propagated failure arrives as a tool_result block with is_error: true (if the subagent is itself invoked as a tool) or as a message describing the failure (if the subagent communicates by message). The category and partial-result metadata are once again application convention, agreed between subagent and coordinator.
Ownership map
Which layer owns which guarantee matters because the exam repeatedly tests whether a candidate knows what the protocol does versus what application code must do. The table below is the authoritative allocation.
| Layer | Owns | Does NOT own |
|---|---|---|
| Protocol (Messages API) | The tool_result block schema: type, tool_use_id, content, is_error. Ordering rules. The 32 MB request-size limit. | Any error category, retry hint, recovery instruction, or content shape beyond the string. |
| Model (Claude) | Reading is_error to know a call failed. Reasoning about recovery from the structured content you provide. Deciding retry, fix-input, escalate, or give-up based on your metadata. | Producing or validating errorCategory / isRetryable. The model does not enforce them. |
| Your application code (tool executor) | Building the tool_result block. Setting is_error correctly. Designing and serializing the error envelope. Distinguishing empty-result from access-failure. Sanitizing credentials and stack traces out of content. | Nothing protocol-level; the API validates the block shape only. |
| SDK / agent loop | Matching tool_use_id to its result. Appending the result as a user turn. Optionally running retry-with-backoff wrappers. | Deciding the error taxonomy; that is your design choice. |
| MCP server (if used) | Returning CallToolResult with isError (camelCase) and content. Mapping internal exceptions to that shape. | The Anthropic block's is_error is the transport's job to set; the server's isError becomes the content of the tool_result block after the MCP client translates it. |
| Upstream infrastructure | The actual transient/permanent behaviour: timeouts, rate limits, outages, auth rejections. The Retry-After header on a 429. | How your tool interprets or surfaces those; that is your code. |
The single sentence to remember: the protocol tells Claude a tool failed; your code tells Claude why and what to do. If your code fails to set is_error and fails to embed a category, the protocol cannot compensate, and Claude will treat the result as success.
Ownership map: Ownership boundary conditions in practice
The ownership table is easy to read as a partition, but the exam probes the seams where two layers touch. The first seam is the tool_use_id match. The protocol validates only that the string on the tool_result block equals some prior tool_use id; it does not validate that the right result reached the right call, nor that the content matches what that call would produce. Your application code owns that correctness. A mismatched id returns a 400, but a matched-yet-wrong id is a silent logic bug the protocol cannot see.
The second seam is the MCP translation. When a tool is served through the Model Context Protocol, the server returns CallToolResult with a camelCase isError. The MCP client is responsible for translating that into the Anthropic tool_result block, setting the snake_case is_error and carrying the content across. The protocol layer owns the block shape; the MCP client owns the field-name translation; your server code owns the meaning of the error inside the content. If the client mis-translates, the failure flag can be dropped, recreating the silent-failure anti-pattern at the transport boundary rather than in your code.
The third seam is the SDK agent loop. The loop owns matching results to calls and appending them as a user turn in the correct order, but it does not own the error taxonomy. A loop that adds its own retry-with-backoff wrapper is exercising application-level discretion, not protocol behaviour; the protocol neither requires nor forbids it. The exam expects a candidate to name the loop as the owner of ordering and matching, and your code as the owner of category and retryability, never the reverse.
The fourth seam is upstream infrastructure. A 429 with Retry-After is produced by the upstream, not by your tool, but your code owns the decision to surface the wait time inside the error description or context because the model has no internal clock. The infrastructure owns the transient behaviour; your envelope owns making that behaviour actionable to the model.
Version and terminology currency
This task spans terminology from two eras of the certification and two layers of the stack. The candidate must keep them straight.
The credential itself was renamed. The exam guide Version 1.0, effective July 2026, brands the credential "Claude Certified Architect - Foundations" with exam code CCAR-F. The older "CCA-F" short name still circulates in community material and in the reference page's footer, but the authoritative code is CCAR-F, and the domain blueprint (27 / 20 / 20 / 18 / 15) has been constant across guide versions v0.1, v0.2, and v1.0. Task 2.2 sits in Domain 2, Tool Design and MCP Integration, weighted at 18 percent.
The field name itself shifted between the two stacks a candidate touches. The reference page says "the MCP protocol provides the isError flag." That is accurate for the Model Context Protocol: CallToolResult carries a camelCase isError. But the Anthropic Messages API, which is what a candidate actually writes against when calling Claude, uses a snake_case is_error on the tool_result content block. The concept is identical; only the casing differs. A candidate who codes a tool_result block must use is_error, while a candidate who writes an MCP server returns isError that the MCP client then translates into the tool_result content. Both forms appear on the exam, and the reference page's MCP framing should be read as "the flag exists in the protocol; in the Anthropic API it is is_error."
The category vocabulary has also widened over time. The exam guide v1.0 frames four categories: transient, validation, business, permission. Our lesson material, written for current production practice, documents a fuller taxonomy that adds rate-limit, auth, not-found, and permanent. The four-category model remains the exam-facing vocabulary; the wider model is how a real tool suite is built. A candidate answers exam items with the four named categories and understands the wider model as refinement.
The isRetryable semantics align with industry conventions the reference page cites explicitly: gRPC treats INVALID_ARGUMENT as non-retryable, and AWS-style retry metadata does the same. The reference page applies that well-established convention to the validation gap the exam guide leaves open. A candidate who reasons "will resending this exact call work?" lands on false for validation every time, matching both the community convention and the gRPC/AWS precedent.
One currency note on the description field: the reference page uses description for the human-readable recovery hint, while our tool-error-handling lesson splits that role across message (always required, shown to Claude and the user) and optional context / technicalDetail (for debugging, never shown to users). Both are valid application conventions; the reference page's description is the lesson's message plus a recovery suggestion. The principle, not the exact key name, is what the exam tests.
Version and terminology currency: Version drift details that affect this task
Three layers each carry their own version story, and the exam expects a candidate to separate them. The protocol layer is the most stable: the tool_result block, its type, tool_use_id, content, and is_error fields, and the two ordering rules have not changed across the current documentation set, so a candidate coding against is_error today is coding against the same field the exam guide assumes. The MCP layer introduced the isError camelCase naming on CallToolResult as part of the protocol specification, and the divergence between isError and is_error is a naming convention, not a behaviour change; both signal the same failure.
The credential layer changed most recently. The exam guide Version 1.0, effective July 2026, brands the credential "Claude Certified Architect - Foundations" with exam code CCAR-F, while the older "CCA-F" short name still appears in community material and in the reference page footer. The authoritative code is CCAR-F, and the domain blueprint weighting (27 / 20 / 20 / 18 / 15) is documented as constant across the rename. A candidate who writes CCAR-F on the exam and reasons about the four-category model framed by that guide is aligned with the current documentation.
The error-reference layer also drifts. The HTTP error codes documented for the API (400, 401, 402, 403, 404, 413, 429, 500, 504) are the API-level mirror of the tool-level category model, and the 429 rate_limit_error carries the Retry-After convention that the rate-limits documentation describes as current. The agent-sdk guidance on local recovery and selective propagation is current and matches our lesson framing of multi-agent error handling. Where a community source uses an older term or an older credential code, documentation wins, and the candidate answers with the current term while recognizing what the older material meant.
Official versus community divergence
Our gap analysis compares the official exam guide against a large community prep bank. For structured errors the finding is reassuring rather than divergent in substance: the community family is strong, not a blind spot. The probe counts (isError 139, errorCategory or category 129, retryable 176, structured-error 225) show the community bank drills exactly the four-category model with retryable metadata. There is no community material contradicting the official framing.
The one genuine divergence is a gap, not a contradiction. The exam guide v1.0 states retriable: false for business rule violations and never assigns a value to validation. The community convention fills that gap by assigning false to validation, reasoning from "will resending this exact call work." Because the guide is silent, a candidate facing a validation-boolean question should reason from first principles and land on false, exactly as the reference page instructs. Documentation does not contradict the community here; the community simply completes an underspecified row.
A second, more dangerous divergence is an exam trap the reference page calls out directly: some material presents validation as isRetryable: true because the agent can recover from it. That is wrong, and the reference page rebuts it at length. The boolean answers whether resending the same call works, not whether the agent can recover. A malformed order ID fails the same format check every time, so validation is false; the agent recovers by correcting the input and issuing a new call, but errorCategory carries that instruction, not the boolean. A candidate who answers "validation is retryable because the agent can fix it" is choosing the trap.
A third divergence is vocabulary scope. Community and older prep material sometimes collapse auth and permission into one bucket, or omit rate-limit and not-found entirely. Our lessons keep them distinct, and the documentation position is that rate-limit is a transient variant requiring a wait, auth is a credential problem needing escalation, not-found is an error distinct from valid empty. The candidate should answer with the fuller, documentation-aligned taxonomy while recognizing the exam's four-category framing.
Where documentation and community material conflict, documentation wins, and the candidate answers with the documentation position. Here the conflict is mild and the guidance is consistent: use the four categories for exam items, set isRetryable: true only for transient (and rate-limit after wait), and reason validation to false from the resend question.
Official versus community divergence: Additional divergence notes
Beyond the validation-boolean gap, two more divergences deserve a candidate's attention. The first is the industry-precedent framing. The reference page anchors isRetryable semantics in established conventions: gRPC treats INVALID_ARGUMENT as non-retryable, and AWS-style retry metadata marks client errors as not retryable while marking server errors and throttling as retryable. Our lessons reach the same conclusion from the resend question, and a candidate who knows the gRPC or AWS precedent has an independent check on the four-category model. Community material that asserts validation is retryable because the agent can recover is not merely incomplete, it contradicts both the documentation position and the broader industry convention.
The second divergence is vocabulary narrowing. Some community prep collapses auth and permission into one bucket, or omits rate-limit and not-found entirely, presenting only the four named categories. Our lessons keep them distinct, and the documentation position is that rate-limit is a transient variant that must carry a wait time, auth is a credential problem needing escalation, and not-found is an error distinct from a valid empty result. The candidate should answer exam items with the four-category framing but recognize the fuller taxonomy when a scenario clearly describes a rate limit, a missing credential, or a missing resource. Where community material uses the narrower vocabulary, the fuller documentation-aligned taxonomy is the safer answer.
A third, subtler divergence is field naming. Community snippets sometimes write errorCategory while others write category, and some write retryable while the reference page writes isRetryable. These are application-level key names with no protocol enforcement, so the exact identifier is a convention, not a correctness issue. The exam tests the concept (a category plus a retry boolean), not the literal key string, and a candidate should not be thrown by either spelling.
Beyond the task statement
The reference page covers the four categories, the isRetryable semantics, the access-failure versus empty-result split, and multi-agent propagation. Our lesson material covers substantially more that the exam can draw on and that a production architect must know. Each adjacent topic below names its lesson slug and why it matters for Task 2.2.
tool-result-handling (slug tool-result-handling) is the closest sibling. It adds the protocol ordering rules that produce a hard 400 if violated, the empty-result versus error-result table, the partial-result pattern (status: "partial" with both successes and errors), returning multiple tool results in a single user turn, result summarization and truncation to protect reasoning quality, the fromCache flag for cached results, streaming partial results, and PostToolUse hooks that normalize heterogeneous backend shapes before the model sees them. The hook point is directly relevant: a normalization hook must never silently drop error semantics, or it recreates the silent-failure anti-pattern.
tool-error-handling (slug tool-error-handling) extends the category model with exponential backoff and jitter, the tool-level-versus-caller-level retry decision, graceful degradation with per-section ok / degraded / unavailable status, fallback chains that try alternative sources with explicit source labelling, and multi-step rollback when a later step fails after earlier steps succeeded. Its three-layer error model (raw errors in code, structured errors in tool results, natural language in Claude's response) is the conceptual spine of this entire task. The anti-pattern list, generic messages, marking all errors retryable, swallowing exceptions, and retrying without backoff, is exactly what the reference page's exam traps target.
error-handling (slug error-handling) supplies the HTTP error reference that the exam tests as a parallel structure: 400 invalid_request_error (no retry, fix the request), 401 authentication_error (no retry), 402 billing_error (no retry), 403 permission_error (no retry), 404 not_found_error (no retry), 413 request_too_large (no retry, shrink), 429 rate_limit_error (retry with backoff, respect Retry-After), 500 api_error (retry with backoff), 504 timeout_error (retry with backoff). This is the API-level mirror of the tool-level category model, and the exam tests the error-to-action mapping for both. It also restates the silent-failure anti-pattern and the core access-failure versus empty-result distinction with a full table.
validation-pipelines (slug validation-pipelines) connects structured errors to output validation. Its retry-with-fix loop feeds validation errors back to Claude as a new user turn so Claude corrects the output rather than repeating it, and its five-stage pipeline (schema, format, semantic, cross-field, external) is the complementary discipline to tool error handling. The irrecoverable-failure handling (typed failure result, route to human review, track failure rates by category) is the escalation path the reference page's business and permission categories demand.
escalation-patterns (slug escalation-patterns) is where the "escalate to a human" and "escalate to a senior agent" recovery actions from the business and permission categories are specified in full: when to escalate, how to preserve context for the human, and how to honor customer preference. The reference page names escalation as the recovery for business and permission errors; this lesson is the mechanism.
retry-strategies (slug retry-strategies) deepens the backoff and fallback material: circuit breakers, jitter variants, and when to fail fast versus retry. The reference page's transient-retry guidance is the entry point; this lesson is the production-grade expansion.
Beyond the task statement: Further adjacent topics
Three more adjacent areas matter for Task 2.2 even though the reference page omits them. The first is prompt caching and tool schema stability. The Anthropic prompt-caching guidance shows that a stable tool definition, including a stable input_schema, lets the provider reuse a cached prefix across turns and cuts latency and cost. The same principle applies to a stable error envelope shape: when your tool_error-handling envelope uses consistent field names, the model's view of your tools stays cacheable across retries, whereas reshaping errors on every call defeats the cache. A candidate who designs one envelope shape and reuses it is applying caching discipline to error handling.
The second is the PreToolUse and PostToolUse hook pair that tool-result-handling introduces. PreToolUse hooks validate or block a call before execution (parameter checks, policy gates, business rules), and PostToolUse hooks normalize, transform, or enrich a result before the model sees it. The exam contrasts this with prompt-based guidance: prompts are probabilistic, hooks are enforced. For Task 2.2 the load-bearing rule is that a PostToolUse normalization hook must never convert a failed execution into a successful-looking empty result, because that recreates the silent-failure anti-pattern at the hook boundary rather than in the tool.
The third is context-size discipline under failure. A tool that returns a verbose raw exception, or a partial result that dumps every failed record in full, erodes reasoning quality just as a huge success result does. The result-size guidance applies to errors too: truncate or summarize error context, keep technicalDetail server-side, and surface only what the model needs to choose a recovery. Treating error payloads with the same size hygiene as success payloads is part of production readiness.
Worked production examples
The four examples below form one connected implementation: a customer lookup tool whose underlying database can fail in each of the four ways, wrapped by an agent loop that branches on the structured metadata. Each example carries a specific field set, a failure boundary it prevents, and an observable output.
Worked production examples: Example 1: the protocol tool_result block with is_error
This example shows the only layer the API validates. It is a JSON user message containing a tool_result block. Note that is_error is a block-level field, not inside content.
{
"role": "user",
"content": [
{
"type": "tool_result",
"tool_use_id": "toolu_01A09q90qw90lq917835lq9",
"content": "ConnectionError: the customer database is not reachable (HTTP 500)",
"is_error": true
}
]
}What it proves: the protocol failure signal is set. Claude reads is_error: true and treats the call as failed. The tool_use_id matches the assistant's tool_use block, so the conversation stays valid.
Failure boundary it prevents: if is_error were omitted, the block defaults to success and Claude would reason from the error text as if it were an answer. That is the silent-failure anti-pattern at the protocol level.
Observable output: Claude responds with recovery reasoning, for example "The customer lookup failed because the database was unreachable. I will retry shortly or fall back to a cached record."
The incorrect sibling, which the exam treats as a trap, is a success block whose content describes an error:
{
"role": "user",
"content": [
{
"type": "tool_result",
"tool_use_id": "toolu_01A09q90qw90lq917835lq9",
"content": "Error: database unreachable"
}
]
}Here is_error is absent (defaults false). Claude sees a successful-but-confusing result and may report "the customer database is unreachable" as if it were a found value. The distinction between these two blocks is the protocol-level version of the access-failure versus empty-result split.
Worked production examples: Example 2: the structured error envelope
This example adds the application-layer convention inside content. The ToolError interface is the envelope; the function builds it and the caller serializes it into the tool_result block with is_error: true.
interface ToolError {
isError: true
errorCategory: "transient" | "validation" | "business" | "permission" | "rate-limit" | "auth" | "not-found" | "permanent"
isRetryable: boolean
message: string
description?: string
context?: {
resource?: string
input?: unknown
attemptNumber?: number
suggestion?: string
}
}
function buildValidationError(badId: string): ToolError {
return {
isError: true,
errorCategory: "validation",
isRetryable: false,
message: `Invalid order ID format: "${badId}".`,
description: "Order ID must be in format #NNNNN (e.g. #12345). Received 'order-abc'. Reformat the ID and call again.",
context: {
input: { orderId: badId },
suggestion: "Use the canonical #NNNNN format before retrying."
}
}
}The wire format pairs the envelope with the block flag:
{
"type": "tool_result",
"tool_use_id": "toolu_01b2c3d4e5",
"is_error": true,
"content": "{\"isError\":true,\"errorCategory\":\"validation\",\"isRetryable\":false,\"message\":\"Invalid order ID format: \\\"order-abc\\\".\",\"description\":\"Order ID must be in format #NNNNN (e.g. #12345). Received 'order-abc'. Reformat the ID and call again.\"}"
}What it proves: the category, retryability, and recovery hint travel together. isRetryable: false plus errorCategory: "validation" tells the agent it must fix the input, never resend. The description and context.suggestion tell it exactly how.
Failure boundary it prevents: a generic "Operation failed" string leaves the agent unable to distinguish a transient timeout from a policy violation, so it cannot choose a recovery. The envelope removes that ambiguity.
Observable output: Claude reformats the ID and issues a corrected call, or explains to the user why the ID was rejected.
Worked production examples: Example 3: retry with exponential backoff and jitter
This example implements the retry wrapper that owns transient recovery. It is the tool-level retry the lessons recommend for fast infrastructure errors (1 to 3 attempts, sub-second), distinct from asking Claude to retry.
interface RetryConfig {
maxAttempts: number
baseDelayMs: number
maxDelayMs: number
useJitter: boolean
}
const DEFAULT_RETRY: RetryConfig = {
maxAttempts: 4,
baseDelayMs: 200,
maxDelayMs: 10_000,
useJitter: true
}
function getBackoffDelay(attempt: number, config: RetryConfig): number {
const exponential = config.baseDelayMs * Math.pow(2, attempt - 1)
const capped = Math.min(exponential, config.maxDelayMs)
if (!config.useJitter) return capped
return Math.random() * capped
}
async function withRetry<T>(
fn: () => Promise<T>,
shouldRetry: (error: unknown) => boolean,
config: RetryConfig = DEFAULT_RETRY
): Promise<T> {
let lastError: unknown
for (let attempt = 1; attempt <= config.maxAttempts; attempt++) {
try {
return await fn()
} catch (error) {
lastError = error
if (!shouldRetry(error) || attempt === config.maxAttempts) break
await new Promise(r => setTimeout(r, getBackoffDelay(attempt, config)))
}
}
throw lastError
}
async function fetchCustomerWithRetry(customerId: string) {
return withRetry(
() => db.queryCustomer(customerId),
(error) => error instanceof TransientError,
{ maxAttempts: 3, baseDelayMs: 200, maxDelayMs: 2000, useJitter: true }
)
}What it proves: transient errors retry with growing, jittered delays (200ms, 400ms, 800ms, capped at 2s here), avoiding the thundering-herd problem when many parallel calls fail at once. shouldRetry restricts retries to TransientError only; validation, business, and permission errors throw immediately and become structured errors for Claude.
Failure boundary it prevents: immediate retries on rate-limited or overloaded services compound the problem, and retrying non-transient errors wastes quota and delays failure detection. The wrapper encodes the rule "only transient is retryable."
Observable output: a transient blip is absorbed silently and the caller receives the result; a persistent transient after 3 attempts throws and becomes an is_error: true block with errorCategory: "transient", isRetryable: true.
Worked production examples: Example 4: validation and propagation in a multi-agent search
This example combines input validation (Zod) with the multi-agent propagation rule. A search subagent validates its upstream response, returns partial results on mixed success, and propagates only unrecoverable failures to the coordinator.
import { z } from "zod"
const SearchResultSchema = z.object({
items: z.array(z.object({
id: z.string(),
title: z.string(),
score: z.number().min(0).max(100)
})),
totalMatches: z.number().nonnegative(),
hasMore: z.boolean()
})
type SubagentReport =
| { status: "ok"; items: unknown[]; searchedSources: number }
| { status: "partial"; items: unknown[]; searchedSources: number; failedSources: string[] }
| { status: "error"; errorCategory: string; isRetryable: boolean; message: string }
async function searchSubagent(query: string, sources: string[]): Promise<SubagentReport> {
const settled = await Promise.allSettled(sources.map(s => upstreamSearch(s, query)))
const items: unknown[] = []
const failedSources: string[] = []
for (let i = 0; i < settled.length; i++) {
const r = settled[i]
if (r.status === "fulfilled") {
const parsed = SearchResultSchema.safeParse(r.value)
if (parsed.success) items.push(...parsed.data.items)
else failedSources.push(`${sources[i]}: schema-invalid`)
} else {
failedSources.push(`${sources[i]}: ${r.reason?.message ?? "unreachable"}`)
}
}
if (items.length === 0 && failedSources.length === sources.length) {
return { status: "error", errorCategory: "transient", isRetryable: true, message: `All ${sources.length} sources failed.` }
}
if (failedSources.length === 0) return { status: "ok", items, searchedSources: sources.length }
return { status: "partial", items, searchedSources: sources.length - failedSources.length, failedSources }
}What it proves: local recovery happens first (each source retried by upstreamSearch internally), partial results are returned when some sources succeed, and only a total failure propagates as an error. The coordinator receives either ok, partial with the failed-source list, or error with category and retryability. This is exactly the reference page's "I searched 3 of 5 sources successfully" pattern.
Failure boundary it prevents: returning empty as success hides the partial failure and leaves the coordinator blind; terminating the whole workflow on one failed source throws away the 3 good results. The partial status avoids both.
Observable output: the coordinator reports "found 12 matches from 3 of 5 sources; sources 4 and 5 timed out" and can decide whether to retry the two failed sources or proceed with partial data.
Worked production examples: The agent loop that consumes the metadata
The four examples converge in the loop that reads the envelope and branches. This is the practical outcome the reference page's build exercise targets.
async function runAgentTurn(toolCall: ToolCall): Promise<void> {
const result = await executeTool(toolCall)
if (!result.isError) {
if (isEmptyResult(result)) {
await tellUser("No results found for that query.")
} else {
await continueReasoning(result.content)
}
return
}
switch (result.errorCategory) {
case "transient":
case "rate-limit":
await retryWithBackoff(toolCall, result)
break
case "validation":
await fixInputAndRetry(toolCall, result.context?.suggestion)
break
case "business":
case "permission":
await escalateToHuman(result)
break
default:
await escalateToHuman(result)
}
}What it proves: the branch on errorCategory is the decision logic the exam expects. Transient retries with backoff; validation fixes input; business and permission escalate. The isError: false plus empty check at the top is the access-failure versus valid-empty-result guard.
Worked production examples: Example 5: multi-agent propagation with partial results and local absorption
This example completes the multi-agent story from Example 4. A coordinator dispatches two search subagents, each of which performs local recovery for transient blips, then reports either ok, partial, or error. The coordinator merges the partial results, suppresses nothing, and propagates only a genuinely unrecoverable failure upward as an is_error: true tool_result block (because the coordinator is itself exposed to the model as a tool). The idempotency key protects any mutating call a subagent makes during recovery.
type SubagentReport =
| { status: "ok"; items: unknown[]; searchedSources: number }
| { status: "partial"; items: unknown[]; searchedSources: number; failedSources: string[] }
| { status: "error"; errorCategory: string; isRetryable: boolean; message: string; attemptedSources: string[] }
async function runSubagent(label: string, sources: string[], idempotencyKey: string): Promise<SubagentReport> {
const settled = await Promise.allSettled(sources.map(s => upstreamSearch(s, idempotencyKey)))
const items: unknown[] = []
const failedSources: string[] = []
for (let i = 0; i < settled.length; i++) {
const r = settled[i]
if (r.status === "fulfilled") items.push(...r.value.items)
else failedSources.push(`${sources[i]}: ${r.reason?.message ?? "unreachable"}`)
}
if (failedSources.length === 0) return { status: "ok", items, searchedSources: sources.length }
if (items.length > 0) return { status: "partial", items, searchedSources: sources.length - failedSources.length, failedSources }
return { status: "error", errorCategory: "transient", isRetryable: true, message: `Subagent ${label} failed on all sources.`, attemptedSources: sources }
}
async function coordinateSearch(query: string): Promise<object> {
const key = crypto.randomUUID()
const [a, b] = await Promise.all([
runSubagent("alpha", ["s1", "s2", "s3"], key),
runSubagent("beta", ["s4", "s5"], key)
])
if (a.status === "error" && b.status === "error") {
return {
isError: true,
errorCategory: "transient",
isRetryable: true,
message: "Both search subagents failed on every source.",
context: { attemptedSources: [...a.attemptedSources, ...b.attemptedSources] }
}
}
const merged = [
...(a.status !== "error" ? a.items : []),
...(b.status !== "error" ? b.items : [])
]
const failed = [
...(a.status === "partial" ? a.failedSources : []),
...(b.status === "partial" ? b.failedSources : [])
]
return {
status: failed.length === 0 ? "ok" : "partial",
items: merged,
failedSources: failed,
summary: `Merged ${merged.length} results from subagents; ${failed.length} sources still failed.`
}
}What it proves: each subagent owns local recovery (the internal Promise.allSettled plus an upstream retry), so transient blips never reach the coordinator. The coordinator merges partial results from both subagents and only emits an is_error: true block when both subagents are unrecoverable, carrying the attempted-source context so the model can decide whether to retry the whole search or proceed. The shared idempotencyKey means a subagent that retries a mutating call during recovery cannot create a duplicate side effect. This is the reference page's local-recovery-with-selective-propagation rule expressed as running code.
Failure boundary it prevents: if the coordinator returned an empty items: [] on partial failure, the model would see "no results" and might report a false negative; if it threw away the good subagent's results on one subagent's failure, it would waste recovered work. The partial status and the merge avoid both, and the idempotency key avoids duplicate writes during subagent retries.
Observable output: with subagent alpha returning partial (source s3 timed out) and beta returning ok, the coordinator returns status: "partial" with merged items and failedSources: ["s3: ..."], and the model reports "found results from most sources; s3 timed out, retrying it." With both subagents erroring, the coordinator returns is_error: true with errorCategory: "transient" and the model retries the whole search after backoff.
Exam trap catalogue
The reference page lists six exam traps. Each maps to a specific mechanism in this document. A candidate who internalizes the mapping will not choose the trap answer.
Trap 1: retrying when a tool returns an empty result from a successful query. The decoy answers raise the retry limit or instruct the agent never to retry. The correct reading is that the tool does not distinguish access failures from valid empty results, so the agent treats no-matches as a retriable failure. The fix lives in the tool: return is_error: false with count: 0 and a message stating the query executed, versus is_error: true for an unreachable source. The agent then accepts the empty result and reports "no results found" without retry.
Trap 2: using generic error messages like "Operation failed" without structured metadata. The decoy is any answer that treats all failures the same. The correct answer is that without errorCategory, isRetryable, and a description, the agent cannot distinguish a transient timeout from a permanent policy violation and cannot choose recovery. The structured envelope is what enables intelligent recovery; generic strings are the primary anti-pattern in our lessons.
Trap 3: treating business errors as retryable. The decoy says "the agent can retry and maybe it works." The correct answer is that a policy violation (for example a refund exceeding a limit) applies every time the same request is sent, so retry is futile; the agent must take an alternative path such as escalation. This is exactly why isRetryable is false for business.
Trap 4: marking a validation error isRetryable: true because the agent can recover from it. The decoy conflates "the agent can recover" with "the same call will succeed." The correct answer is that isRetryable answers only the narrow question "will resending this exact call work?" A malformed order ID fails the same format check every time, so the boolean is false; the agent recovers by correcting the input and issuing a new call, and errorCategory carries that instruction.
Trap 5: reading isRetryable: false as "abandon the task." The decoy says a false boolean means give up. The correct answer is that three categories are non-retryable but only two are dead ends. validation is false because the input must change first, and the agent fixes it unaided. business and permission are the ones that need an alternative path or a different principal. False means "not this call again," not "stop."
Trap 6: silently suppressing subagent errors by returning empty results as success. The decoy says hide failures to keep the workflow moving. The correct answer is that this hides failure information from the coordinator, preventing intelligent recovery and possibly producing incomplete or inaccurate output. The coordinator cannot distinguish "found nothing" from "could not search." Local recovery should retry transient failures, but unrecoverable failures must propagate with partial results.
The canonical scenario that anchors Traps 1 and 6 is the customer lookup that returns an empty array, the agent retries three times, then escalates, and analysis shows the account simply does not exist. The root cause is the missing distinction between access failure and valid empty result. The observable proof that a tool gets this right is that a successful empty query returns is_error: false with count: 0 and never triggers a retry, while an unreachable database returns is_error: true with errorCategory: "transient" and does trigger retry-then-escalation.
A secondary scenario the exam may use is the API HTTP error analogue. A 400 invalid_request_error (context overflow, malformed JSON) is non-retryable and requires input reduction, not resend. A 429 rate_limit_error is retryable only after Retry-After. A 500 or 504 is retryable with backoff. A 401 is credential escalation. The error-to-action mapping must match the actual code; a generic catch-all retry for every error type is the API-level version of Trap 2 and is wrong.
The recovery decision matrix the exam expects, in full:
| Category | isRetryable | Agent action | Dead end? |
|---|---|---|---|
| transient | true | Resend same call after backoff | no |
| rate-limit | true (after wait) | Wait, then resend | no |
| validation | false | Correct input, send new call | no (agent fixes unaided) |
| business | false | Alternative path or escalate | yes |
| permission | false | Retry as principal with access | yes |
| auth | false | Escalate for credential update | yes |
| not-found | false | Report missing, try alternative | no (sometimes) |
| permanent | false | Report failure, do not retry | yes |
The four reference categories sit in this matrix as transient, validation, business, permission. A candidate should be able to place any failure into this matrix and name the action without hesitation.
Exam trap catalogue: End-to-end walkthrough: the customer lookup
To make the four examples concrete, trace a single customer_lookup request through each of the four categories and the empty-result case. The tool contract is fixed: it accepts an email and an optional failure_mode. The agent loop from Example 4 consumes the result. Each case below shows the wire tool_result block and the agent's observable action.
Case A, valid empty result. The database is reachable and the email matches no record.
{
"type": "tool_result",
"tool_use_id": "toolu_01lkupA",
"content": "{\"items\":[],\"count\":0,\"message\":\"No customer found matching email '[email protected]'. The query executed successfully but returned no matches.\"}"
}Observable agent action: the loop sees is_error absent and count: 0, reports "No customer found for [email protected]," and stops. Zero retries. This is the correct handling of a successful empty query.
Case B, access failure. The database connection times out.
{
"type": "tool_result",
"tool_use_id": "toolu_01lkupB",
"is_error": true,
"content": "{\"isError\":true,\"errorCategory\":\"transient\",\"isRetryable\":true,\"message\":\"Could not reach customer database.\",\"description\":\"Connection to the customer database timed out after 5 seconds. The query did not execute.\"}"
}Observable agent action: the loop sees errorCategory: "transient" and isRetryable: true, waits with backoff, and resends the same call. If the database recovers, the next result is Case A or a hit. If it stays down after 3 attempts, the loop escalates to a human with the partial context "lookup attempted 3 times, all timed out."
Case C, validation error. The email is malformed.
{
"type": "tool_result",
"tool_use_id": "toolu_01lkupC",
"is_error": true,
"content": "{\"isError\":true,\"errorCategory\":\"validation\",\"isRetryable\":false,\"message\":\"Invalid email format: 'john(at)example'.\",\"description\":\"Email must contain a single @ and a domain. Reformat and call again.\",\"context\":{\"input\":{\"email\":\"john(at)example\"},\"suggestion\":\"Use a valid RFC 5322 email address.\"}}"
}Observable agent action: the loop sees isRetryable: false with errorCategory: "validation", reformats the email from the context.suggestion, and issues a new call. It does NOT resend the malformed value. This is the precise behaviour the exam rewards and the trap (marking validation retryable) would break.
Case D, business error. The customer exists but a requested refund exceeds the policy limit.
{
"type": "tool_result",
"tool_use_id": "toolu_01lkupD",
"is_error": true,
"content": "{\"isError\":true,\"errorCategory\":\"business\",\"isRetryable\":false,\"message\":\"Refund amount of 750 exceeds the 500 automatic refund limit.\",\"description\":\"This requires manager approval. Escalate to a human agent with the refund details.\"}"
}Observable agent action: the loop sees errorCategory: "business" and isRetryable: false, never retries, and escalates to a human with the refund details and a customer-friendly explanation. Retrying would hit the same policy wall forever.
Case E, permission error. The service account lacks access to financial records.
{
"type": "tool_result",
"tool_use_id": "toolu_01lkupE",
"is_error": true,
"content": "{\"isError\":true,\"errorCategory\":\"permission\",\"isRetryable\":false,\"message\":\"Access denied.\",\"description\":\"The current service account does not have permission to access financial records. Escalate to a senior agent with financial system access.\"}"
}Observable agent action: the loop sees errorCategory: "permission" and isRetryable: false, escalates to a principal with the right access rather than resending the same call.
The walkthrough proves the central thesis of this task: the protocol gives Claude only is_error; the application envelope gives Claude the category, retryability, and recovery hint; and the agent loop's branch on errorCategory is what turns structured metadata into correct behaviour. Cases A and B are the access-failure versus valid-empty-result split made observable. Cases C, D, and E are the three non-retryable categories with their distinct recoveries, only two of which are dead ends.
Build exercise material
The reference page's build exercise asks for an MCP-style customer_lookup tool with a failure_mode parameter that triggers each error condition on demand, plus an agent loop that branches on metadata. The steps below are verifiable: each step lists the observable outcome that proves it worked.
Step 1: implement the customer_lookup tool with a failure_mode parameter accepting none, transient, validation, business, permission, and empty. The tool reads failure_mode before touching the database.
Observable outcome: calling customer_lookup({ email: "[email protected]", failure_mode: "none" }) returns a success tool_result whose content is a JSON object with the customer record. No is_error flag appears on the block.
Step 2: implement the four error responses. For transient return an is_error: true block whose content carries errorCategory: "transient", isRetryable: true, and a description stating the query did not execute. For validation return errorCategory: "validation", isRetryable: false, with a description giving the expected ID format. For business return errorCategory: "business", isRetryable: false, with a message about a policy limit and an escalation suggestion. For permission return errorCategory: "permission", isRetryable: false, with an escalation message.
Observable outcome: each call returns exactly the three named fields (errorCategory, isRetryable, description) plus is_error: true on the block. Parsing the content yields a JSON object whose errorCategory value matches the failure_mode used. This proves the structured metadata is present and categorised.
Step 3: implement the valid empty result. With failure_mode: "empty", return an is_error: false block whose content is { items: [], count: 0, message: "No customer found matching email '[email protected]'. The query executed successfully but returned no matches." }.
Observable outcome: the block has no is_error flag and the content reports count: 0 with an explicit "executed successfully" message. This is structurally different from any failure_mode that sets is_error: true. The distinction is observable by inspecting the block's flag and the count field.
Step 4: implement the access failure for a real unreachable database (simulate by pointing the tool at a closed port). Return is_error: true, errorCategory: "transient", isRetryable: true, description: "Connection to the customer database timed out after 5 seconds. The query did not execute."
Observable outcome: the block is an error, not an empty success. Comparing this to Step 3 shows the two shapes differ only in is_error and in whether a count is reported, which is the exact lever an agent uses to tell "could not reach" from "reached and found nothing."
Step 5: write the agent loop (as in Example 4 of the worked examples) that parses each result, branches on errorCategory, retries transient up to 3 times with backoff, reformats input for validation, escalates business and permission to a human, and accepts empty results without retry.
Observable outcome: drive the loop with failure_mode: "empty" five times and confirm it never retries and reports "no results found." Drive it with failure_mode: "transient" and confirm it retries up to 3 times with growing delays, then escalates. Drive it with failure_mode: "validation" and confirm it reformats once and stops. This observable behaviour is the proof that the metadata drives correct recovery.
Step 6: add a PostToolUse normalization hook that reshapes backend responses into one schema, with the invariant that it must never turn a failed execution into a successful-looking empty result.
Observable outcome: pass a backend response that returned [] on exception through the hook and confirm the hook emits is_error: true with structured metadata rather than an empty success. This proves the silent-failure anti-pattern is blocked at the hook boundary.
The exercise demonstrates every tested distinction: the four categories, the isRetryable semantics, the access-failure versus valid-empty-result split, and the agent loop's branch logic. A candidate who can build and observe these six steps has covered the full task.
Build exercise material: Extended build steps
Steps 7 through 9 harden the exercise against the failure modes the exam treats as traps. Each adds a verifiable observable outcome.
Step 7: add a rate-limit failure mode that returns is_error: true, errorCategory: "rate-limit", isRetryable: true, and a context.retryAfterMs value taken from a simulated Retry-After header. The tool must include the wait time in the envelope because the model has no internal clock.
Observable outcome: drive the loop with failure_mode: "rate-limit" and confirm the agent surfaces the wait ("the API is rate limited, retrying in N seconds") rather than retrying immediately, and that it does not retry before the indicated window. This proves the wait-time-carries-in-envelope rule and the rate-limit-as-transient-variant distinction.
Step 8: make one of the tool's operations mutating (for example an update_customer_note call) and wrap it with an idempotency key, as in the backoff-and-idempotency example. Trigger a write timeout and confirm that repeated transient retries produce exactly one side effect on the upstream.
Observable outcome: inspect the upstream and confirm a single note write despite three retries from the agent loop. This proves the write-timeout boundary is handled safely and that isRetryable: true on a mutating call does not cause duplication. A test that finds two writes reveals a missing idempotency key.
Step 9: add a two-subagent coordinator, as in Example 5, and drive it so that one subagent returns partial (one source times out) and the other returns ok. Then drive it so both subagents error on every source.
Observable outcome: in the first case confirm the coordinator returns status: "partial" with merged items and a failedSources list, and the agent reports the partial outcome and retries only the failed source. In the second case confirm the coordinator returns is_error: true with errorCategory: "transient" and attempted-source context, and the agent retries the whole search after backoff rather than reporting a false empty result. This proves local recovery with selective propagation and that partial results are never hidden.
The nine-step exercise now spans the full surface: the four categories, the isRetryable semantics, the access-failure versus valid-empty-result split, rate-limit wait handling, write-timeout idempotency, multi-agent partial-result propagation, PostToolUse normalization, and the agent loop's branch logic. A candidate who builds and observes all nine steps has covered Task 2.2 end to end.
Production readiness checklist
A concise verification list an architect can apply to any tool before shipping it. Each item maps to a mechanism in this document. The list is ordered so that the protocol-level guarantees (items 1 and the empty-versus-error split) come first, because a tool that fails those cannot be rescued by better metadata later.
- Every
tool_resultblock setsis_errorcorrectly. Success with empty data isfalse(or omitted); any failure istrue. A block whosecontentdescribes an error but whoseis_erroris absent is a silent failure. - Every error block embeds the envelope:
errorCategory,isRetryable, and amessage. Generic strings without category are rejected in review. isRetryableistrueonly fortransientandrate-limit(after wait). Validation, business, permission, auth, not-found, and permanent arefalse.- Valid empty results carry an explicit empty-state marker (
count: 0, "executed successfully") so they are never confused with access failures. - Access failures set
errorCategory: "transient"(orauth/not-found) withisRetryable: truewhere retry is viable, and adescriptionstating the query did not execute. - Rate-limit errors include the wait time from
Retry-AfterorretryAfterSeconds, because Claude has no internal clock. - Credentials, tokens, stack traces, and internal addresses never appear in
contentorcontext; they are logged server-side only. - Tool-level retries use exponential backoff with jitter and restrict retries to transient errors; they never retry validation, business, or permission.
- Multi-agent subagents perform local recovery for transient failures and propagate only unrecoverable failures, carrying partial results and attempted-source context.
- PostToolUse normalization hooks reshape backend shapes but never convert a failed execution into a successful-looking empty result.
- The agent loop branches on
errorCategory: retry transient with backoff, fix input for validation, escalate business and permission, accept empty results without retry. - Validation failures in output pipelines are fed back to Claude as correction context, not bare-retried, and capped at 2 to 3 attempts before routing to human review.
This checklist is the operational expression of the ownership map: the protocol guarantees the block shape, your code guarantees the metadata and the empty-versus-error distinction, and the agent loop guarantees the recovery branch.
Mechanism and API surface
- MCP error envelope fields
- Return isError boolean, errorCategory enum such as transient, validation, business, permission, isRetryable boolean, message for the model, optional technicalDetail hidden from the user, and context with resource, sanitised input, attemptNumber, retryAfterMs, and suggestion.
- Transient and rate-limit as retryable
- System temporarily unavailable such as timeout, 503, or rate limit. The request is valid. Retry the same call after exponential backoff with jitter. Only this class gets isRetryable true.
- Validation as correction not retry
- Invalid format, missing required field, or out of range enum such as USR dash XXXXX expected. isRetryable false. The response includes expected format and example so the agent can reformulate the input rather than resending the same payload.
- Business and permission as alternative path
- Policy violation such as refund over threshold or access denied due to missing scope. isRetryable false. The agent must take an alternative workflow or escalate as a different principal rather than looping on the same amount or caller.
- Valid empty versus access failure
- Valid empty is isError false with descriptive text and resultCount zero, reached the source and found no match, never retried. Access failure is isError true with category transient and isRetryable true, did not reach the source.
- Executor owned backoff and jitter
- Tool level detection on status greater than or equal to 500 or TIMEOUT or RATE_LIMITED, retry one to three attempts with baseDelayMs scaled exponentially and capped at maxDelayMs with jitter, then surface structured failure to the model rather than model side immediate retry.
The decision rules in play
Each rule below is a distinct decision the exam tests, stated with its mechanism, the boundary where the answer flips, and the wrong answers written against it.
The `isError` flag is the protocol-level failure signal
A tool result that represents a failure must set the boolean isError to true at the protocol level. The flag is the single reliable, programmatic signal the model checks before deciding whether the call succeeded. Without it, any accompanying text is just content; the model may treat the whole result as a normal successful return and continue as if nothing went wrong. The flag lives on the tool result object, not inside a nested status field, not in free text, and not as a side-channel the model must parse.
The canonical shape pairs the flag with a content block carrying a human-readable message, so the model both sees the failure programmatically and reads what happened:
{
"isError": true,
"content": [
{ "type": "text", "text": "Could not reach the customer database within 5 seconds." }
],
"errorCategory": "transient",
"isRetryable": true,
"description": "Connection to the customer database timed out. The query did not execute."
}The model reasons over tool results in the same channel as successful output. If failure is encoded only as an empty array, a null, a success status, or an embedded error string, the model has no consistent, tool-independent cue. The flag standardizes that cue across every tool, so error handling keys off one field instead of inventing per-tool conventions. A status: "success" plus an embedded "message": "No refunds found" trains the model to inspect custom fields that differ per tool, which is the fragility the standard flag removes.
Boundary. The boundary is the difference between "the call ran and returned nothing meaningful" and "the call failed". When the call genuinely executed and found nothing, isError must be false even though the payload is empty. The nearby opposite case that flips the answer: a customer record that does not exist is a successful query with zero matches, so isError: false. A database that timed out before executing is a failure, so isError: true. Conflating them is the single most-tested mistake in this domain.
Recurring specifics. The flag is named isError in the canonical schema. Some evidence uses snake_case is_error; that casing is an artifact of how the question was written and is not the authoritative field name. The flag alone is necessary but not sufficient: an isError: true with no category or retryability still forces blind behavior. The flag signals failure; the metadata beside it signals the response.
Proposal. return an empty array and let the model infer failure from the missing data.
Why it attracts. it keeps the response well-formed.
Why it fails. the model cannot tell "I looked and found nothing" from "I never looked".
Proposal. throw an unhandled exception so the framework propagates it.
Why it attracts. it is the naive default in application code.
Why it fails. the exception bypasses the model's reasoning loop and arrives as a generic framework error with no structured metadata.
Proposal. set isError: false with a warning field.
Why it attracts. the call "completed".
Why it fails. it hides the failure at the protocol level, so any generic consumer treats it as success.
- When the empty shape is reused for both success and failure, the model retries a successful empty query or closes a ticket that should have been escalated. When the flag is present but metadata is absent, retries become blind. When the flag is present with correct metadata, recovery becomes deterministic.
The four canonical error categories drive recovery strategy
Every failure is one of four recovery-oriented categories: transient, validation, business, and permission. The category tells the model what to do next, independent of the literal wording of any message. The mapping is fixed:
type ErrorCategory = "transient" | "validation" | "business" | "permission";
function recoveryFor(category: ErrorCategory, isRetryable: boolean): string {
switch (category) {
case "transient": return "Resend the same call after a delay.";
case "validation": return "Correct the input, then send a new call.";
case "business": return "Take an alternative path or escalate.";
case "permission": return "Retry as a principal with the right access, or escalate.";
}
}The four categories partition failures by what must change for the call to succeed. For transient, nothing about the request changes; the system was briefly unavailable. For validation, the input must change. For business, the request itself conflicts with a policy, so the request must change or be routed elsewhere. For permission, the caller must change. Because each category implies a different next move, a single uniform string forces the model to guess, and guessing produces opposite behaviors for opposite failures.
Boundary. The boundary between business and permission is the subtlest. A business rule (refund exceeds a limit) is a property of the request; a permission error (session lacks authority) is a property of the caller. Both are non-retryable and both need an alternative path, but the nearby opposite case is a policy limit mislabeled as permission: a refund amount over a cap is business, not permission, because the caller is fine and only the amount is wrong. Likewise a restricted journal with no licence is business, not permission, because no credential change fixes it.
Recurring specifics. The category appears as errorCategory (camelCase) in canonical form; evidence also shows error_category. The four values are stable across the tested material. Extended values such as rate_limited, not_found, locked, already_current, permanent, and transient_exhausted appear in fan-out and device scenarios, but each still maps onto one of the four canonical recoveries. rate_limited behaves like transient; not_found behaves like a valid empty or validation depending on framing; locked behaves like permission.
Proposal. use HTTP-style categories such as client, server, network, timeout.
Why it attracts. they feel technical.
Why it fails. they describe where the error came from, not what to do.
Proposal. use severity levels such as fatal, warning, recoverable, informational.
Why it attracts. logging teams think this way.
Why it fails. severity is orthogonal to recovery; a warning can be fatal to a workflow.
Proposal. relabel a permission failure as transient so retries "behave correctly".
Why it attracts. it seems to fix wasted retries.
Why it fails. it legitimates wrong behavior and the retry still fails.
- A malformed identifier is validation in one framing and a missing resource is validation-or-not-found in another. A policy cap is business in refunds and business in warranties. A 403 on a journal is permission in one framing and business (no licence) in another; the mutation that matters is whether a credential change would resolve it.
`isRetryable` answers one narrow question
The boolean isRetryable answers exactly one question: would resending this exact request succeed? It is not a proxy for "should we give up", and it is not a proxy for "is this serious". Only transient earns true. Every other category is false, because something other than the system's mood must change first.
{
"isError": true,
"errorCategory": "validation",
"isRetryable": false,
"description": "Order ID must be in format #NNNNN. Received 'order-abc'. Reformat and call again."
}Resending an unchanged request only works when the request was valid and the failure was the environment. A malformed ID fails the same format check on every resend. A policy violation applies on every resend. A permission denial applies to the same credentials on every resend. Only a timeout or rate limit may clear with time. Therefore isRetryable: true is reserved for transient; false means "not this call again", not "abandon the task".
Boundary. The nearby opposite case is validation. A naive reader sees "the agent can recover from a validation error by fixing the input, so mark it retryable". That is wrong: isRetryable is about resending the same call. The agent recovers by issuing a different call, which errorCategory: validation already instructs. Marking validation true invites pointless resends of the broken input.
Recurring specifics. The flag is boolean. Evidence occasionally shows it as a string or as a retryAfterMs companion; the boolean is canonical. After a subagent exhausts its own retries, it may propagate the same failure to the coordinator with isRetryable: false, because the resend opportunity was already spent at the lower layer. That is consistent, not contradictory: at each boundary the flag means "would a resend here work".
Proposal. set isRetryable: true for permission so access can be granted mid-session.
Why it attracts. sessions sometimes gain rights.
Why it fails. the current call still lacks them; resending identical input with identical credentials fails.
Proposal. make the flag a string for easier parsing.
Why it attracts. typed configs do this.
Why it fails. a boolean is what the decision tree reads.
- A device already on the target build is a no-work success (
isError: false), not afalse-retryable error. A sold-out fare class is business (false), not transient. A rate limit is transient (true). Each mutation checks the candidate resists relabeling.
Valid empty results must be structurally distinct from access failures
This is the most heavily tested distinction. A valid empty result is a successful query that matched nothing; an access failure is a query that could not execute. They require opposite handling, so their wire shapes must be unmistakably different. The valid empty result uses isError: false and often carries an explicit resultCount: 0 or an empty_result marker. The access failure uses isError: true with a category and retryability.
// Valid empty result - NOT an error
{
"isError": false,
"content": [
{ "type": "text", "text": "No customer found matching '[email protected]'. Query executed successfully." }
],
"resultCount": 0
}
// Access failure - IS an error
{
"isError": true,
"errorCategory": "transient",
"isRetryable": true,
"description": "Connection to the customer database timed out after 5 seconds. Query did not execute."
}If both return an empty array, the model cannot separate "the database says no match" from "the database was never reached". The first is the answer; the second needs a retry or escalation. Collapsing them makes the model retry a successful query (wasting turns and confusing users) or close a ticket that should have been escalated (silently dropping a real coverage gap). The structural difference is what lets the model branch correctly without guessing.
Boundary. The boundary is whether the query executed. A zero-match journal response executed and found nothing, so it is valid empty. A connection timeout never executed, so it is an access failure. The nearby opposite case that flips the answer: a genuinely absent field in a document is a valid empty (the field is absent, not an error), whereas a corrupted unreadable document is transient. The model must not treat "field not present" as something to retry.
Recurring specifics. The valid empty shape carries isError: false. The resultCount: 0 field is a recurring explicit signal. Some evidence uses resultType: "empty_result". The access failure carries errorCategory plus isRetryable. When a tool returns the same empty shape for both, the canonical fix is to split the response at the tool boundary where the signal originates; once collapsed at the source, no downstream coordinator or heuristic can recover it.
Proposal. attach errorCategory to every response but keep isError: false even for timeouts.
Why it attracts. the result object stays well-formed.
Why it fails. the flag still says success, so the model proceeds as if the query worked.
Proposal. let the model infer the difference from response timing.
Why it attracts. timeouts feel slow.
Why it fails. timing is unreliable and the model should not measure latency to decide.
Proposal. return raw HTTP status codes as the result.
Why it attracts. engineers trust codes.
Why it fails. it pushes transport interpretation onto the model and still does not set the flag.
- A sold-out fare class (genuinely no inventory) is valid empty, while a timed-out carrier is access failure. A route with zero flights is valid empty, while an unreachable city pair is access failure. A customer with no orders is valid empty, while a 503 on the orders service is access failure. Each mutation checks the candidate splits success from failure at the boundary.
Transient errors warrant retry after a delay
A transient error means the request was valid but the system was briefly unreachable: timeouts, overloads, rate limits, 503s, temporary network partitions. The tool reports errorCategory: "transient" and isRetryable: true. The model (or the tool layer) retries the same call after a growing delay.
{
"isError": true,
"errorCategory": "transient",
"isRetryable": true,
"description": "The order database is experiencing high load. The request is valid and should succeed on retry."
}The defining property of transient failure is that time heals it. The request itself is correct, so resending it later is the right move. Marking it retryable lets the model apply backoff instead of giving up or escalating. The flip side is that retrying a non-transient error is pure waste, so the category must be precise.
Boundary. The boundary is whether the request is intrinsically sound. A database timeout is transient; a malformed query is not. The nearby opposite case: a 403 forbidden looks like a transient access blip but is a permission error that will not clear with time. A 429 looks like a server error but is a rate limit that resolved via Retry-After. Category assignment changes the entire recovery path.
Recurring specifics. Transient is the only category with isRetryable: true. Evidence sometimes extends it to transient_exhausted after local retries are spent, at which point the flag flips to false for the next layer up because the resend opportunity was consumed. Retry mechanics (backoff, jitter, cap) belong to Rules 22 through 24.
Proposal. relabel business or permission as transient to "make retries work".
Why it attracts. stops premature escalation.
Why it fails. the underlying condition never clears; retries burn budget.
Proposal. retry a transient error instantly in a tight loop.
Why it attracts. seems fast.
Why it fails. hammers an already struggling service.
- A rate limit (429) is transient but carries a
Retry-Afterhint, mutating the backoff from generic to server-directed. A timeout that clears on the second attempt mutates the correct answer from "retry locally" to "propagate after exhaustion". A prolonged region outage mutates retry into circuit-break.
Validation errors require corrected input, not a resend
A validation error means the request was malformed: wrong ID format, missing field, out-of-range value, letters in a phone number. The tool reports errorCategory: "validation" and isRetryable: false. The model fixes the input and issues a new call.
{
"isError": true,
"errorCategory": "validation",
"isRetryable": false,
"description": "Phone number contains letters. Expected format +1NNNNNNNNNN. Ask the user for a corrected number."
}Resending the identical call fails the same check forever, so isRetryable is false. But the failure is recoverable by the agent alone, because only the input is wrong. The errorCategory carries that instruction; the description names the offending field and the expected format so the model can self-correct or ask the user.
Boundary. The nearby opposite case is the uncertain one: official guidance is explicit that business errors are false and is silent on validation's boolean, while the wider ecosystem convention treats validation as false (same call fails). When a question turns on validation's boolean, reason from "will resending this exact call work" and you land on false. Flag this as an open point: do not assume validation is true just because the agent can recover.
Recurring specifics. The description should name the failing field and the expected shape. A malformed DOI, a bad order ID, a letter-laden phone, and an invalid query syntax are all validation. The model recovers without escalation.
Proposal. mark validation isRetryable: true because the agent can recover.
Why it attracts. recovery feels like retryability.
Why it fails. it invites resending the broken input.
Proposal. return a successful empty result for a missing field.
Why it attracts. the field is "empty".
Why it fails. absence of a required field is a failure, not a valid find.
- A missing required field in a document is validation (
false); the same document corrupted and unreadable is transient (retryable).
Business errors demand an alternative path and never a retry
A business error means the request was technically valid but violates a policy: refund over a limit, final-sale no-refund, out-of-window claim, missing referral. The tool reports errorCategory: "business" and isRetryable: false. The model takes an alternative path, usually escalation or a customer-facing explanation.
{
"isError": true,
"errorCategory": "business",
"isRetryable": false,
"description": "Refund amount of $750 exceeds the $500 automatic limit. Requires manager approval. Escalate to a human agent."
}The policy applies on every resend, so retry is futile and must be marked false. The model still recovers, just not by resending; it explains the constraint to the user or escalates. The description carries the policy reason so the model can communicate it rather than inventing an availability excuse.
Boundary. The boundary with permission is the caller versus the request. A refund over a cap is business (request is wrong). A session lacking refund authority is permission (caller is wrong). The nearby opposite case that flips the answer: a warranty outside its coverage window is business, but a missing supervisor role that would grant it is permission. The recovery (escalate) is similar; the category tells you whether a credential change could help.
Recurring specifics. Business errors are the canonical isRetryable: false case in official guidance. The description should state the limit and the next step. A recommended_action field that names the exact step is a distractor (see Rule 19); the model should decide from the category.
Proposal. mark business isRetryable: true so the model paces retries.
Why it attracts. paces load.
Why it fails. retries always fail and delay the user.
Proposal. return a generic "Operation failed".
Why it attracts. simple.
Why it fails. gives no metadata to stop retries or explain.
- A refund over a cap is business; retrying with a smaller amount changes the transaction, not the call, so it is still not a resend. A no-refund final sale is business; the model must explain, not retry. A decommissioned SKU is business-or-validation depending on framing; the mutation checks the candidate stops retrying.
Permission errors require escalation or different credentials
A permission error means the caller lacks the rights: 403, insufficient privileges, no licence, expired token. The tool reports errorCategory: "permission" and isRetryable: false. The model escalates or requests elevated credentials; it never retries with the same identity.
{
"isError": true,
"errorCategory": "permission",
"isRetryable": false,
"description": "Insufficient privileges to reset executive passwords. Escalation to Tier 3 required."
}Retrying with the same credentials fails identically, so isRetryable is false. The fix is a different principal, not a better call, so the recovery is escalation. The description must carry an actionable escalation path; a vague "access denied, try later" contradicts the false flag and sends the model into pointless retries.
Boundary. The nearby opposite case is a transient auth blip mislabeled permission. A token that expires mid-session is permission if it cannot be refreshed by the tool, but a momentary 401 that the tool refreshes is transient. The boundary is whether the caller's rights can change without human intervention. A 403 on a journal with no licence is business, not permission, because no credential fixes a missing licence.
Recurring specifics. Permission errors are non-retryable and need an escalation path in the message. Evidence stresses that mislabeling permission as transient or validation causes the model to retry a permanently blocked operation. The description should name who can resolve it.
Proposal. retry permission "once, then escalate".
Why it attracts. cautious.
Why it fails. the first retry already failed; nothing changes.
Proposal. a generic "access denied" with no next step.
Why it attracts. terse.
Why it fails. the model has no path; it may stall or hallucinate.
- A locked sensor (permission-like) is non-retryable and surfaces the reason; an already-current sensor is a valid no-work success (
isError: false).
The description field must carry actionable recovery guidance
The description (sometimes message or userMessage) is the human-readable explanation of what failed and what to do next. It is not a debug dump; it is the text the model relays to the user or uses to choose a path. A good description states the limit, the cause, and the next step.
{
"isError": true,
"errorCategory": "business",
"isRetryable": false,
"userMessage": "A primary care referral is required to book this specialist. Ask the user to obtain a referral first.",
"technicalDetail": "FK_Referral_Missing"
}The category tells the model the class of recovery; the description tells it the specifics. A raw database exception helps only engineers, not the model, so technical detail belongs in a separate field for logs while a userMessage carries the agent-facing guidance. Without actionable text, the model may invent an explanation ("the system is unavailable") that contradicts the real policy.
Boundary. The boundary is agent-facing versus log-only. The nearby opposite case: a longer free-text message that still says "Operation failed because..." helps humans but gives the model no structured metadata, so it remains a distractor. Actionable means the model can act, not just that a human could read it.
Recurring specifics. Recurring pattern: userMessage for the model, technicalDetail for logs. The description names the offending value (the $750 over $500) and the next step (escalate). A vague message that contradicts the boolean (saying "try later" on a false) is explicitly wrong.
Proposal. a longer natural-language message instead of structured fields.
Why it attracts. more words feel more helpful.
Why it fails. no machine-readable signal; the model still guesses.
Proposal. a raw stack trace.
Why it attracts. complete.
Why it fails. jargon confuses the model and helps no recovery.
- A business message that names the limit and escalation path is correct; one that says "try again later" is wrong because it contradicts
isRetryable: false.
Uniform generic error strings strip all signal
When every failure returns the same string such as "Operation failed", the model cannot distinguish transient from validation from business from permission. All four receive identical treatment, so transient outages get abandoned and permanent policy violations get retried to exhaustion. The fix is structured metadata at the tool, not a longer sentence.
// Anti-pattern: one shape for every failure
{ "isError": true, "content": "Operation failed" }
// Fix: typed, actionable metadata
{
"isError": true,
"errorCategory": "transient",
"isRetryable": true,
"description": "Payment gateway temporarily unavailable."
}Recovery depends on category. A uniform string removes the only input the decision tree needs, so the model falls back to a single default behavior. That default is wrong for at least three of the four categories. The cost shows up as wasted retries, premature escalations, and wrong customer messages.
Boundary. The nearby opposite case: a more detailed but still uniform natural-language message. It helps a human reader but still gives the model no structured signal, so it remains a distractor. The dividing line is structured fields (errorCategory, isRetryable), not prose length.
Recurring specifics. The string "Operation failed" is the canonical anti-pattern across support, billing, claims, and extraction scenarios. The minimum required change is adding errorCategory and isRetryable. A numeric error code helps humans but not the model's recovery logic, so it is a weaker fix than structured metadata.
Proposal. a longer free-text message describing what went wrong.
Why it attracts. looks informative.
Why it fails. still requires the model to parse prose and infer category.
Proposal. add a fixed retry count to the system prompt.
Why it attracts. bounds attempts.
Why it fails. applies uniformly, retrying permanent failures and under-retrying transient ones.
Proposal. few-shot examples teaching the model to read the message patterns.
Why it attracts. pattern matching.
Why it fails. uniform messages have no patterns to match.
- A uniform string plus a system-prompt instruction to "decide if temporary" cuts some waste but leaves both behaviors, because the model still cannot tell. A uniform string plus a numeric code helps logs but not recovery.
Silent suppression (empty as success) is the worst anti-pattern
Silent suppression is returning an empty result marked successful when a failure occurred, so the coordinator believes the work completed. The model then produces output that quietly omits a whole data source, subtopic, or coverage channel.
// Anti-pattern: failure disguised as success
{ "results": [], "status": "success" }
// Correct: failure is visible
{
"isError": true,
"errorCategory": "permission",
"isRetryable": false,
"description": "403 on journal X. No academic sources retrieved."
}Suppressing the error destroys the coordinator's ability to recover or even notice the gap. Downstream, a report claims full coverage on a domain never searched, or a net position is computed on substituted zeros, producing confidently wrong output. This is worse than a hard failure because nothing signals that anything went wrong.
Boundary. The nearby opposite case is a valid empty result, which is also empty but is isError: false and genuinely executed. The boundary is whether the query ran. A 403 returning empty is suppression; a zero-match journal returning empty with isError: false is correct.
Recurring specifics. Silent suppression appears as returning [] for a timeout, returning {"status": "success"} for a permission denial, or returning {"records": []} for an unreachable ledger. the tested material calls it the most dangerous anti-pattern. Substituting zero for a missing balance is the headline instance of confidently-wrong output.
Proposal. mark the run successful and omit the failed source so synthesis proceeds.
Why it attracts. uninterrupted pipeline.
Why it fails. hides the gap; the report lies about coverage.
Proposal. a generic "partial failure" status deferring detail to logs.
Why it attracts. signals something.
Why it fails. still omits which source, what category, what partial data.
Proposal. best-effort summary even when inaccessible.
Why it attracts. keeps output flowing.
Why it fails. masks the failure and may misrepresent the source.
- A subagent that returns empty for a 403 journal versus one that returns empty for a zero-match journal: the candidate must split them. A ledger substituted with zero versus one that propagates the error: the candidate must prefer propagation.
Total-workflow termination on a single failure discards good work
Aborting the entire workflow when one subagent fails throws away every successful partial result and bypasses the coordinator's context-aware decision making. The correct pattern sits between silent suppression and total abort: recover locally what you can, propagate only what you cannot, and let the coordinator decide.
function onSubagentFailure(f: Failure): CoordinatorInput {
if (f.resolvedLocally) return f.partialResults; // do not surface
return {
partialResults: f.partialResults,
error: {
isError: true,
errorCategory: f.category,
isRetryable: f.isRetryable,
attempted: f.attempted,
},
};
}A single failed source among several successful ones does not invalidate the good work. Aborting treats a partial failure as total failure. The coordinator can annotate a coverage gap, try an alternative, or proceed with partial results. Termination is reserved for cases where no meaningful output can be produced at all, which is rare.
Boundary. The boundary is whether meaningful output remains possible. If two of three databases returned results, a report with an annotated gap is meaningful, so abort is wrong. If the entire task depends on the one failed call, abort may be right. The nearby opposite case is silent suppression, which is the other extreme; both leave the coordinator blind.
Recurring specifics. Evidence explicitly names two anti-patterns to avoid: silently suppressing errors and terminating entire workflows on single failures. The correct middle path is structured propagation. Throwing an exception to a top-level handler is a variant of termination because it bypasses the coordinator.
Proposal. throw to a top-level handler that aborts the task.
Why it attracts. keeps data clean.
Why it fails. discards partial results and the coordinator's judgment.
Proposal. retry the whole pipeline from scratch on any subagent error.
Why it attracts. clean slate.
Why it fails. wastes completed steps and amplifies latency.
- A single failed subagent among many successful ones mutates the answer from "abort" to "propagate with partials". A failed prerequisite that blocks all downstream mutates it back toward halt.
Local recovery with selective propagation in multi-agent systems
Subagents implement local recovery for transient failures (retry with backoff) and propagate only what they cannot resolve, including category, what was attempted, and any partial results. The coordinator handles genuine escalations, not routine blips.
async function searchWithLocalRetry(query: string): Promise<SubagentResult> {
let attempt = 0;
while (attempt < 3) {
const r = await searchTool(query);
if (!r.isError) return { partialResults: r.hits, error: null };
if (r.errorCategory !== "transient") break; // do not retry non-transient
attempt++;
await backoff(attempt);
}
return {
partialResults: collectedSoFar,
error: { isError: true, errorCategory: r.errorCategory, isRetryable: false, attempted: query },
};
}Propagating every transient blip floods the coordinator with noise it cannot act on better than the subagent can. Recovering locally keeps the coordinator's decision surface focused on genuine escalations. But local recovery must not hide unrecoverable errors; when retries exhaust, the structured error still travels up.
Boundary. The boundary is retryability at the subagent layer. Transient errors are retried locally; validation, business, and permission are propagated immediately because local retry cannot fix them. The opposite cases are a subagent that propagates every error (overwhelms coordinator) versus one that suppresses unrecoverable errors (hides them). The middle path is selective.
Recurring specifics. Evidence repeats the phrase: recover transient locally, propagate only unresolved, with attempts and partial results. After local exhaustion, the propagated flag may be isRetryable: false for the coordinator because the resend was already spent. A PostToolUse hook that normalizes opaque tool errors before the model sees them is an accepted implementation of this at the tool boundary.
Proposal. propagate every error for full visibility.
Why it attracts. transparent.
Why it fails. overwhelms the coordinator with routine transient noise.
Proposal. suppress unrecoverable errors to keep the pipeline clean.
Why it attracts. no disruptions.
Why it fails. produces silently incomplete output.
Proposal. a dedicated error-handling agent watching a failure queue.
Why it attracts. separation of concerns.
Why it fails. over-engineers a responsibility the coordinator already owns.
- A rate limit that resolves on the second attempt mutates the answer to "do not surface at all". A timeout that never clears mutates it to "propagate after exhaustion with partials".
Propagate partial results plus what was attempted
When a subagent cannot resolve a failure, its upward message must include the successes already gathered, the failure category, what was attempted, and any partial data. This lets the coordinator retry, reroute, substitute, or annotate rather than restart.
{
"partialResults": [
{ "source": "academic", "papers": 15 },
{ "source": "industry", "papers": 0 }
],
"error": {
"isError": true,
"errorCategory": "transient",
"isRetryable": true,
"attempted": "patent database query for topic X",
"description": "Connection timeout on patent database."
}
}Partial results are real work; discarding them forces the coordinator to redo successful steps or produce a thinner report. Stating what was attempted lets the coordinator retry the right call or pick an alternative source. Omitting the attempted query or partial results is a recurring distractor that keeps the payload small at the cost of recoverability.
Boundary. The boundary is whether the partial data is usable. If a field extractor failed after the parser succeeded, the parser output is reusable and must be flagged as such. The opposite case is a generic status that omits shipment ID, partial events, and retry-exhaustion state to "keep the payload small". That brevity defeats the purpose.
Recurring specifics. Recurring fields: attempted, partialResults, retriesAttempted, alternativeApproaches. Evidence warns against dropping the shipment ID or partial events to shrink the payload. A test subagent should report how many tests passed before the timeout so the coordinator retries only the rest.
Proposal. omit partial events and attempted query to keep the propagated payload small.
Why it attracts. bandwidth.
Why it fails. coordinator cannot recover or annotate.
Proposal. return only the error, discarding partial results.
Why it attracts. simple.
Why it fails. throws away valid work.
- A subagent that returns two good sources plus one failed mutates the answer to "include both". One that returns only the error mutates it to "include the partials".
The coordinator, not the subagent, decides retry versus disclosure
For unresolved access failures in a fan-out, the subagent propagates the full structured error (failure type, attempted, partial, retry-exhausted) and the coordinator chooses whether to retry, substitute, or disclose the gap. The decision maker is the layer with the whole picture, not the leaf worker.
function coordinatorMerge(reports: SubagentReport[]): Brief {
const gaps: string[] = [];
for (const r of reports) {
if (r.error && r.error.isRetryable) retrySource(r);
else if (r.error) gaps.push(r.error.attempted); // disclose, do not re-dispatch
}
return annotateBrief(mergeSuccesses(reports), gaps);
}A leaf subagent knows only its own call; the coordinator knows the other sources, the user-facing brief, and the business rule about disclosure. Pushing the retry-or-disclose decision down to the subagent either over-retries (re-dispatching a carrier for minutes) or under-discloses (reporting "no events" for a live shipment). Keeping the decision at the coordinator preserves both resilience and honesty.
Boundary. The boundary is information. The subagent decides local transient retry because it has the call; the coordinator decides unresolved retry and disclosure because it has the brief. The nearby opposite case: a subagent that "resolves" an access failure by returning a single cleaned status and swallowing the error object. That removes the coordinator's ability to decide, which is the failure.
Recurring specifics. Evidence in fan-out scenarios consistently selects the option where the subagent surfaces the raw structured result unchanged and the coordinator reads it when merging. The wrong option has the subagent "report a single resolved status without forwarding the error object". Disclosure of a confirmed empty result is a coordinator decision, not a subagent assumption.
Proposal. subagent resolves to a single status and hides the error object.
Why it attracts. clean handoff.
Why it fails. coordinator cannot retry or disclose.
Proposal. coordinator retries every failure identically.
Why it attracts. uniform.
Why it fails. retries permanent failures and ignores empties.
- A subagent that returns the structured error unchanged mutates the answer to "coordinator retries unresolved". One that collapses to a status mutates it to "coordinator re-dispatches blindly".
Classify at the source; do not reconstruct downstream
The error category must be assigned where the failure originates, in the tool, not reconstructed later from HTTP status codes, response timing, or message text at the coordinator or agent. The tool is the layer that knows what happened; everything above should consume, not infer.
// Tool layer: classify once, at the boundary
function toToolResult(e: ToolError): ToolResult {
return {
isError: true,
errorCategory: mapToCategory(e), // transient | validation | business | permission
isRetryable: isRetryable(e),
description: e.message,
};
}Reconstructing category downstream pushes transport interpretation onto the model and produces per-tool conventions the model must learn. A coordinator that parses HTTP codes duplicates logic that belongs in the tool. Classifying at the source means every consumer gets the same signal from the same field.
Boundary. The boundary is ownership of meaning. The tool knows a 503 is transient; the coordinator should not re-derive that. The nearby opposite case: a PostToolUse hook that rewrites an opaque payload into the normalized object before the model sees it. That is still source-side classification (at the boundary), just implemented in a hook rather than inside the tool, and is accepted.
Recurring specifics. Evidence rejects "propagate raw HTTP status codes, let the coordinator parse them" and "infer from latency". A hook-based normalizer is the approved way to fix a legacy tool that emits opaque errors. The canonical four-category map is applied at the tool or hook.
Proposal. return raw status code as the result.
Why it attracts. precise.
Why it fails. pushes parsing onto the model.
Proposal. coordinator infers transient versus persistent from consecutive retry counts.
Why it attracts. observable.
Why it fails. retroactive and unreliable.
- A tool that classifies directly mutates the answer to "consume as-is". A tool that emits opaque text plus a hook that normalizes mutates it to "normalize at boundary". A tool that emits opaque text with no normalization mutates it to "wrong".
Translate raw exceptions and HTTP codes into typed errors
Raw exceptions, stack traces, and bare HTTP codes are for logs, not for the model. The tool translates them into a typed error with errorCategory, isRetryable, and an agent-facing message. Technical detail may ride along in a separate field.
{
"isError": true,
"errorCategory": "business",
"isRetryable": false,
"userMessage": "A primary care referral is required to book this specialist.",
"technicalDetail": "FK_Referral_Missing"
}A raw ConstraintViolationException tells the model nothing actionable and may confuse the user if relayed. Translating it to a business error with a user message gives the model a complete resolution path: explain the rule, ask for the referral, retry after. The model reasons over the typed shape, not over stack frames.
Boundary. The boundary is agent-facing versus log-only. The nearby opposite case: returning the raw exception with isError: true and errorCategory: "transient" (mislabeled). That both confuses the model and invites wrong retries. The correct translation preserves the true category.
Recurring specifics. Recurring pattern: catch exception, map to category, keep technicalDetail for logs. A 403 forbidden is translated to permission; a 422 malformed input to validation; a 503 to transient. Mislabeling a foreign-key constraint as transient is the canonical wrong translation.
Proposal. pass the raw exception with isError true.
Why it attracts. honest.
Why it fails. jargon, no recovery path.
Proposal. return success with a Pending flag in the database.
Why it attracts. user sees success.
Why it fails. corrupts the model's view of state.
- A raw FK constraint translated to business mutates the answer to "explain and ask for referral". The same constraint passed raw mutates it to "confuse the model".
Judgment prompts cannot recover what uniform text hides
A system-prompt instruction telling the model to "use judgment" or to "decide if each failure is temporary" does not fix a tool that returns uniform errors, because there is no signal to judge. The fix is structured data at the tool, not model cleverness.
// Insufficient on its own
systemPrompt: "Decide whether each failure looks temporary before retrying.";
// Required instead: the tool emits category + isRetryableJudgment operates on information. When every failure is "Operation failed", the model has nothing to discriminate on, so its judgment is a coin flip. A prompt line may reduce some waste (by encouraging caution) but leaves both wrong behaviors in place, because the underlying data is still uniform. The decision must be driven by metadata the tool provides.
Boundary. The boundary is data versus instruction. The nearby opposite case: few-shot examples that teach pattern recognition on distinguishable messages. That works only when errors already carry distinguishing text; on uniform text it is inert. So the prompt helps after the data is fixed, not before.
Recurring specifics. Evidence shows a prompt note cutting needless retries from 41% to 29% but leaving both behaviors, proving the prompt is a palliative, not a cure. The canonical fix is structured metadata; the prompt is at best a backstop.
Proposal. add few-shot examples showing how to interpret error patterns.
Why it attracts. teaches recognition.
Why it fails. uniform patterns have nothing to recognize.
Proposal. instruct the model to vary retry behavior by message wording.
Why it attracts. adaptive.
Why it fails. no wording variation exists.
- A uniform tool plus a judgment prompt mutates the answer to "still wrong". A typed tool plus the same prompt mutates it to "acceptable".
A prescribed `recommended_action` field removes agent judgment
Some designs add a recommended_action field that names the exact recovery step and have the model execute it blindly. the tested material rejects this: the model should decide recovery from the category, not obey a prescribed step. Structured metadata informs; it does not command.
// Tempting but rejected as the primary pattern
{
"isError": true,
"errorCategory": "business",
"isRetryable": false,
"recommended_action": "escalate_to_human"
}Recovery often needs context the tool lacks: the user's tone, the business rule about disclosure, alternative workflows. Hard-coding the step in the tool couples recovery to one caller's assumption and removes the model's ability to adapt. The category plus description already let the model choose correctly, so a prescribed action is redundant at best and wrong when the context differs.
Boundary. The boundary is inform versus command. The nearby opposite case: a description that states the next step in prose ("escalate to a human agent"). That is fine, because it is guidance the model weighs, not a field it must execute verbatim. The rejected pattern is the structured recommended_action the model is told to run.
Recurring specifics. Evidence presents recommended_action as a distractor in two scenarios (inventory, observability). The selected option lets the agent decide per failure. The field survives only as description text, not as an enforced instruction.
Proposal. include recommended_action so recovery stays uniform across call sites.
Why it attracts. consistency.
Why it fails. removes model adaptability and misroutes when context differs.
Proposal. execute the tool's prescribed step for every failure.
Why it attracts. deterministic.
Why it fails. ignores caller context.
- A tool with category plus description mutates the answer to "agent decides". A tool with
recommended_actionmutates it to "blind execution, wrong".
A second classification tool doubles latency for no signal
A proposed fix adds a separate analyze_error tool the model calls after every failure to determine the category. This is rejected: it adds a second call per error and still receives the same useless "Operation failed" message, so it has nothing to analyze.
// Anti-pattern: a tool to classify a tool's error
async function recover(call: ToolCall): Promise<Result> {
const r = await call();
if (r.isError) return await analyzeError(r); // analyzeError sees "Operation failed" too
return r;
}Classification needs information about what went wrong, which the original tool failed to capture. A second tool receives the same empty signal and cannot extract detail that was never produced. It also doubles latency on every error path. The correct fix is structured metadata in the original response, which is both more efficient and more informative.
Boundary. The boundary is where signal is created. The nearby opposite case: a PostToolUse hook that rewrites the opaque payload into a normalized object. That is not a second model-facing tool call; it runs at the boundary and gives the model structured data directly. So hooks are fine, second classification tools are not.
Recurring specifics. Evidence presents analyze_error as a distractor with explicit notes: same information vacuum, double latency. The selected fix is structured metadata in the first response.
Proposal. call analyze_error after every failure.
Why it attracts. separation of concerns.
Why it fails. nothing to analyze, double latency.
Proposal. few-shot examples for classification.
Why it attracts. cheap.
Why it fails. operates on uniform text.
- A tool with structured metadata mutates the answer to "no second call needed". A tool plus analyze_error mutates it to "wrong".
Server-side blanket retry harms non-transient failures
Wrapping every tool call in server-side retry with backoff, returning to the model only after exhaustion, hides retryability and actively harms non-transient failures. Validation and permission errors fail identically on every retry, wasting time before the model ever sees them.
// Anti-pattern: retry ALL errors inside the server
async function callWithServerRetry(req: Req): Promise<ToolResult> {
for (let i = 0; i < 3; i++) {
const r = await tool(req);
if (!r.isError) return r;
await backoff(i);
}
return lastFailure; // model sees only the exhausted failure, category-free
}Blanket retry conflates transient (worth retrying) with validation and permission (worth surfacing immediately). For a malformed ID, three backoff retries are pure waste. Worse, by the time the model sees the result, the category may be obscured. The model needs the error to decide; server-side retry that conceals it undermines that.
Boundary. The boundary is retryability of the error type. The nearby opposite case: a tool that retries only transient errors internally and surfaces non-transient immediately. That is correct, because it saves the model only from recoverable blips. The rejected pattern retries regardless of category.
Recurring specifics. Evidence rejects "server handles retries for all errors, returns after exhaustion" and "retry inside the tool for all errors". It accepts "handle transient inside the tool, surface non-transient to the agent". The distinction is category-aware versus blanket.
Proposal. hide all retries inside the tool.
Why it attracts. clean model loop.
Why it fails. wastes retries on permanent errors, hides context.
Proposal. retry non-transient errors server-side to "be safe".
Why it attracts. resilient feel.
Why it fails. always fails.
- A tool that retries only transient mutates the answer to "correct". One that retries all mutates it to "wrong".
Exponential backoff with jitter and a retry cap
Transient retries use exponential backoff (wait grows: 1s, 2s, 4s, 8s) plus random jitter (so many clients do not retry in lockstep) and a maximum attempt cap (so a prolonged outage does not loop forever).
async function retryWithBackoff(fn: () => Promise<T>, max = 5): Promise<T> {
let delay = 1000;
for (let attempt = 0; attempt < max; attempt++) {
try { return await fn(); }
catch (e) {
if (!isTransient(e) || attempt === max - 1) throw e;
await sleep(delay + jitter()); // jitter decorrelates clients
delay *= 2;
}
}
throw new Error("exhausted");
}Exponential backoff gives the service time to recover instead of hammering it. Jitter prevents the thundering herd: if every client retries at exactly 2s, the service is hit again at the worst moment. A cap bounds the effort so a region-wide outage does not burn the retry budget indefinitely. Together they maximize recovery while protecting the service.
Boundary. The boundary is the cap and the jitter. The nearby opposite case: a fixed 1-second delay. That still hits the service at a steady cadence and ignores recovery time; it is rejected. Immediate tight-loop retry is worse, amplifying load. The cap is what separates resilient retry from infinite retry.
Recurring specifics. Evidence repeats "exponential backoff with jitter, capped at a max attempt count" as the standard resilient pattern. Starting at 1s, doubling, with random jitter, up to 3 to 5 attempts, is the recurring concrete recipe. Unbounded retries and fixed delays are the wrong variants.
Proposal. immediate fixed-interval retries with no limit.
Why it attracts. simple.
Why it fails. thundering herd, infinite on outage.
Proposal. retry once after exactly 10 minutes.
Why it attracts. patient.
Why it fails. brittle and slow to recover.
Proposal. never retry.
Why it attracts. safe.
Why it fails. throws away recoverable transient errors.
- A brief blip mutates the answer to "retry at 1s succeeds". A long outage mutates it to "cap bounds the waste".
Honor `Retry-After` for rate-limit responses
A 429 rate-limit response carries a Retry-After header stating the precise wait. The client honors that duration exactly, then falls back to exponential backoff with jitter for subsequent retries if the limit persists.
async function handle429(resp: HttpResponse): Promise<void> {
const retryAfter = resp.headers.get("Retry-After");
if (retryAfter) {
await sleep(parseInt(retryAfter, 10) * 1000); // honor server signal
return;
}
await backoffWithJitter(); // safe default
}The server knows its limit window; respecting Retry-After minimizes wasted retries and stays within policy. Ignoring it (retrying immediately, or applying a mismatched fixed delay) either hammers the limit or waits too long. When the header is absent, generic backoff with jitter is the safe default.
Boundary. The boundary is presence of the header. The nearby opposite case: treating 429 like a generic 500 with the same backoff ignores the explicit timing signal and wastes retries. The correct split is 429 uses Retry-After; 5xx uses backoff.
Recurring specifics. Evidence is explicit: parse Retry-After when present, else backoff with jitter; wait exactly that long, then resume backoff. A 429 is transient; a 400 or 401 is not retryable. Treating 429 and 500 identically is rejected.
Proposal. ignore the header, retry after 1 second.
Why it attracts. simple.
Why it fails. violates the limit, wastes retries.
Proposal. retry immediately to "catch" the limit.
Why it attracts. fast.
Why it fails. intensifies rate limiting.
Proposal. treat 429 and 500 the same.
Why it attracts. uniform.
Why it fails. ignores Retry-After.
- A 429 with a header mutates the answer to "wait that long". A 429 without a header mutates it to "backoff with jitter".
A circuit breaker bounds retries during prolonged outages
During a region-wide outage, every retry fails, consuming time and budget. A circuit breaker stops retrying after a threshold of consecutive failures and periodically probes for recovery, rather than looping until the cap on every request.
class CircuitBreaker {
private failures = 0;
private open = false;
async call(fn: () => Promise<T>): Promise<T> {
if (this.open) {
if (this.probe()) { this.open = false; this.failures = 0; }
else throw new Error("circuit open");
}
try { const r = await fn(); this.failures = 0; return r; }
catch (e) { if (++this.failures > 5) this.open = true; throw e; }
}
}A plain retry cap still wastes the full budget on every request during a known outage. The breaker detects persistent failure, halts retries, and probes cheaply, recovering automatically when the service returns. It prevents wasted retries without removing retry logic for brief blips.
Boundary. The boundary is failure persistence. The nearby opposite case: raising the max retries from 3 to 10 during an outage. That only wastes more time, because the outage exceeds any reasonable window. The breaker is the targeted fix.
Recurring specifics. Evidence introduces the breaker as the improvement for prolonged outages: stop after a threshold, probe periodically. Unlimited retries and faster retries are the wrong variants.
Proposal. increase max retries to 10.
Why it attracts. more chances.
Why it fails. outage exceeds the window; more waste.
Proposal. remove retry logic entirely.
Why it attracts. no waste.
Why it fails. brief blips now fail.
- A brief blip mutates the answer to "normal backoff suffices". A prolonged outage mutates it to "breaker needed".
Structured errors can preserve completed prerequisites
A structured error may carry context beyond the failure, such as steps already completed that need not be repeated on retry. This prevents the model from redoing a prerequisite it already finished.
{
"isError": true,
"errorCategory": "transient",
"isRetryable": true,
"description": "Transfer timed out. AML check already completed this session; retry only the transfer.",
"completedSteps": ["aml_check"]
}On retry, the model may otherwise re-run the prerequisite (the AML check) because it treats the whole task as unfinished. Noting the completed step lets it retry only the failed part, speeding recovery and avoiding redundant side effects. The error object is a carrier for workflow context, not just a fault report.
Boundary. The boundary is whether the prerequisite is genuinely reusable. The nearby opposite case: an error that omits this context, causing the model to redo the AML check. That is inefficient but not wrong; the rule is about optimization, not correctness. The failure type itself still drives retry.
Recurring specifics. Evidence explicitly shows a transfer timeout where noting the completed AML check prevents redundant re-execution. The description or a completedSteps field carries it. This is an enrichment, not a replacement for category and retryability.
Proposal. retry the whole task from scratch on any error.
Why it attracts. simple.
Why it fails. repeats completed work.
Proposal. omit context to keep the payload small.
Why it attracts. brevity.
Why it fails. redundant retries.
- An error noting completed steps mutates the answer to "retry only the failure". One omitting them mutates it to "redo everything". The candidate should preserve context.
Graceful degradation annotates gaps; it never erases them
When a source fails, the system continues with what it has but annotates the output with the coverage gap, rather than re-weighting into a single clean number that hides the missing channel or substituting an assumption for the missing data.
function synthesize(reports: SubagentReport[]): Brief {
const gaps = reports.filter(r => r.error).map(r => r.error.attempted);
return {
tam: sumWeighted(reports.filter(r => !r.error)),
coverage: annotate(gaps), // "estimate scoped to actual coverage"
};
}Erasing the gap (re-weighting so the headline looks complete, or assuming clear weather) produces a confidently wrong output that decision makers trust. Annotating the gap preserves honesty and lets the consumer weight the result appropriately. Graceful degradation means continue where possible, but never silently.
Boundary. The boundary is transparency. The opposite case is a coordinator that re-weights surviving channels so the TAM figure "stays internally consistent" while the dropped channel is invisible. That is fail-open by omission; the correct pattern keeps the gap visible in the annotation.
Recurring specifics. Evidence in market-sizing and weather scenarios selects per-channel coverage annotations that scope the figure to actual coverage. Substituting zero, assuming clear weather, or re-weighting into a clean number are all rejected.
Proposal. re-weight surviving channels into one clean number.
Why it attracts. tidy headline.
Why it fails. erases the gap.
Proposal. use historical averages but do not annotate.
Why it attracts. continuous output.
Why it fails. hides that data is estimated.
Proposal. substitute zero for missing balance.
Why it attracts. computable.
Why it fails. confidently wrong.
- A coordinator that annotates mutates the answer to "correct". One that re-weights mutates it to "wrong".
Drive loops by protocol signals, not by reading model text
Agent loops should continue or stop based on the protocol stop_reason (continue while it is tool_use, stop on end_turn) and should decide retry based on isRetryable, not by scanning the model's reply text for phrases like "inspection complete".
while (turn.reason === "tool_use" && turns < CAP) {
const r = await runTurn();
if (r.isError && r.isRetryable) continue; // retry by flag, not by text
if (r.isError) surface(r); // surface non-retryable
}Phrase matching is fragile: the model may say "survey of this segment is finished" in wording the check misses, so the loop burns turns or stops early. A turn cap alone also mis-stops long tasks. The protocol signal stop_reason is deterministic, and isRetryable is the correct retry gate, not the model's wording.
Boundary. The boundary is deterministic versus heuristic. The nearby opposite case: deciding retry by reading the failure content string and judging whether it "sounds momentary". That pushes classification back onto text, which Rule 18 already rejects. The flag is the gate.
Recurring specifics. Evidence in drone inspection selects the option that drives the loop by stop_reason and retries only when isRetryable is true, extending the standard categories with permanent and not_found. Deciding either by text or by a turn cap is the anti-pattern.
Proposal. keep max_turns as the stop and match more phrases.
Why it attracts. covers wording.
Why it fails. still heuristic and cap-limited.
Proposal. instruct the model to read failure strings and decide.
Why it attracts. adaptive.
Why it fails. text-based classification.
- A loop gated by
stop_reasonmutates the answer to "correct". One gated by text mutates it to "wrong".
Fan-out must avoid both fail-open and fail-closed
In a parallel fan-out, an access failure must be typed and distinguished from a valid empty result, then propagated so the coordinator recovers per source. It must neither fail-open (substitute an all-clear for the missing source, approving blindly) nor fail-closed (abort the whole fan-out on one missing source).
type Outcome = "clear" | "hit" | "timeout" | "bad_request";
function merge(adapters: AdapterReply[]): Decision {
if (adapters.some(a => a.outcome === "hit")) return decline;
if (adapters.every(a => a.outcome === "clear")) return approve;
// some unresolved: route only those to manual review, keep the rest
return routeToManualReview(adapters.filter(a => a.outcome !== "clear"));
}Fail-open (treating a timeout as an all-clear) caused a fraud pass on a transaction a manual review later flagged. Fail-closed (aborting all approvals) blocked the whole pipeline for minutes. The middle path types each outcome and recovers per source: retry transient, surface non-retryable without aborting others, route only genuinely unresolved coverage to review.
Boundary. The boundary is per-source recovery. The nearby opposite case: a coordinator default that fails closed whenever fewer than all replies arrive. That protects correctness but sacrifices availability wholesale; the typed per-source path is better.
Recurring specifics. Evidence in fraud fan-out selects the typed outcome enum with per-source recovery. Fail-open and fail-closed are both explicit wrong answers. A generic status that flattens failure type is also rejected.
Proposal. fail open by substituting all-clear.
Why it attracts. keeps flow.
Why it fails. approves on missing data.
Proposal. fail closed, abort the fan-out.
Why it attracts. safe.
Why it fails. blocks all approvals.
Proposal. flatten failure to a generic status.
Why it attracts. simple.
Why it fails. loses type.
- A typed outcome mutates the answer to "recover per source". A generic status mutates it to "wrong".
`isError: false` separates real failure from valid no-work success
Some outcomes are not failures at all: a device already on the target build, a record that already exists, a task that needs no work. These return isError: false with a content noting the no-work completion, so they are never retried or miscounted as errors.
{
"isError": false,
"content": [{ "type": "text", "text": "Sensor already on target build. Update satisfied, no action taken." }]
}Marking a no-work success as an error category (already_current) still gets it retried or counted as a failure on the dashboard. The dividing line is the flag: isError: false means the operation completed its intent (here, by finding nothing to do). Only genuine failures set the flag. This keeps dashboards honest and loops from re-queuing satisfied work.
Boundary. The boundary is intent completion. The nearby opposite case: a locked sensor, which is a real failure (permission-like, isError: true, not retryable). The mutation that flips the answer is whether the work was already done (no-work success) or blocked (failure). Both may carry a category-like label, but only the failure sets the flag.
Recurring specifics. Evidence in firmware push selects the option that returns already-current as isError: false while transient drops and locked sensors remain isError: true. The flag, not a category name, is the dividing line between success and failure.
Proposal. return already-current as an error category so it is visible.
Why it attracts. visible.
Why it fails. gets retried or miscounted.
Proposal. treat no-work as success with no note.
Why it attracts. simple.
Why it fails. dashboard cannot distinguish satisfied from unrun.
- An already-current sensor mutates the answer to "isError false". A locked sensor mutates it to "isError true, not retryable".
Extended categories are allowed but map to the four canonical
Domain scenarios extend the four canonical categories with values like rate_limited, not_found, locked, already_current, permanent, bad_request, transient_exhausted. Each still maps onto one of the four canonical recoveries; the extension is vocabulary, not a new recovery strategy.
const CANONICAL: Record<string, ErrorCategory> = {
rate_limited: "transient",
transient_exhausted: "transient", // but isRetryable false at this layer
not_found: "validation",
locked: "permission",
permanent: "business",
bad_request: "validation",
};The exam tests the four canonical recoveries (retry / fix input / alternative path / escalate). Extended enums are scenario-specific naming that must still resolve to one of those four behaviors. A rate_limited is retried like transient; a locked is escalated like permission; a not_found is handled like a valid empty or validation depending on framing. The candidate should map the extension, not treat it as novel.
Boundary. The boundary is recovery behavior. The nearby opposite case: an extended value that implies a fifth recovery (e.g., already_current as a non-canonical "satisfied" state). That is fine, but it is a no-work success (isError: false), not a fifth failure category. The four failure recoveries remain the backbone.
Recurring specifics. Recurring extensions: rate_limited (transient), transient_exhausted (transient but isRetryable: false after local retry), not_found (validation or valid empty), locked/already_current (permission-like / no-work), permanent/bad_request (business / validation). The canonical four are the anchor.
Proposal. treat an extended value as a brand-new recovery.
Why it attracts. seems specific.
Why it fails. breaks the decision tree.
Proposal. ignore extensions and force everything into the four.
Why it attracts. uniform.
Why it fails. loses useful signal.
- A
rate_limitedvalue mutates the answer to "transient handling". Alockedvalue mutates it to "permission handling". The candidate maps the extension.
Distinctions that decide answers
| This | Not this | How to tell them apart |
|---|---|---|
| Access failure | Valid empty result | Access failure did not reach the source and is isError true with category transient and retryable true. Valid emptiness reached the source, found no match, and is isError false with resultCount zero. |
| Transient | Validation | Transient is retryable as is after backoff. Validation is retryable only after reformulating the input, so isRetryable is false and the response names the expected format. |
| Validation | Business | Both are not retryable, but validation is recoverable by the agent alone by correcting fields, while business requires an alternative path or escalation because the policy will block every resend. |
| isRetryable true | errorCategory recovery | The boolean answers should I resend this exact call. The category answers what to do when I cannot, including correcting input, seeking approval, or trying a different workflow. |
| Structured envelope | Generic error string | The envelope carries isError, errorCategory, isRetryable, and a suggestion or context the model can act on. The generic string carries no branchable metadata and forces the model to guess. |
Traps
The tempting answer. Retry when the tool returns an empty array for a customer that does not exist because emptiness looks like failure.
Why it fails. Both cases produce empty data yet only access failures are retryable, and a successful query that finds nothing will return the same emptiness on every retry, burning turns before escalating to a human who has no better information.
What is correct. Treat isError false with resultCount zero as success with absence, report not found without retry, and keep retry for isError true with transient.
The tempting answer. Set isRetryable true for a validation error because the agent can correct the input and succeed, so it feels recoverable.
Why it fails. isRetryable answers whether this exact request would succeed on resend, and a string failing a format check will fail the same check every time until the input changes, so the correct value is false with recovery carried in errorCategory and the suggestion that says how to fix the input.
What is correct. Return isRetryable false for validation, include expected format and example, and let the agent reformulate rather than resend as is.
The tempting answer. Retry a refund over the threshold because retry is the default reflex for any tool that returns an error.
Why it fails. The policy that blocks the first call blocks every retry of the same amount with the same principal, so looping wastes turns and never changes the outcome, while the correct action is to take an alternative path or escalate to a human reviewer.
What is correct. Branch business to alternative workflow or escalation and do not loop on the same payload, using the structured reason to explain why to the user.
The tempting answer. Treat isRetryable false as stop entirely because false reads as end of recovery.
Why it fails. False splits into three distinct branches, validation correctable by the agent alone, business and permission requiring a different workflow or principal, and only treating the three separately yields the right branching rather than premature abandonment.
What is correct. Read isRetryable false as cannot retry this call as written, then read errorCategory to decide whether to correct the identifier, restructure the call, or escalate.
The tempting answer. Return an empty success from a subagent when its tool failed, to keep the coordinator input clean.
Why it fails. That hides failure information from the coordinator, prevents intelligent recovery such as retry or partial synthesis, and leads to output that silently omits the failed source with no indication of degraded coverage.
What is correct. Propagate the structured failure upward with partial results and a record of what was tried so the coordinator can distinguish found nothing from could not search.
Tool Distribution and Choice
Keep four to five role scoped tools per agent, consolidate homogeneous variants into one enum tool, and apply tool_choice modes to enforce structure where needed.
What you need to know
How many tools an agent sees on a turn, and how relevant each tool is to that agent's role, determines the reliability of tool selection more directly than any prompt refinement can compensate for. Our curriculum's distribution guidance states the optimal range as four to five tools per agent, scoped to that agent's specific role, with reliability degrading as more options compete for attention. Each additional tool adds description overhead to the context budget, one more option to weigh before invoking anything, and one more chance to pick a plausible sounding tool over the correct one, so quantity and relevance must both be managed together rather than optimised separately.
Role scoping answers relevance. A synthesis agent should not have web search tools, a web search agent should not have document analysis tools, and more generally each agent should see only the tools it needs for its defined responsibilities. Giving an agent tools outside its specialisation invites misuse, such as a synthesis agent running its own searches instead of using results already handed to it, duplicating work and bloating context with unverified claims. The same discipline applies when a synthesis agent needs occasional verification of simple facts. Routing every such fact check back through the coordinator adds two to three round trips per verification and in our material inflates latency by roughly forty percent for workloads where eighty five percent of verifications are simple single site lookups. The tested fix is a scoped cross role tool such as verify_fact given directly to the synthesis agent for the common simple case, while complex verifications that need multiple sources, cross referencing, or judgement continue to route through the coordinator, combining local speed for the frequent path with full pipeline quality where needed.
Consolidation answers overload when the overload itself is homogeneous. A data platform server with twenty two tools where three query tools serve distinct sources but nineteen transformation tools such as pivot, percentile normalisation, and currency conversion all share the shape data in, operation, data out, is not well served by splitting by role, because a transformation agent handed nineteen tools still carries almost the whole problem moved one level down and still cannot choose reliably. For such groups, the correct move is to collapse the nineteen into a single parameterised tool such as transform_data with dataset, transform_type as an enum of the operations, and an options object, reducing twenty two tools to four with no capability lost, because every operation remains reachable as a value the model chooses inside one call rather than a tool it must locate among near identical descriptions. The diagnostic for this distinction is to count the tools before reaching for a remedy. Two similar tools in a set of five is a description problem addressed in Task 2.1, while twenty plus tools where many are variations on one job signals a toolkit shape problem that no description rewrite resolves.
Least privilege is the companion constraint on scope. Replacing a generic fetch_url that can fetch any URL with a constrained load_document that validates document URLs only prevents misuse by capability removal, clarifies the tool's purpose through its narrowed description, and reduces the risk of unintended side effects such as fetching non document resources that should never have been in the agent's repertoire. The interface becomes self documenting because the name itself conveys the allowed scope without extra prose.
A framing that consistently traps candidates is where tool boundaries sit physically. Server boundaries are invisible to the model. A client hands the model every tool from every connected MCP server as one flat list, so a twenty two tool problem split across two servers remains a twenty two tool problem, and consolidation and scoping must be evaluated on the merged prompt not per server. That single fact explains why moving tools between servers is not a distribution fix and why per agent allowlists and consolidation are the levers that actually reduce choice complexity.
Role scoping and the four to five ideal
Scoping by role means each agent's allowlist contains only the tools that implement its responsibilities, with synthesis holding compile_report, format_citation, assess_coverage, and a scoped verify_fact for simple lookups, while web search holds search_web, fetch_page, extract_links, and save_snippet, and document analysis holds extract_metadata, extract_data_points, summarize_content, and verify_claim. Each agent now sees four domain focused tools, and cross role misuse disappears because synthesis no longer carries search tools at all where it might otherwise run its own searches and duplicate coordinator handed results with unverified claims.
The four to five ideal is not a stylistic preference but a measured planning limit. Beyond that per turn choice degrades irrespective of description quality, which is why adding descriptions to an eighteen tool agent is still the incorrect action compared with distributing those tools across role scoped agents. The exam probes this by offering improved wording as a distractor when the toolkit size itself is the cause, and expects distribution rather than narration.
Consolidation for homogeneous overloads
Homogeneous overloads are groups of tools that share an input output shape and differ primarily by operation name, such as nineteen transform tools that all take data in and return data out with operation as the varying part. Splitting by role does not help here because the transformation agent still holds nineteen near identical contracts after the split, leaving selection almost as hard as before. Consolidation replaces the group with one parameterised entry point that exposes transform_type as an enum of the operations and an options envelope for per operation parameters, so the model's decision collapses from which tool among nineteen to which enum value inside one call while preserving every capability.
Counting before acting diagnoses the regime. Two similar tools where the set size is five signals a description problem addressed in Task 2.1. Twenty plus tools where many are variations on one job signals a toolkit shape problem where no description rewrite rescues selection and consolidation is the correct first step. Batch parameter design as a single endpoint that accepts an array of parameter sets for server side fan out is a companion cost reduction that follows the same shape thinking.
tool_choice modes and least privilege
The Messages API tool_choice parameter controls how the model interacts with the available set on this turn. The default auto lets the model decide freely whether to call a tool or return text and is used for general conversation where some turns legitimately need no tool. Any forces the model to call a tool, letting it choose which one, and is used to guarantee structured output against one of several schemas when the document type is unknown, ensuring the model always produces a tool call rather than a conversational response which is valuable for pipelines where unstructured text would otherwise require reparsing. Forced selection with type tool and a name property requires the model to call exactly the named tool and is used to enforce mandatory first steps such as metadata extraction before enrichment, after which subsequent turns typically revert to auto for the remaining steps.
Least privilege sharpens both selection and safety. A constrained load_document that validates document URL shape and host allowlist or extension is preferable to a generic fetch_url that can reach any URL, because the narrowed tool's description is sharply indicative of when to use it and its capability removal prevents misuse by construction. Applying required at the load_document level to URLs while leaving generic fetch capabilities off the allowlist creates a workflow shape that is enforced outside the model's discretion rather than narrated into it.
Why server splits do not reduce the flat list
MCP project level dot mcp dot json carries the shared set and supports ENV_VAR expansion so all tools from every configured server become the single list the model ultimately reasons over. The Agent SDK adds options dot agents and options dot allowedTools with Task or Agent as the delegation meta tool, letting the coordinator assign four to five tools per specialist while keeping the coordinator itself lean without domain specific tools. Moving tools between servers without merging does not change that flat size, so overload persists after the move and only per agent allowlist trimming and enum consolidation actually reduce selection pressure.
That invisibility also interacts with cost heuristics. Where tools are few enough to handle yet two read alike, sharpen descriptions as in Task 2.1. Where tools belong to different jobs such as query, transform, and export, split by role at four to five tools each. Where many tools share a shape such as data in plus operation plus data out, consolidate into one parameterised tool. Producing those three before reaching for a server boundary change is the operational check the exam rewards.
Authoritative mechanism reference
The sections below expand the tested rules into the complete documented surface: ownership, lifecycle, version differences, production boundaries, and adjacent mechanisms.
Mechanism reference: The four tool_choice modes
The tool_choice parameter is a top-level field on the Messages API request. It controls how Claude decides whether to call a tool, which tool to call, and whether it may respond with text. There are four documented values .
auto is the default when tools are provided. Claude decides for itself whether a tool call is necessary. If the request can be answered without a tool, Claude responds with text and stop_reason of "end_turn". If a tool helps, Claude produces a tool_use block with stop_reason of "tool_use". This is the only mode that lets the loop terminate naturally, so it is the right default for general-purpose agents and the only safe mode for an unbounded loop.
any tells Claude it must use one of the provided tools but does not force a particular one. The model cannot respond with text alone; stop_reason is always "tool_use". This is the mode for classification, routing, moderation, and extraction pipelines where every input must produce a structured tool call. The caveat: because any forbids text, Claude will force a tool call even when no tool is appropriate, sometimes emitting nonsensical parameters. Design the tool set so every tool is a valid choice for any input, or pair any with strict mode to guarantee schema conformance.
{ type: "tool", name: "<tool>" } forces Claude to call exactly that named tool. Text-only response is not allowed and no other tool may be chosen. This is the mode for enforcing workflow ordering: a mandatory first step such as extract_metadata before any enrichment, or escalate_to_human when a policy gap is detected. After the forced call completes, subsequent turns typically switch back to auto.
none prevents Claude from using any tool and forces a text-only response. This is the default when no tools are provided, and it can also be set explicitly while tools remain in the request, preserving the tool definitions in context without allowing calls. Use it for greeting or onboarding turns, human-in-the-loop confirmation before a destructive action, context-building reasoning turns, and fallback after a tool error so Claude explains the error instead of retrying .
A critical behavioural note that the reference page does not state: when tool_choice is any or tool, the API prefills the assistant turn to force a tool call, so Claude does not emit explanatory text before the tool_use block, even if the prompt asks for it. Only auto (and separately the extended-thinking feature) preserves visible reasoning ahead of a tool call. Another note: changing tool_choice between requests invalidates cached message blocks; tool definitions and system prompts stay cached.
The interaction with stop_reason is the loop-control backbone. With auto, stop_reason may be "end_turn" or "tool_use"; the loop continues on "tool_use" and terminates on "end_turn". With any, stop_reason is always "tool_use", so a loop with no maximum-turn guard runs forever; the standard fix is a turn cap then a switch to auto. With none, stop_reason is always "end_turn" and the loop exits after one response. With a specific tool, stop_reason is "tool_use" for that tool only; keep forcing the same tool and the loop never terminates.
Mechanism reference: disable_parallel_tool_use
disable_parallel_tool_use is a boolean field that rides alongside tool_choice. By default Claude may emit multiple tool_use blocks in one response when the calls are independent. Setting disable_parallel_tool_use: true constrains that .
The exact semantics differ by mode. With tool_choice: { type: "auto" }, the flag caps Claude at at most one tool call per turn: it may still skip tools and respond with text. With tool_choice: { type: "any" } or a specific tool name, the flag caps Claude at exactly one call per turn . The API itself does not prescribe how you execute multiple tool_use blocks once they arrive; running them concurrently, sequentially, or in a custom order is your application's decision . So disable_parallel_tool_use governs generation, not execution.
This flag matters for Task 2.3 because a scoped, single-purpose agent often wants exactly one tool call per turn: a UI that renders one in-flight call, a billing model that charges per call and must cap it, or a strict workflow where each turn does one thing. It composes with every tool_choice mode and is set per request, so you can enable it only on the turns that need it.
Mechanism reference: Tool count, the tool-overload problem, and the real degradation threshold
The reference page asserts 4-5 tools per agent as the optimal range. Our lessons state the same and add the selection-accuracy table: 1-3 excellent, 4-5 good, 6-10 degrading, 10+ poor . This is a reliability guideline, not a hard API limit.
Anthropic's own tool-search documentation gives a higher, measured threshold: selection accuracy degrades once you exceed 30-50 available tools, and tool search is recommended when you have 10 or more tools, when definitions consume more than 10k tokens, or when you aggregate multiple MCP servers. The two statements are not contradictory but the candidate must hold both: the exam and lessons teach the conservative 4-5 optimum (the number the questions test), while the platform docs describe where hard degradation actually begins. For the exam, answer with 4-5 and role scoping; for a real system at 30-50 tools, tool search is the documented escape hatch.
Mechanism reference: Relevance scoping and cross-role misuse
Beyond raw count, relevance determines correctness. An agent given tools outside its specialisation tends to misuse them: a synthesis agent with web_search might run its own searches instead of using results already handed to it, duplicating work and wasting context. The fix is not "more tools" but "the right tools for this role only" . The exam frames this as a trap: a synthesis agent that drifts into web search is a scoping failure, not a description failure.
Mechanism reference: Scoped cross-role tools
When an agent needs occasional access to a capability that belongs to another role, the naive design routes every such request through the coordinator, adding 2-3 round trips per request and up to 40% latency. The documented pattern is a scoped cross-role tool: a constrained version of the capability, given directly to the agent that needs it .
Concretely, a synthesis agent that must verify simple facts during report generation gets a scoped verify_fact that handles simple single-source lookups; complex verifications (multiple sources, cross-referencing, judgement) still route through the coordinator. The 85% simple case is handled locally; the 15% complex case uses the full pipeline. The description of the scoped tool must state the limitation explicitly and name the escalation path .
This pattern is exactly official sample question Q9, whose community analogues are near-verbatim duplicates, confirming it is a tested distinction.
Mechanism reference: Least-privilege constrained tools
Instead of a generic fetch_url that can fetch anything from anywhere, give a subagent load_document that validates document URLs only. The constrained tool prevents misuse, makes the purpose clearer, and reduces unintended side effects. This is least privilege applied to tool design: each tool does exactly what the agent needs and nothing more. The specific names fetch_url and load_document are reference-page illustrations, not documented Anthropic tool names, so the naming is not independently confirmed while the principle is CONFIRMED.
Mechanism reference: Consolidation with an enum
When tools vary on one job and share a shape (data in, an operation, data out), collapse them into a single parameterised tool. The reference page's data platform example has 22 tools: three query tools, one per data source, and 19 transformations (pivot_table, calculate_percentile, normalise_currency, and so on). Splitting by role hands a transformation agent 19 tools, which is the original problem moved down one level. Consolidating the 19 into transform_data with a transform_type enum of ["pivot", "percentile", "normalise_currency", "..."] turns 22 tools into four. Every transformation stays reachable, now as an enum value the model picks inside one call rather than a tool it must find among nineteen near-identical descriptions.
Anthropic documents the same principle: "consolidate related operations into fewer tools. Rather than creating a separate tool for every action (create_pr, review_pr, merge_pr), group them into a single tool with an action parameter. Fewer, more capable tools reduce selection ambiguity" . Important: consolidation reduces how many tools an agent chooses between; it does not hand the agent new powers, so it does not undo least privilege. That distinction is an exam favourite .
Mechanism reference: Server boundaries are invisible to the model
The reference page claims moving tools onto a second MCP server is "not a fix" because server boundaries are invisible to the model: a client hands every tool from every connected server to the model as one flat list, so a 22-tool problem split across two servers is still a 22-tool problem. The underlying mechanism is CONFIRMED: client tools from every source land in the single tools array of one request. The literal "invisible to the model" wording is not a verbatim Anthropic sentence, so the phrasing is marked not independently confirmed; the design conclusion (MCP server splitting does not reduce model-side tool count) is sound and CONFIRMED by the flat aggregation property.
Mechanism reference: input_examples
A tool definition accepts an optional input_examples array of example input objects. Each example must be valid according to the tool's input_schema; invalid examples return a 400 error. Examples are included in the prompt alongside the schema, showing Claude concrete patterns for well-formed calls: when to include optional parameters, what formats to use, how to structure complex inputs. They cost tokens (roughly 20-50 for simple examples, 100-200 for complex nested objects).
input_examples is available on user-defined and Anthropic-schema client tools, but not on server tools (web search, code execution) and not on the computer use or browser use toolsets. It composes with tool search: when Claude discovers a deferred tool, the API expands its input_examples along with its definition. This is a beyond-the-task feature worth knowing: for complex tools, descriptions remain most important, but input_examples closes the gap on format-sensitive inputs.
Mechanism reference: Deferred loading and the tool search tool
defer_loading is an optional property on any tool definition. It excludes the tool from the initial system prompt and loads it on demand when tool search returns a tool_reference for it. You still send every tool's full definition in the tools array on every request, including the deferred ones; the API needs them server-side to run the search and expand references. Deferred tools load only when discovered; the tool search tool itself must never be deferred.
The tool search tool (tool_search_tool_regex_20251119 and tool_search_tool_bm25_20251119) is a server-side tool that lets Claude work with hundreds or thousands of tools by discovering and loading them on demand. You include a tool search tool in tools, set defer_loading: true on the tools that should not load up front, keep at least one tool (normally the search tool) non-deferred, and Claude searches by name, description, argument names, and argument descriptions, then receives tool_reference blocks the API expands into full definitions. The regex variant takes Python re.search patterns (max 200 characters); the BM25 variant takes natural language queries (max 500 characters). Each search returns up to 5 tools by default; Claude can set limit from 1 to 10,000. The maximum deferred tools per request is 10,000.
defer_loading preserves prompt caching: deferred tools are stripped from the rendered tools section before the cache key is computed, so adding deferred tools does not invalidate an existing cache entry. A deferred tool cannot also carry cache_control; put the breakpoint on a non-deferred tool. This is the advanced answer to tool overload at scale, well beyond the 4-5 lesson guidance but directly relevant when a single agent must see a large catalog.
Mechanism reference: Role-scoped toolsets and caller scoping
"Role-scoped toolsets" is the pattern of assigning a coherent group of tools (toolset) to a specific agent role and restricting who may invoke each tool. In the Agent SDK, subagents are spawned with their own focused tool sets, and tool access is controlled by tools, allowedTools, and disallowedTools. A bare tool name or an allowedTools entry changes availability (whether the tool appears in Claude's context); a scoped disallowedTools rule changes permission (which calls are approved). Omitting a built-in from tools, or listing its bare name in disallowedTools, keeps it out of context so Claude never attempts it.
At the Messages API level, the allowed_callers field restricts which callers can invoke a tool. It accepts "direct" (the model calls the tool in a tool_use block; the default) and "code_execution_20260120" (code running inside a code-execution sandbox can call the tool). Omitting "direct" guides Claude to call the tool only from within code execution; the response tool_use block then includes a caller field identifying the caller. The toolset_name field on tool_use and tool_result blocks identifies the toolset family a member belongs to, and client toolsets (computer use, browser use) are single tools entries with no name that declare a fixed set of member tools. Anthropic's own multiagent cookbook builds a coordinator that runs specialists (web-search researcher, file-reading librarian, rules-based pricer) with scoped toolsets and per-role tool scoping. So role-scoped toolsets are documented both as an SDK/subagent pattern and as a tool-definition caller-scope pattern; the exact "role-scoped toolset" label is the cookbook's phrasing, so the label is cited as a cookbook term while the mechanism is CONFIRMED.
Mechanism reference: strict mode and cache_control as composing properties
Two more tool-definition properties matter for production distribution. strict: true guarantees schema validation on tool names and inputs; it composes with tool_choice: { type: "any" } to guarantee both that a tool is called and that inputs follow the schema. cache_control sets a prompt-cache breakpoint at a tool definition. Both compose with defer_loading. Strict mode is unavailable on mcp_toolset, computer use, and browser use toolsets, and input_examples is unavailable on server tools and those toolsets.
Mechanism reference: Deepening the four modes: per-mode interactions with disable_parallel_tool_use, deferred loading, per-tool configuration, and prompt cache
The four tool_choice modes do not operate in isolation. Each composes with the parallel-disable flag, with deferred loading and the tool search tool, with the per-tool configuration fields, and with prompt caching in a distinct way. The reference page states only the four names; the production behaviour lives in the combinations, and the exam rewards candidates who see the interaction rather than the single flag.
auto interactions. With tool_choice: { type: "auto" }, setting disable_parallel_tool_use: true caps the model at at most one tool_use block per turn while still permitting a text-only end_turn when no tool is warranted. Deferred loading interacts cleanly with auto because auto already allows a tool-free turn: on a conversational turn, a request whose only non-deferred member is the tool search tool simply returns text, and on a turn that needs a tool the model emits a tool_search server-tool call, the API expands tool_reference blocks into full definitions, and the now-present tool becomes callable in the same episode. Per-tool configuration under auto is where the independent levers sit on each definition: cache_control places a prompt-cache breakpoint at that tool, input_examples shows concrete argument shapes only for that tool, strict: true validates that tool's inputs, and allowed_callers can restrict that tool to code execution rather than direct model calls. The prompt-cache implication is the one candidates most often miss: the tools array is part of the cached prefix, so adding, removing, renaming, or reordering any tool changes the prefix hash and invalidates the cached system-plus-tools block unless a cache_control breakpoint sits after a stable early portion A stable tool list across turns is what makes auto cheap at scale; mutating the list to add a tool mid-conversation is what silently doubles input-token cost.
any interactions. With tool_choice: { type: "any" }, disable_parallel_tool_use: true caps the model at exactly one forced call per turn, because any already forbids text . The deferred-loading interaction is subtle: any forces a call among the tools already present in context, and deferred tools are not present until tool search discovers and expands them, so pairing any with a fully deferred catalog leaves no callable tool in context unless a prior search turn has already run not independently confirmed. In practice any is paired with a small, fully-presented tool set, and tool search is the escape hatch used with auto rather than with any. Per-tool configuration under any reaches its strongest guarantee when combined with strict: true: any forces a call and strict forces schema conformance, so the pair is the standard way to guarantee both that structured extraction happens and that its arguments are valid . The prompt-cache rule for any matches auto on the tools side but differs on the messages side: changing tool_choice between requests (for example any on one turn, auto on the next) invalidates the cached message blocks while tool definitions and the system prompt stay cached . The lesson is to keep tool_choice constant within a batch of turns where you want message-cache hits and to switch it only at a deliberate boundary such as the move from a classification turn to a synthesis turn.
none interactions. Under tool_choice: { type: "none" }, disable_parallel_tool_use is inert because no tool call is possible; the flag's presence is harmless but meaningless . Deferred loading under none keeps all tools (including deferred ones) out of the callable set while preserving their definitions in the request, so the next turn can switch to auto or any without resending the catalog . This is a genuine prompt-cache advantage: a none greeting turn does not alter the tools array, so the cached system-plus-tools prefix survives the turn, and when the following turn switches to auto the tools-prefix cache still hits Per-tool configuration under none still applies to caching, because a cache_control breakpoint on a tool in the ignored list is computed regardless, bounding the cached prefix exactly as on a callable turn. The distribution lesson is that none is not merely a suppression mode but a cache-preserving mode: it holds a large tool set resident in cache across conversational turns that need no tools.
Specific-tool interactions. Under { type: "tool", name: "<tool>" }, disable_parallel_tool_use: true is redundant because forced selection already permits exactly one call, but pairing them documents intent and guards against a later code change that swaps the mode. The deferred-loading interaction is a hard rule: the forced tool must be present in context to be called, and a deferred tool is not present until discovered, so you must never set defer_loading: true on a tool you intend to force, or the request fails because the named tool is absent from the presented set. Per-tool configuration under forced selection again pairs with strict: true, guaranteeing both that the call occurs and that it validates. The prompt-cache rule matches the others on the tools side, and the message-side caveat applies: forcing a specific tool on one turn and auto on the next invalidates cached message blocks, so keep forced-selection turns grouped when message-cache economy matters.
Prompt-cache implications of changing tool lists, stated directly. Several operations change the cached prefix. Adding a tool, removing a tool, renaming a tool, reordering tools, or editing any tool's description or input_schema all alter the hash of the system-plus-tools prefix and force a cache miss from the break to the end of the prefix Two operations preserve it: leaving the tools array byte-for-byte stable across turns, and placing a cache_control breakpoint on a non-deferred tool so that later, frequently edited tools sit after the breakpoint and their churn does not invalidate the earlier, stable portion. A deferred tool cannot itself carry cache_control, so the breakpoint must sit on a non-deferred tool while the deferred long tail stays out of the cache key entirely The distribution takeaway: treat the tool list as a cache boundary. Bind a stable core set of 4-5 role tools, mark them non-deferred with a breakpoint, and let volatile or numerous tools be deferred so their changes never disturb the core cache. This is how a role-scoped agent stays both accurate and cheap as the catalog grows.
Why the exam probes these combinations. A scenario question rarely asks what disable_parallel_tool_use does in isolation; it asks what happens to a billing agent that must cap calls, to a catalog of thousands of tools where context would blow up, or to a forced-step pipeline that must preserve cache across turns. Each combination above maps to one of those scenarios, and the correct answer is the interaction, not the single flag. A candidate who knows only the four mode names can describe a tool but cannot predict system behaviour when the flag, the cache, and the deferral meet; that gap is exactly what this grounding must close.
Ownership map
Which layer owns each guarantee:
- Model (Claude). Owns tool selection given the presented list: which tool to call, whether to call, and what arguments. Owns the decision under
auto; owns choice-among-list underany; owns argument filling under forced selection. The model does not own the tool list itself. - Application code (your agent loop). Owns assembling the
toolsarray per role, settingtool_choiceanddisable_parallel_tool_useper turn, executingtool_useblocks, correlating results bytool_use_id, appendingtool_resultblocks, and enforcing the maximum-turn guard that stopsanyfrom looping forever. Owns execution strategy (parallel viaPromise.allversus sequential). - SDK (Anthropic client or Agent SDK). Owns the request envelope, streaming,
stop_reasonparsing, and (in the Agent SDK) the agent loop and subagent spawning with scoped toolsets. The Agent SDK's in-process MCP server catches handler exceptions and returns them as error results. - CLI / harness (Claude Code). Owns permission prompts and
allowedTools/disallowedToolsevaluation that gate which tool calls run automatically versus need approval. - Infrastructure (Anthropic servers). Owns server-tool execution (web search, web fetch, code execution, tool search) and the deferred-loading expansion of
tool_referenceblocks. You never execute server tools or expand references yourself. - Configuration / tenancy. Owns dynamic tool binding: assembling the
toolsarray at request time from role, caller, or tenant, and tool-permission scoping (read-only, standard, admin).
The key ownership insight for Task 2.3: tool distribution is an application and configuration concern, not a model concern. The model only reacts to the list you hand it. Overload, cross-role misuse, and missing forced steps are all failures of the layer that assembles the list and sets tool_choice, never of the model.
The ownership boundaries can be summarised as a table, where each guarantee is traced to the layer that owns it:
| Guarantee | Owning layer | Notes |
|---|---|---|
| Which tool to call, whether to call, what arguments | Model | Reacts to the presented tools list only |
Assembling the tools array per role | Application code or configuration | Dynamic binding by role, caller, tenant |
Setting tool_choice and disable_parallel_tool_use per turn | Application code | Drives loop control and single-call capping |
Executing tool_use and correlating by tool_use_id | Application code | Parallel via Promise.all, sequential otherwise |
Maximum-turn guard that stops any looping | Application code | Required whenever any or a forced tool is used |
| Agent loop and subagent spawning with scoped toolsets | SDK (Agent SDK) | Owns the orchestration harness |
Permission gating (allowedTools/disallowedTools) | CLI / harness | Decides automatic versus approved calls |
Server-tool execution and tool_reference expansion | Infrastructure | Web search, code execution, tool search; never run by you |
defer_loading expansion and prompt-cache preservation | Infrastructure | Deferred tools expanded server-side |
allowed_callers enforcement | API / model | Restricts which caller may invoke a tool |
This table is the answer to "who owns what" on the exam: when a question asks why an agent misused a tool or skipped a step, the cause is almost always the application or configuration layer, not the model.
Tool description craft and resolution order
Task 2.3 sits next to Task 2.1 (tool descriptions), and the two are inseparable in practice. The reference page's decision matrix says that when two tools read alike in a small set, the fix is sharpened descriptions, not consolidation or splitting. The lesson on tool binding gives the resolution order Claude uses when tools compete: description clarity first, then parameter specificity (enums and required fields), then name uniqueness, then array position (tools earlier in the array have a slight selection bias).
Description craft rules that reduce cross-role and same-role misrouting: write at least three or four sentences explaining what the tool does, when to use it, when not to, what each parameter means, and what the tool does not return; use meaningful namespacing so one search matches a whole group (for example github_, slack_); and add keywords that match how users describe tasks so tool search discovers the tool. Anthropic's own guidance is explicit that detailed descriptions are the single largest factor in tool performance, and that for tools with complex or format-sensitive inputs you should also provide input_examples. The resolution-order list explains a subtle exam point: in a set of five tools, get_customer versus lookup_order is a description problem, but in a set of 22 it is not, because array-position bias and description clarity cannot rescue selection once decision complexity saturates. Count the tools before picking the remedy.
Dynamic tool binding and tenant or role scoping
Tool distribution need not be static. A support agent may expose different tools depending on whether the caller is an admin, a team member, or a billing contact, and a multi-tenant system may load a per-tenant tool manifest from storage at request time. The lesson on tool binding shows assembling the tools array from a callerRole or a tenantId, merging base tools with role tools, and applying a permission scope (read-only, standard, admin). This is the same role-scoping idea as Example B but expressed at request time rather than at subagent spawn time.
// Assemble the tool list from the caller's role at request time.
function getToolsForRole(role: "admin" | "member" | "billing"): Anthropic.Tool[] {
const readTools = [searchOrdersTool, getProductTool, getAccountTool];
const writeTools = [createOrderTool, updateProfileTool, cancelOrderTool];
if (role === "admin") return [...readTools, ...writeTools, ...adminTools];
if (role === "billing") return [...readTools, ...writeTools];
return readTools; // members see read-only tools only
}
// A tenant manifest loaded from SQL at request time.
async function getTenantTools(tenantId: string): Promise<Anthropic.Tool[]> {
const config = await sql<{ tools_json: string }>(
`SELECT tools_json FROM tenant_config WHERE tenant_id = ${tenantId}`,
);
return config.length ? JSON.parse(config[0].tools_json) : defaultTools;
}What this proves: role and tenant scoping are application-layer assembly of the tools array, reinforcing that distribution is a configuration concern. Failure boundary: merging must not accidentally grant a lower-privilege role a higher privilege tool; the permission scope must be the source of truth, and the tools array must be rebuilt per request so a role change takes effect immediately.
Version and terminology currency
- Exam guide currency. The official CCAR-F Exam Guide is v1.0, effective July
- Domain 2 weights 18%; Task 2.3 sits among five Domain 2 task statements (2.1 tool descriptions, 2.2 structured errors, 2.3 tool distribution, 2.4 MCP integration, 2.5 built-in tools). Domain weights and the 60-item, 120-minute, 720/1000 format have been stable across v0.1, v0.2, and v1.0.
tool_choiceterminology. The four-mode vocabulary (auto,any,none, specific name) is stable and matches the current API. No rename has occurred between the exam guide and the current product.disable_parallel_tool_use. Present and stable in the current API; the lesson and API agree on the at-most-one versus exactly-one semantics.input_examples,defer_loading,allowed_callers,strict. These are current API tool-definition properties.defer_loadingand the tool search tool are comparatively recent (tool search type strings carry the20251119date version), so they postdate the earliest exam-guide authorship and are advanced platform features rather than exam-task content.- 4-5 versus 30-50. The exam and lessons use 4-5; the current tool-search docs use 30-50 as the degradation point. Both are current, describing different points on the same curve. Answer the exam with 4-5
noneomission. The reference task page lists three modes and omitsnone. This is a documentation gap in the reference page, not a product change;noneis current and exam-tested.
Official versus community divergence
- Tool count ceiling. Community and lesson material often state a hard "5 tool limit." Official Anthropic docs state no hard limit; 4-5 is an optimum and degradation is measured at 30-50 tools, with tool search handling 10+. The candidate should answer exam questions with the 4-5 optimum (that is what the exam tests) and understand the higher real-world threshold. Documentation wins on the "no hard limit" point.
nonemode. Some community prep omitsnonebecause the reference task page omits it. Official docs and our deep-dive lesson includenoneas a distinct fourth mode. Answer with four modes; documentation and our lesson win- Forced-selection semantics. Community sometimes claims forced
toolchoice still allows text. Official docs are explicit:anyandtoolprefill the assistant turn so no explanatory text precedes thetool_useblock. Documentation wins. - MCP server split as a fix. Community occasionally suggests splitting tools across MCP servers to reduce overload. The reference page and the flat-aggregation property show this does not reduce model-side tool count. The reference page and aggregation property win
- Consolidation versus constraint. Community can blur these. The reference page is explicit and the exam tests the tension: consolidation reduces choice count (does not grant new power, preserves least privilege); constraint reduces what a tool can reach (may remove power). Both can coexist in one system
- Latency figure (40%). The 40% latency and 85% simple-case figures come from the official exam-guide sample (Q9), not from community invention. Treat them as official scenario framing.
Decision guidance: which fix applies
The reference page's fix matrix is the spine of Task 2.3, and the exam rewards applying it correctly. The matrix, annotated with the mechanism each row triggers:
| The tools are... | The fix | Mechanism triggered | Exam signal |
|---|---|---|---|
| Few enough to handle, but two read alike | Sharpen the descriptions (Task 2.1) | Improves resolution-order rank 1, fixes misrouting without changing count | Small set, symptom is wrong tool chosen |
| Different jobs (query, transform, export) | Split by role, 4-5 each | Reduces per-agent decision complexity; each agent sees only its job | Distinct purposes, not variations |
| Variations on one job, sharing a shape | Consolidate into one parameterised tool | Shrinks choice count via enum; no new power granted | Same data-in, operation, data-out shape |
| Doing more than the agent should be able to do | Constrain them (least privilege) | Reduces what a tool can reach; may remove power | Generic tool enables misuse |
The first row is the one candidates trip on, and the exam tests the distinction. Task 2.1 teaches descriptions as the fix for misrouting, and it is right when the toolkit is small enough to reason about. An agent choosing get_customer over lookup_order from a set of five is a description problem. The same symptom from a set of 22 is not: the agent is past the point where description quality rescues selection, and rewriting all 22 leaves decision complexity exactly where it was. Same symptom, different disease. Count the tools before picking the remedy.
The third and fourth rows pull in opposite directions, and the exam likes that tension. Consolidation reduces how many tools an agent chooses between; it does not hand the agent new powers, so it does not undo least privilege. Replacing fetch_url with load_document does the opposite job and both can be right in the same system: consolidate the 19 transformations, constrain the generic fetcher. Neither fix is "the" fix; the right one depends on whether the problem is choice count or reach.
A second decision guide, for tool_choice, pairs with the first:
| Need | Mode | Why |
|---|---|---|
| General assistant, may skip tools | auto | Only mode that terminates the loop naturally |
| Every input must be classified or routed | any | Guarantees a tool call, but cap turns |
| Mandatory first or specific step | { type: "tool", name } | Enforces ordering |
| Greeting, confirmation, error explanation | none | Text only, no tool call |
| Load only relevant tools from a huge catalog | tool search + defer_loading | Keeps context small, accuracy high |
Note that none belongs in this table even though the reference page omits it; the exam includes it.
Beyond the task statement
Material our lesson library covers that the reference page omits, each with its slug and why it matters for 2.3.
parallel-tool-calling(lesson:parallel-tool-calling). Independent versus dependent calls,disable_parallel_tool_usesemantics, and result correlation bytool_use_id. Matters because scoped agents that call one tool per turn rely on the disable flag, and any multi-tool turn must correlate results by id, not position. Direct relevance to 2.3 ownership and the disable flag.tool-choice-deep-dive(lesson:tool-choice-deep-dive). The fourth modenone, the fullstop_reasoninteraction table, the infinite-loop antipattern withany, and the token or latency comparison matrix. Matters because the reference page omitsnoneand the loop-control reasoning; the exam tests all four modes.tool-binding(lesson:tool-binding). Tool categories (built-in, MCP, custom), the 4-5 table, dynamic tool binding per role or tenant, tool resolution order (description clarity, parameter specificity, name uniqueness, array position), and permission scoping (read-only, standard, admin). Matters because dynamic binding and permission scoping are the implementation surface for role-scoped distribution; the reference page assumes static binding.tool-orchestration(lesson:tool-orchestration). Distributing tools across subagents, sequential versus parallel orchestration, and the anti-patterns (more than 5 tools, semantically overlapping tools, sequential execution of parallel calls,anywhen text is valid). Matters because subagent distribution is the production answer when one agent needs more than the optimum.agents-sdksubagents and MCP (lesson:tool-bindingreferences; Agent SDK overview). Spawning specialised subagents with focused toolsets is the coordinator pattern the reference page's table implies. The Agent SDKallowedTools/disallowedToolsmodel is the concrete role-scope mechanism.stricttool use andcache_control(beyond task). Combininganywithstrictguarantees both a call and schema conformance;cache_controlon tool definitions cuts cost across turns. Relevant when forcing extraction output.input_examples,defer_loading,allowed_callers(beyond task). These are the advanced distribution levers: examples for complex tools, deferred loading for thousand-tool catalogs, caller scoping for code-execution isolation- MCP integration (Task 2.4). Resources as content catalogs reduce exploratory calls;
.mcp.jsonversus~/.claude.json;${ENV}expansion. Adjacent because MCP is where large tool counts originate and wheredefer_loadingonmcp_toolsetapplies. - Agentic-architecture connection (
lesson:tool-orchestrationand the multi-agent overview). Tool distribution is the implementation half of coordinator-subagent design: the coordinator owns a small meta-toolset (delegation to subagents) while each subagent owns a focused 4-5 tool set. The exam's Domain 1 (Agentic Architecture and Orchestration, 27% weight) and Domain 2 (Tool Design and MCP, this task) meet exactly at this boundary, so a candidate who understands role-scoped distribution understands both domains' highest-weight material.
Worked production examples
The four examples below are connected, not disconnected toys. Example A establishes the tool_choice and disable_parallel_tool_use mechanics on a research agent loop and evolves through forced, auto, and any modes. Example B shows role-scoped toolsets in the Agent SDK, the production form of the reference page's research table. Example C demonstrates consolidation with an enum and input_examples, the reference page's 22-to-4 collapse. Example D shows deferred loading with the tool search tool, the scale answer when a single agent must see a large catalog. Together they cover every required surface: all four tool_choice modes with disable_parallel_tool_use, role-scoped toolsets, consolidation with an enum, and deferred loading with tool search. Each block states what it proves, its failure boundary, and its observable output.
Worked production examples: Example A: a research agent loop using all four tool_choice modes with disable_parallel_tool_use
This connected implementation evolves through the section. It models the reference page's research system: a document-analysis agent that forces extract_metadata first, a synthesis agent that uses auto with single-call capping, and a coordinator that routes. The model identifier is a placeholder; substitute your deployed model.
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic();
const MODEL = "<model>"; // substitute your deployed model
// A small helper that runs one turn and returns the response.
// disable_parallel_tool_use rides alongside tool_choice so a scoped agent
// emits at most one tool call per turn under auto, exactly one under any/tool.
async function turn(
tools: Anthropic.Tool[],
messages: Anthropic.MessageParam[],
toolChoice: Anthropic.ToolChoice,
): Promise<Anthropic.Message> {
return client.messages.create({
model: MODEL,
max_tokens: 4096,
tools,
tool_choice: toolChoice,
messages,
});
}
// 1. Document-analysis agent: forced first step.
// Turn 1 uses { type: "tool", name: "extract_metadata" } so metadata runs
// before any enrichment. Subsequent turns switch to auto.
const docTools: Anthropic.Tool[] = [
{
name: "extract_metadata",
description:
"Extract document title, author, publish date, and section headings. " +
"MUST run as the first analysis step before any enrichment tool.",
input_schema: {
type: "object",
properties: { documentId: { type: "string" } },
required: ["documentId"],
},
},
{
name: "extract_data_points",
description:
"Pull quantitative claims and entities from an already-metadata'd document.",
input_schema: {
type: "object",
properties: { documentId: { type: "string" } },
required: ["documentId"],
},
},
{
name: "summarize_content",
description: "Summarize a document whose metadata is already known.",
input_schema: {
type: "object",
properties: { documentId: { type: "string" } },
required: ["documentId"],
},
},
{
name: "verify_claim",
description:
"Check a single factual claim against the document. Use only for simple, " +
"single-source checks; escalate multi-source verification to the coordinator.",
input_schema: {
type: "object",
properties: {
claim: { type: "string" },
documentId: { type: "string" },
},
required: ["claim", "documentId"],
},
},
];
// Observable output: turn 1 stop_reason is "tool_use" and the only possible
// tool is extract_metadata. The loop cannot skip or reorder the step.
const step1 = await turn(
docTools,
[{ role: "user", content: "Analyze document doc-4471." }],
{ type: "tool", name: "extract_metadata" },
);
console.log(step1.stop_reason); // "tool_use"
console.log((step1.content[0] as Anthropic.ToolUseBlock).name); // "extract_metadata"
// 2. Synthesis agent: auto with single-call cap.
// disable_parallel_tool_use: true under auto means at most one tool call per
// turn; the agent may still answer with text when no tool is needed.
const synthTools: Anthropic.Tool[] = [
{
name: "compile_report",
description: "Assemble the final report from provided analysis results.",
input_schema: {
type: "object",
properties: { findings: { type: "array", items: { type: "string" } } },
required: ["findings"],
},
},
{
name: "verify_fact",
description:
"Scoped cross-role tool. Verify a SIMPLE, single-source fact during report " +
"generation. For complex multi-source verification, return a note to escalate " +
"to the coordinator; do not attempt it here.",
input_schema: {
type: "object",
properties: {
fact: { type: "string" },
source: { type: "string" },
},
required: ["fact", "source"],
},
},
{
name: "format_citation",
description: "Format a citation for a given source id.",
input_schema: {
type: "object",
properties: { sourceId: { type: "string" } },
required: ["sourceId"],
},
},
{
name: "assess_coverage",
description: "Assess whether the report covers every required angle.",
input_schema: {
type: "object",
properties: { report: { type: "string" } },
required: ["report"],
},
},
];
const synthTurn = await turn(
synthTools,
[{ role: "user", content: "Draft the report from the analysis results." }],
{ type: "auto", disable_parallel_tool_use: true },
);
// Observable output: if the agent calls a tool, exactly one tool_use block
// appears; if it can answer directly, stop_reason is "end_turn".
// 3. Coordinator: any on a classification step, then auto for the rest.
const coordTools: Anthropic.Tool[] = [
{
name: "spawn_subagent",
description: "Delegate a subtask to a specialised subagent.",
input_schema: {
type: "object",
properties: {
role: { type: "string", enum: ["web_search", "document_analysis", "synthesis"] },
task: { type: "string" },
},
required: ["role", "task"],
},
},
{
name: "review_output",
description: "Review a subagent's output against the request.",
input_schema: {
type: "object",
properties: { output: { type: "string" } },
required: ["output"],
},
},
{
name: "request_revision",
description: "Send a revision request back to a subagent.",
input_schema: {
type: "object",
properties: { feedback: { type: "string" } },
required: ["feedback"],
},
},
];
// Turn 1 forces a routing decision via any (every input must be classified).
const route = await turn(
coordTools,
[{ role: "user", content: "Handle this research request." }],
{ type: "any" },
);
// Observable output: stop_reason is always "tool_use"; the agent picked one of
// spawn_subagent / review_output / request_revision. A loop here MUST cap turns
// or it never terminates, because any never yields end_turn.What each block proves: forced selection guarantees ordering; auto with disable_parallel_tool_use caps to one call yet still allows text; any forces a call but needs a turn cap. Failure boundary: forgetting the cap on any produces the endless-loop antipattern. Observable output is the stop_reason and the single tool_use name.
Worked production examples: Example B: role-scoped toolsets in the Agent SDK
This example scopes a document-analysis subagent to exactly its role and scopes a sensitive tool to a caller. It mirrors Anthropic's per-role tool-scoping cookbook pattern.
import { Client } from "@anthropic-ai/claude-agent-sdk";
// A document-analysis subagent: only its four role tools are available.
// Omitting a built-in from `tools` keeps it out of context, so the subagent
// cannot drift into web search or shell commands.
const docAnalysisAgent = new Client({
tools: ["Read", "extract_metadata", "extract_data_points", "verify_claim"],
// allowedTools gates which calls run without approval; a scoped rule such as
// "Bash(rm *)" would deny only matching calls but keep the tool visible.
allowedTools: ["Read", "extract_metadata", "extract_data_points", "verify_claim"],
// disallowedTools with a bare name removes a tool from context entirely.
// Here we ensure the agent never reaches a generic fetch.
disallowedTools: ["WebFetch", "Bash"],
});
// A sensitive tool scoped to code execution only, not direct model calls.
// allowed_callers omits "direct", so the model cannot name it in a tool_use
// block; only code running inside the sandbox may call it.
const sensitiveTool = {
name: "delete_record",
description: "Permanently delete a record. Only callable from code execution.",
input_schema: {
type: "object",
properties: { recordId: { type: "string" } },
required: ["recordId"],
},
allowed_callers: ["code_execution_20260120"],
};
// A web-search researcher subagent gets only search-scoped tools, never the
// document-analysis tools, preventing cross-role misuse.
const webSearchAgent = new Client({
tools: ["WebSearch", "WebFetch", "save_snippet"],
allowedTools: ["WebSearch", "WebFetch", "save_snippet"],
disallowedTools: ["extract_metadata", "extract_data_points", "Read"],
});What this proves: role scoping is enforced at two layers, availability (which tools appear in context) and permission (which calls are approved). A synthesis agent that holds only compile_report, verify_fact, format_citation, assess_coverage cannot run its own web searches. allowed_callers adds a caller scope so a destructive tool is reachable only from sandboxed code, not from the model directly. Failure boundary: a scoped disallowedTools rule that leaves the tool visible lets the model waste a turn attempting it; use a bare name to remove it entirely.
Worked production examples: Example C: consolidation with an enum and input_examples
The 22-tool data platform collapses to four. The 19 transformations become one transform_data tool with a transform_type enum. A query agent keeps three query tools (one per source); a transformation agent keeps the single consolidated tool plus load_dataset and save_dataset. The example also shows input_examples on a format-sensitive tool.
{
"name": "transform_data",
"description": "Apply a transformation to a dataset. Use transform_type to select the operation. Every former per-operation tool (pivot_table, calculate_percentile, normalise_currency, and 16 more) is now one enum value, so the agent chooses an operation inside a single call instead of picking among nineteen near-identical tools.",
"input_schema": {
"type": "object",
"properties": {
"dataset": { "type": "string", "description": "Identifier of the dataset to transform" },
"transform_type": {
"type": "string",
"enum": ["pivot", "percentile", "normalise_currency", "rank", "rolling_average", "fill_missing", "dedupe_rows", "cast_type", "group_by", "join", "filter_rows", "sort", "round", "clip", "encode_category", "window_diff", "cumulative_sum", "resample", "bin"]
},
"options": { "type": "object", "description": "Operation-specific parameters" }
},
"required": ["dataset", "transform_type"]
},
"input_examples": [
{ "dataset": "sales_q1", "transform_type": "pivot", "options": { "index": "region", "columns": "product", "values": "revenue" } },
{ "dataset": "sessions", "transform_type": "percentile", "options": { "column": "duration_ms", "p": 0.95 } },
{ "dataset": "ledger", "transform_type": "normalise_currency", "options": { "target": "USD" } }
]
}// Before consolidation the transformation agent held 19 tools; selection
// accuracy degraded. After consolidation it holds four:
const transformAgentTools = [
{
name: "load_dataset",
description: "Load a dataset by id into the working context.",
input_schema: {
type: "object",
properties: { datasetId: { type: "string" } },
required: ["datasetId"],
},
},
transformDataTool, // the consolidated enum tool above
{
name: "save_dataset",
description: "Persist a transformed dataset.",
input_schema: {
type: "object",
properties: { datasetId: { type: "string" } },
required: ["datasetId"],
},
},
{
name: "query_source",
description: "Query one of three data sources. Use source to pick which.",
input_schema: {
type: "object",
properties: {
source: { type: "string", enum: ["warehouse", "lake", "stream"] },
sql: { type: "string" },
},
required: ["source", "sql"],
},
},
];What this proves: consolidation shrinks the choice from nineteen to one enum, so selection accuracy improves because the hard choice got smaller, not because capability changed. input_examples shows the model the exact options shape for the three trickiest operations, reducing format errors. The query agent keeps three tools (not consolidated, because they are genuinely different jobs, one per source) and the transformation agent keeps four. Nothing is lost: every former transformation is reachable as an enum value. Failure boundary: do not consolidate tools that are different jobs (that is the split-by-role fix, not consolidation); and do not mistake consolidation for constraint, because consolidation grants no new power.
Worked production examples: Example D: deferred loading with the tool search tool
This example gives Claude a thousand-tool catalog but loads only what each request needs. The tool search tool stays non-deferred; the rest defer.
{
"model": "<model>",
"max_tokens": 2048,
"tools": [
{ "type": "tool_search_tool_regex_20251119", "name": "tool_search_tool_regex" },
{
"name": "get_weather",
"description": "Get the weather at a specific location",
"input_schema": {
"type": "object",
"properties": {
"location": { "type": "string" },
"unit": { "type": "string", "enum": ["celsius", "fahrenheit"] }
},
"required": ["location"]
},
"defer_loading": true
},
{
"name": "search_files",
"description": "Search through files in the workspace",
"input_schema": {
"type": "object",
"properties": {
"query": { "type": "string" },
"file_types": { "type": "array", "items": { "type": "string" } }
},
"required": ["query"]
},
"defer_loading": true
}
]
}import anthropic
client = anthropic.Anthropic()
MODEL = "<model>" # substitute your deployed model
# Every deferred definition is still sent each request; the API searches server
# side and expands tool_reference blocks. Keep the search tool non-deferred.
response = client.messages.create(
model=MODEL,
max_tokens=2048,
messages=[{"role": "user", "content": "What is the weather in San Francisco?"}],
tools=[
{"type": "tool_search_tool_regex_20251119", "name": "tool_search_tool_regex"},
{
"name": "get_weather",
"description": "Get the weather at a specific location",
"input_schema": {
"type": "object",
"properties": {
"location": {"type": "string"},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
},
"required": ["location"],
},
"defer_loading": True,
},
{
"name": "search_files",
"description": "Search through files in the workspace",
"input_schema": {
"type": "object",
"properties": {
"query": {"type": "string"},
"file_types": {"type": "array", "items": {"type": "string"}},
},
"required": ["query"],
},
"defer_loading": True,
},
],
)
# Observable output: response contains a server_tool_use block for the search,
# a tool_search_tool_result with tool_references, then a tool_use for get_weather.
# You execute get_weather and return a tool_result; you never expand the reference.What this proves: deferred loading keeps the initial context to the search tool plus any non-deferred tools, so context stays small even with 10,000 deferred tools, and selection accuracy stays high because only 3-5 discovered tools load per request. It preserves prompt caching because deferred tools are excluded from the cache prefix. Failure boundary: setting defer_loading: true on every tool (including the search tool) returns a 400 "At least one tool must have defer_loading=false"; and a tool_reference pointing to a tool not in tools returns a 400 "not found". Keep the 3-5 most used tools non-deferred so common calls skip the search.
Build exercise material
A verifiable build sequence for "Configure Tool Distribution Across a Multi-Agent System", adapted from the reference page's build exercise. Each step lists the observable outcome that proves it worked.
Step 1. Design three agent roles (web search, document analysis, synthesis) and assign 4-5 tools to each, scoped to its role. Observable outcome: a configuration object or table listing three agents, each with exactly 4-5 tools; no tool appears in more than one role except scoped cross-role tools added later. Tool names clearly indicate purpose and scope.
const distribution = {
web_search: ["search_web", "fetch_page", "extract_links", "save_snippet"],
document_analysis: ["extract_metadata", "extract_data_points", "summarize_content", "verify_claim"],
synthesis: ["compile_report", "verify_fact", "format_citation", "assess_coverage"],
coordinator: ["spawn_subagent", "review_output", "request_revision"],
};
// Observable: each array length is 4. No tool repeats across the three role agents.Step 2. Add a scoped verify_fact to the synthesis agent for simple lookups, routing only complex verifications to the coordinator. Observable outcome: the synthesis toolset includes verify_fact with a description that limits it to simple single-source lookups and states complex cases escalate to the coordinator.
Step 3. Configure forced selection on the document-analysis agent so extract_metadata runs as the mandatory first step. Observable outcome: the first API call uses tool_choice: { type: "tool", name: "extract_metadata" }; subsequent calls switch to tool_choice: { type: "auto" }. You can assert this by logging the tool_choice sent per turn and confirming turn 1 names extract_metadata.
Step 4. Replace a generic fetch_url with a constrained load_document that validates document URLs only. Observable outcome: load_document includes URL validation logic (checking document file extensions or trusted domains) and rejects non-document URLs with a clear error. A test that passes a non-document URL receives the rejection, proving the constraint holds.
Step 5. Test with a query that requires all three agents and verify no cross-role misuse occurs. Observable outcome: a run log showing the web search agent using only its tools, the document-analysis agent starting with extract_metadata (forced), and the synthesis agent using compile_report plus verify_fact for simple checks; no agent calls a tool outside its assigned set. You can assert this by checking every tool_use name against the role's allowed list.
Step 6. Add disable_parallel_tool_use: true on the scoped single-purpose turns and confirm exactly one call per turn. Observable outcome: parse each response; assert at most one tool_use block under auto with the flag, exactly one under any or forced. This validates the disable behaviour end to end.
The whole exercise is verifiable by assertions, not by inspection: every step has an observable outcome a test can check. That is the production discipline the exam expects. A configuration where any agent holds more than 5 tools, where a tool appears in two role sets, where a generic fetcher is unconstrained, or where a forced step is absent is, by construction, a distribution defect. Run the six steps and the run log itself becomes the proof that tool distribution prevents cross-role misuse.
Failure modes and how distribution prevents them
Concrete failure scenarios make the mechanisms stick. Each maps a distribution decision to the failure it prevents.
Failure: tool overload misrouting. An agent with 18 tools picks lookup_order when the task needed get_customer, or hallucinates a tool name that is not in the list. Prevention: cap each agent at 4-5 role-scoped tools, or use tool search with defer_loading so only 3-5 relevant tools load per request. The 4-5 lesson optimum and the 30-50 documented degradation threshold describe the same curve at different points; both counsel fewer tools in context.
Failure: cross-role misuse. A synthesis agent runs its own web searches instead of using results already handed to it, duplicating work and burning context. Prevention: give the synthesis agent only compile_report, verify_fact, format_citation, assess_coverage; keep web search tools out of its context entirely via disallowedTools or by simply not binding them. Relevance scoping, not description quality, is the fix.
Failure: coordinator round-trip latency. A synthesis agent returns control to the coordinator for every simple fact check, adding 2-3 hops per request and up to 40% latency, when 85% of checks are simple. Prevention: a scoped verify_fact on the synthesis agent for simple single-source lookups, escalating only complex cases. This is official sample Q9 and a direct exam target.
Failure: skipped mandatory step. An agent jumps straight to enrichment and never runs extract_metadata, producing analysis on unknown inputs. Prevention: forced selection tool_choice: { type: "tool", name: "extract_metadata" } on turn 1, then auto afterward. Forced selection is the only mode that guarantees a specific call.
Failure: infinite loop under any. A refund agent with any re-looks-up the order, re-verifies, re-processes, re-notifies forever because stop_reason is always "tool_use". Prevention: a maximum-turn guard, then a switch to auto so the agent can emit end_turn with a summary. This is the single most-tested tool_choice antipattern.
Failure: generic-tool misuse. A subagent with fetch_url fetches an arbitrary, non-document URL, causing side effects or wasted calls. Prevention: replace with a constrained load_document that validates document URLs. Least privilege applied to tool design.
Failure: parallel-call side effects. Two tool_use blocks in one turn run concurrently when one should precede the other (create order, then send confirmation). Prevention: disable_parallel_tool_use: true on turns that must be sequential, or execute dependent calls across turns as Claude naturally sequences them. The flag governs generation, not execution; your code still decides run order.
Failure: context bloat at scale. A multi-server setup loads 55k tokens of tool definitions before any work. Prevention: defer_loading: true on the long tail and tool search to load only what each request needs, preserving prompt caching.
Each failure is an application or configuration defect, never a model defect. The recurring lesson: distribution is owned by the layer that assembles the tools array and sets tool_choice, so that is where every fix lives.
Mechanism and API surface
- Per agent allowlists and coordinator leanness
- Use options dot agents and options dot allowedTools with Task or Agent as the meta tool, assigning four to five domain tools per specialist such as web search, document analysis, and synthesis, while the coordinator stays without domain tools and owns routing.
- Scoping versus packing
- Scoping keeps each turn's choice among four or five role specific tools and eliminates cross role misuse. Packing eighteen tools onto one agent raises decision complexity and misrouting irrespective of description quality.
- Homogeneous consolidation pattern
- Collapse nineteen near duplicate tools sharing data in, operation, data out into one parameterised tool such as transform_data with dataset, transform_type as an enum of operations, and an options object, preserving capability at lower choice complexity.
- Scoped cross role verify_fact
- Give synthesis a verify_fact limited to simple single source lookups for the eighty five percent frequent path, with description stating complex multi source verification should be escalated through the coordinator for full pipeline quality.
- tool_choice auto, any, and forced named
- auto lets the model decide freely whether to call a tool, any forces at least one tool call while letting the model choose which, forced named with type tool and a name enforces a mandatory first step such as extract_metadata on this turn.
- Flat list invisibility of server boundaries
- Every tool from every connected MCP server is merged into one prompt visible list. Splitting across servers without consolidation leaves the model's option count unchanged and is not a distribution fix.
The decision rules in play
Each rule below is a distinct decision the exam tests, stated with its mechanism, the boundary where the answer flips, and the wrong answers written against it.
Role-scoped small tool sets preserve reliable selection
Each agent receives only the tools its role requires in this turn. The orchestrator holds the delegation primitive often called Agent or Task, domain agents hold three to five tools that map directly to their job. in the tested material the recurring healthy pattern looks like a web search agent with search_web, fetch_page, extract_links, save_snippet, a document analysis agent with extract_metadata, extract_data_points, summarize_content, verify_claim, and a synthesis agent with compile_report, a narrowly scoped verify_fact, format_citation, assess_coverage.
A small set implementation in the Agent SDK style often looks like this:
The same principle appears on the Messages API side as an allowlist in the request:
Inline references such as allowedTools, lookup_order, and tool_choice keep the scope explicit. The agent never negotiates its own scope at runtime.
// role-scoped definitions for a web search specialist
const webSearchAgent = defineAgent({
name: "web_search",
description: "Retrieves and extracts web content for the coordinator",
allowedTools: ["search_web", "fetch_page", "extract_links", "save_snippet"],
systemPrompt: "Use only retrieval tools. Do not summarize or synthesize."
});{
"tools": [
{"name": "lookup_order", "description": "Returns order status by order identifier"},
{"name": "get_customer", "description": "Returns profile by customer identifier"},
{"name": "check_return_eligibility", "description": "Checks eligibility for a return"},
{"name": "create_return_label", "description": "Creates a label after eligibility passes"}
],
"tool_choice": {"type": "auto"}
}Tool selection is a discrimination task. With three to five candidates that differ by purpose and input shape, the model's comparison is low dimensional and the description signal dominates. Each additional tool adds competing descriptions that may share verbs such as lookup, search, fetch, or get, and share argument names such as id or query.
Boundary. This rule governs when the symptom is too many options. When the set is already small but misrouting persists, the nearby opposite case applies. Two tools inside a four-tool set can still be confused if their descriptions overlap or their identifiers look alike, for example fetch_account versus retrieve_contract both described as returning data from the system. In that small-set case the fix is not a further split.
Recurring specifics. the tested material repeatedly treats a small set around four to five tools per agent as the reliable operating range, described as optimal or recommended in the underlying guidance. Counts in the high teens and twenties appear as the canonical overloaded case, and very large catalogs in the hundreds appear as the context-bloat case. Role names that recur include web search, document analysis, synthesis, report generation, billing, shipping, returns, and account administration.
Proposal. improve every description but keep all eighteen to twenty plus tools on one agent.
Why it attracts. it preserves capability without re-architecture and looks cheaper.
Why it fails. eighteen overlapping descriptions still require eighteen-way discrimination on every turn, so misrouting and hallucinated arguments persist.
When it would be right. when the overloaded count is actually small, for example four or five tools with overlapping wording, and the symptom is ambiguous boundaries rather than decision overload.
Proposal. add a routing classifier that pre-selects the tool before the model reasons.
Why it attracts. it looks like deterministic control.
Why it fails. it adds a separate brittle component that must replicate the model's language understanding and bypasses the model's own judgment, while leaving the overloaded set in place for other turns.
When it would be right. there is no recurring case in the tested material where this is preferred over scoping. It is offered as a complexity trap.
- - One variant grows a single server by adding nineteen transformation tools to three query tools and asks which redesign helps. The mutation teaches that splitting the nineteen across roles does not help when they share one shape, which links to Rule 4. - Another variant distributes tools across multiple servers but keeps the visible count high and asks whether moving tools between servers helps. The mutation teaches flat list merging from Rule 6. The correct answer still requires per-agent scoping or consolidation, not server shuffling. - A third variant keeps the synthesis agent at four tools but gives it a full eighteen-tool set just in case. The mutation teaches temptation to act outside specialization, which is also Rule 3 and Rule 18. The fix is to pare back to the synthesis role set plus one scoped cross-role lookup.
Selection pressure grows with tool count and consumes context and reasoning budget
Every tool definition, the automatically injected tool-use instruction, and every tool_use and tool_result block counts toward tokens and context. On the wire the request carries a tools array with full schemas. In context the model sees tool names, descriptions, argument names, and argument descriptions.
An overloaded request header illustrates the pressure:
With four tools the header is light. The same pattern with seventy or three hundred tools replaces the short list with a long catalog that the model must scan before every decision. Inline costs such as token spend on unused definitions recur in the tested material as a measurable concern.
{
"tools": [
{"name": "search_knowledge_base", "description": "Finds help articles"},
{"name": "process_action", "description": "Handles customer actions"},
{"name": "create_ticket", "description": "Opens a support ticket"},
{"name": "update_customer_profile", "description": "Updates stored profile fields"}
],
"tool_choice": {"type": "auto"}
}Two costs compound. First, token cost and latency: unused definitions are paid for on every request even when the workflow never invokes them. Second, accuracy cost: more candidates lower the chance that the best description wins the comparison, especially when verbs overlap or when system prompt keywords bias the choice toward an adjacent tool.
Boundary. When the catalog is small and every tool is used in a typical conversation, the opposite principle applies. the tested material shows a twelve-tool catalog that is almost always exercised and occupies only a small fraction of context. Deferring such a catalog adds a discovery step on nearly every turn while saving little, so immediate availability is preferred.
Recurring specifics. Terms that recur include context window, input tokens, tools array, tool_use, tool_result, prompt caching, and defer_loading. Catalog sizes that trigger discussion include a few dozen, around seventy, around one hundred eighty, around two hundred fifty, around three hundred fifty, and five hundred plus.
Proposal. keep all definitions but enable prompt caching so they stop counting.
Why it attracts. it promises cost relief without changing scope.
Why it fails. cached prefixes still occupy the context window and still participate in selection. Caching changes billing, not attention.
Proposal. move to a larger model to tolerate larger headers.
Why it attracts. it looks like a capability upgrade.
Why it fails. it scales the bloat rather than removing it and does not address misrouting. Selection noise remains.
- - A variant with seventy tools notes that only about a dozen are routinely invoked. The mutation teaches pruning or deferral rather than keeping the full header, linking to Rules 8 and 14. - A variant with twelve tools notes that most are invoked every conversation and latency is critical. The mutation teaches keeping the full header and not deferring, which is the opposite boundary of the same rule.
Cross-role misuse is structural, not motivational, and requires structural removal
Agents that hold tools outside their specialization tend to use them. the tested material shows synthesis agents given web search and document parsing tools launching ad-hoc searches mid-synthesis, web search agents given summarization and sentiment tools invoking analysis tools when they should only fetch raw data, account agents given returns and loyalty tools adjusting points during a return, and developer productivity agents given deployment tools invoking scale_replicas or update_config during codebase exploration.
A coordinator with isolated specialists illustrates the intended shape:
Inline references such as compile_report, verify_fact, and search_web mark the intended boundary. The synthesis agent can still perform quick lookups via its one scoped cross-role tool described in Rule 18, but it cannot launch the full search pipeline on its own.
# .claude/agents/synthesis.md
name: synthesis
description: Combine verified findings into a cited report
tools: [compile_report, verify_fact, format_citation, assess_coverage]# .claude/agents/web_search.md
name: web_search
description: Retrieve raw web content for later synthesis
tools: [search_web, fetch_page, extract_links, save_snippet]Availability shapes behavior more reliably than instruction. A prompt reminder such as only use tools appropriate to your role leaves all eleven capabilities callable, so any ambiguous request or injected phrase can trigger a foreign tool. Capability presence also expands the decision space in Rule 2.
Boundary. The nearby opposite case is when cross-role access is genuinely needed at high frequency. If eighty five percent of an agent's turns need a simple lookup that nominally belongs to another role, forcing every one through a coordinator adds two to three hops and large latency. In that narrow case, grant the agent a single scoped tool that handles only the simple variant, for example a verify_fact that handles single-source quick lookups and docs that complex multi-source verifications still route through the coordinator.
Recurring specifics. Phrases that recur include scoped tool access, least privilege, specialization, and incentives for misuse such as duplicate searches or verification during exploration. Tool categories that recur across misuse cases include search and fetch versus synthesis and formatting, and deployment tools appearing in exploration sessions.
Proposal. keep all tools but rewrite each description with do NOT use unless you are the X agent boundaries.
Why it attracts. it looks like a precise description fix.
Why it fails. it leaves the foreign capability reachable, so ambiguous phrasing or a prompt injection can still trigger it, and the decision complexity in Rule 2 remains.
Proposal. add a PreToolUse hook that blocks foreign calls.
Why it attracts. it looks like enforcement.
Why it fails. it is an over-engineered workaround for what should be a declarative scope, and it still leaves the foreign definitions occupying context and selection space.
- - A variant holds eighteen tools but only needs three and occasionally adjusts loyalty points during a return. The mutation teaches that the single fix is scope, not a prompt line. - A variant gives a synthesis agent the full eighteen-tool set just in case and sees duplicate unvetted findings mid-synthesis. The mutation teaches that just in case tools are never free, linking to Rules 2 and 18.
Homogeneous tools sharing a shape consolidate into one parameterized tool
When many tools vary only by one parameter while sharing inputs and outputs, they collapse into a single tool with an enum argument. The canonical case in the tested material is a data platform with three query tools plus nineteen transformation tools such as pivot_table, calculate_percentile, and normalise_currency. Each transformation takes a dataset, an operation, and options, and returns a transformed dataset. Rather than nineteen separate definitions, the design becomes one transform_data tool that switches on transform_type.
Inline details such as dataset, transform_type, and options preserve capability while collapsing the decision from nineteen candidates to one tool with enum selection. The catalog that was twenty two becomes four, and the hard choice moves inside the tool's typed parameter where schema validation applies.
Other homogeneous families in the tested material include lookup tools that differ only by identifier type such as by email, by phone, by identifier, and candidate search tools that differ only by scope. The same consolidation applies, for example a single lookup_user with identifier plus identifier_type rather than eight near identical tools.
{
"name": "transform_data",
"description": "Apply a transformation to a dataset. Use transform_type to select the operation.",
"input_schema": {
"type": "object",
"properties": {
"dataset": {"type": "string", "description": "Identifier or inline data for the input"},
"transform_type": {
"type": "string",
"enum": ["pivot", "percentile", "normalise_currency", "deduplicate", "filter"]
},
"options": {"type": "object", "description": "Operation-specific options"}
},
"required": ["dataset", "transform_type"]
}
}Homogeneous tools compete on nearly identical language, so description differentiation alone cannot reliably separate them, and splitting them across roles recreated the overload at the subagent level. Nineteen transformations handed to a transformation specialist is still nineteen-way discrimination. Collapsing them replaces nineteen descriptions with one description plus a constrained enum, which shrinks the per-turn decision space without removing capability.
Boundary. Consolidation is correct when tools share a shape. When tools represent different jobs such as query versus transformation versus export, consolidation is wrong because it hides distinct responsibilities behind one generic dispatcher and recreates a multipurpose blur that Rule 17 warns against. That opposite case requires role split, which is Rule 5.
Recurring specifics. Identifiers that recur include transform_data, transform_type, dataset, options, and the enum members pivot and percentile. Tool name patterns that signal homogeneity include lookup_user_by_email, lookup_user_by_phone, search_candidates, search_employees, search_requisitions, and find_talent.
Proposal. keep all nineteen tools and improve descriptions with examples and boundaries.
Why it attracts. it keeps tools narrow.
Why it fails. nineteen near identical descriptions still produce nineteen-way ambiguity, and the catalog still pays the context cost in Rule 2.
Proposal. split the nineteen across two servers.
Why it attracts. it looks like distribution.
Why it fails. flat merging in Rule 6 means the visible count is unchanged, so selection pressure is unchanged.
- - One variant offers to merge into
universal_queryand route by syntax. The mutation teaches that data source distinctions matter to the agent and must remain visible to selection, not hidden behind a universal dispatcher. - Another variant proposesmanage_accountwithoperation_typeenum for four distinct lifecycle operations. the tested material rejects that merge because lifecycle operations differ in side effects and contracts, which links to Rule 17. The correct fix there is to split into purpose-specific tools, not to consolidate. - A third variant offers a routing classifier to pick among eight identifier tools. The mutation teaches that a classifier is unnecessary when a single typed search tool covers the use case.
Heterogeneous capabilities across concerns require role split, not consolidation
When tools cover different concerns such as scheduling, invoicing, document parsing, email, CRM lookup,, each concern maps to a role. The observed fix is to create role-specific agents with four to five tools each and a coordinator that delegates by concern. An analytics platform with ingestion, transformation, reporting, and alerting concerns becomes four specialists.
Inline role definitions such as billing, shipping, and allowedTools make the split explicit. Tasks that span concerns are decomposed and delegated in parallel rather than handled by one overloaded agent.
const billingAgent = defineAgent({
name: "billing",
description: "Handles billing inquiries and disputes",
allowedTools: ["lookup_order", "check_refund_eligibility", "issue_refund", "get_customer"]
});
const shippingAgent = defineAgent({
name: "shipping",
description: "Handles shipment tracking and delivery",
allowedTools: ["track_shipment", "estimate_delivery", "update_shipping_address", "get_order"]
});Heterogeneous tools do not share a parameter shape, so collapsing them into one dispatch_operation tool merely moves ambiguity inside the dispatcher's stringly typed argument while removing clear tool boundaries. Splitting preserves distinct contracts, so each agent's selection problem stays low-dimensional and its prompt stays focused on one concern. Evidence shows that splitting also helps with parallelism and context focus: separable subtasks execute concurrently against shared customer context, and each agent's context stays narrow, which links to Rule 2.
Boundary. When tools are heterogeneous but a single task routinely needs three of them in one turn on the same concern, splitting can add unnecessary hops. The nearby opposite is Rule 8 when the concern is the same but the trigger varies only by identifier, where a single parameterized lookup is better than three specialists with one tool each. The decision rule from the reference material is explicit.
Recurring specifics. Domain partitions that recur include web search versus document analysis versus synthesis, billing versus shipping versus returns versus account, and code intelligence versus deployment. Overload counts that trigger the split in the tested material include eighteen, twenty, twenty two, and twenty four tools on a single orchestrator.
Proposal. replace twenty four tools with one dispatch_operation with operation_name.
Why it attracts. it reduces visible count to one.
Why it fails. it reintroduces the generic dispatch blur and loses the input and output contracts that make selection reliable.
When it would be right. when the operations share a true common shape, which is Rule 4, not this case.
Proposal. keep one orchestrator and describe each tool in the system prompt alongside auto mode.
Why it attracts. richer prompt signal.
Why it fails. it does not reduce decision complexity and adds no structural guarantee.
When it would be right. when the set is already small and the symptom is ambiguous wording, which is Rule 10.
- - A variant proposes consolidating query tools into
universal_query. The mutation teaches that source distinction matters for selection and should not be hidden. - A variant gives each specialized agent a copy of all twelve tools to make each self sufficient. The mutation teaches that self sufficiency recreated by duplication reintroduces overload and cross-role misuse, linking to Rules 1 and 3, and is distinguished from the scoped exception in Rule 18.
Server boundaries are invisible, every connected tool merges into one flat decision list
The client gathers every tool from every connected MCP server and presents one flat list to the model. Configuration entries such as mcp_servers or .mcp.json entries are connection details. Toolset entries such as mcp_toolset entries in the tools array are the capability scope.
A configuration snippet illustrates the mechanism on the connector side:
Inline keys such as mcp_servers, mcp_toolset, default_config, and configs make the separation between where a server runs and what the model sees explicit.
# .mcp.json - connection definitions, not scoping
mcp_servers:
issue_tracker:
command: "node"
args: ["./servers/issue-tracker.js"]
docs_wiki:
command: "node"
args: ["./servers/docs-wiki.js"]// request tools array - scoping happens here
{
"tools": [
{"type": "mcp_toolset", "mcp_server": "issue_tracker", "default_config": {"enabled": true}},
{"type": "mcp_toolset", "mcp_server": "docs_wiki", "default_config": {"enabled": false}, "configs": [{"name": "search_docs", "enabled": true}]}
]
}Discovery happens at connection time. The agent connects to every configured server and assembles a single unified tool set. There is no per-server session isolation, no lazy fetch by server name, and no automatic namespace routing.
Boundary. Server boundaries do matter for ownership, deployment, and versioning, and for resource hierarchies covered in Rule 19. They simply do not affect selection pressure.
Recurring specifics. Artifacts that recur include .mcp.json at the project root versus ~/.claude.json for personal servers, .claude/agents/ frontmatter, and the mcp__<server>__<tool> naming convention for SDK-exposed tools. Scenario symptoms that recur include a codebase exploration agent calling view_logs while browsing files because deployment tools were present, and a returns agent adjusting loyalty points because an eighteen-tool set mixed concerns.
Proposal. split twenty two overloaded tools across two servers to reduce cognitive load.
Why it attracts. it looks like distribution.
Why it fails. the list remains twenty two and every tool remains reachable, so selection noise is unchanged.
Proposal. rely on tool namespacing by server to auto-route loyalty versus support requests.
Why it attracts. it suggests automatic isolation.
Why it fails. no such isolation exists, and the agent can invoke any tool regardless of which server supplied it.
- - A variant adds a loyalty server to
.mcp.jsonand asks whether the concern about cross-domain calls is valid. The mutation teaches that the concern is valid and scoped access is needed. - A variant configures personal staging versus shared production database servers and asks how to prevent routing to production when the query lacks environment context. The mutation teaches that distinct names, environment cues in descriptions, and session-level precedence instructions are needed because mere presence of both servers does not self-route.
Least-privilege allowlisting is the deterministic guard for excess capability
The MCP connector and SDK tool definitions separate definition from authorization. On the Messages API side the toolset is where scoping happens. Setting default_config.enabled to false and then enabling only needed entries by name in configs produces an allowlist. On the SDK side the equivalent is per-agent allowedTools whitelisting. The result is that capabilities that should never be reachable in this request never appear in context and can never be invoked, regardless of prompt phrasing or retrieved content.
Inline references such as default_config, configs, lookup_clinic_hours, and check_appointment_availability record the exact allowlist. No record-update, cancellation, or refund tool is present.
{
"tools": [
{
"type": "mcp_toolset",
"mcp_server": "practice_management",
"default_config": {"enabled": false},
"configs": [
{"name": "lookup_clinic_hours", "enabled": true},
{"name": "check_appointment_availability", "enabled": true}
]
}
],
"tool_choice": {"type": "auto"}
}Prompt instructions shape behavior but are not enforced by any system component. A model that is steered or manipulated can still emit a tool_use block for a tool it was told to avoid, and every definition still counts toward context.
Boundary. Allowlisting is the right guard when the workflow legitimately never needs the capability in this path, such as read-only scheduling that should never reach refund or write tools, or closing cycles where disbursement is irresponsible until validation is clean. The nearby opposite case is progressive discovery in Rule 14: when a large catalog contains legitimate occasional capabilities that are rarely but genuinely needed, allowlisting that permanently deletes them is too strong.
Recurring specifics. Keys that recur include mcp_toolset, mcp_servers, default_config.enabled, configs, allowedTools, and tools: frontmatter. Scenario triggers that recur include platforms exposing forty or seventy tools where only two to twelve are needed, and the phrase least privilege appears as the design intent.
Proposal. add a system prompt line such as never invoke refund tools.
Why it attracts. it is the smallest change.
Why it fails. prompt rules are probabilistic and the definitions remain present, so a nudge from the user or from retrieved content can still trigger the tool.
When it would be right. as a secondary nudge already assuming scope is correct, not as the primary guard for sensitive or destructive tools.
Proposal. enable defer_loading on the toolset so unused definitions stay out of context.
Why it attracts. it promises scoping.
Why it fails. deferral changes when definitions become visible, not whether they are authorized. Permission is still needed.
When it would be right. when the problem is context pressure or catalog size rather than excess authority.
- - A variant with forty tools needs two and asks which configuration change best addresses excess capability. The mutation isolates allowlisting from deferred loading and from prompt steering. - A variant with seventy tools proposes pruning unused definitions and adopting search. The mutation teaches that pruning is the least privilege move while search handles the remaining long tail, linking Rules 8 and 14.
Pruning uninvoked definitions reduces both token cost and selection noise
Capabilities that no workflow invokes are deleted from the tools array entirely. This is distinct from deferring them in Rule 14. Pruning is a permanent removal for the request.
A pruned request reflects the decision directly:
Inline names such as lookup_order, check_return_eligibility, and create_return_label record the minimal set. Removed entries like loyalty adjustment or subscription changes are not present in any form.
{
"tools": [
{"name": "lookup_order", "description": "Returns order status, shipment, and item detail by order identifier"},
{"name": "check_return_eligibility", "description": "Evaluates policy against order state and timestamps"},
{"name": "create_return_label", "description": "Generates a label for an eligible return"}
],
"tool_choice": {"type": "auto"}
}Every definition you send has two compounding costs. It is billed as input context, including the automatically injected tool-use instruction and the future tool_result blocks, and it competes for selection attention. Pruning is the simplest least-privilege move: a capability that does not exist cannot be misused, and a decision you do not have to make cannot be made wrongly. It simultaneously shrinks the selection space the model reasons over, which links to Rule 2.
Boundary. Pruning applies when no approved scenario for the agent's role needs the capability. When a capability is genuinely needed but rare, deletion is too strong and progressive discovery in Rule 14 is preferred. That keeps the capability available server side without paying its context share until discovered.
Recurring specifics. Numbers that recur as signals for pruning include forty, seventy, and eighteen tool sets trimmed to two to five. Phrases that recur include never invokes, only ever needs, and capability bloat. Related tactics cited in the same evidence set include consolidating related operations into fewer tools with an action parameter when the family shares a shape, which is Rule 4, and keeping tool responses high signal to reduce context after invocation.
Proposal. set tool_choice to any so the model always selects a tool.
Why it attracts. it forces action.
Why it fails. it guarantees a call happens, not that it is correct, and suppresses reasoning while degrading accuracy.
Proposal. enable prompt caching on the tools array.
Why it attracts. it promises cheaper reuse.
Why it fails. cached prefixes are billed differently but still occupy context and still participate in selection, so neither cost nor misrouting is removed.
- - A variant shows seventy tools but also asks about a second fix alongside pruning. The mutation teaches that pruning pairs with progressive discovery for the remaining long tail rather than with forcing or caching. - A variant shows an agent with hundreds of tools and asks whether to remove unused definitions entirely. The mutation isolates pruning from payload truncation tricks and teaches that definitions should be fully described when present.
Prompt instructions are probabilistic steering, never enforcement
System prompts, CLAUDE.md guidance, and per-turn instructions shape the model's preference but do not remove capabilities. The model may still emit a tool_use block for a tool it was told to avoid, especially when the user request is ambiguous, contains trigger words, or when retrieved content contains embedded directives.
Prompt following varies with model generation and with prompt strength, and instruction sensitivity itself shifts across releases. Emphatic directives like CRITICAL: You MUST that compensated for undertriggering on earlier models now overshoot on more instruction-sensitive models, causing overtriggering on simple conversational turns.
Boundary. Prompt shaping is appropriate as the primary fix when the problem is tone, default posture, or conservative versus proactive action, for example telling a coding assistant to default to research and recommendations and to act only when explicitly requested. It is also appropriate as a secondary nudge after scope and descriptions are correct, for example biasing toward tool use without forcing it.
Recurring specifics. Phrasings that recur include CRITICAL, You MUST use, If in doubt, use, anti-laziness prompting, and tuning across model migrations. System artifacts that recur include CLAUDE.md, .claude/rules/, and system prompt preambles that enumerate tool purpose. Symptom language that recurs includes misfiring on trigger words, reduced but not eliminated misroutes, and new misroutes introduced in the other direction after a one-sided prompt addition.
Proposal. rewrite the system prompt to list every tool and when to use it.
Why it attracts. it looks like comprehensive guidance.
Why it fails. eighteen listed tools still produce an eighteen-way comparison and no enforcement, so selection noise persists.
Proposal. add few-shot examples showing the right tool for each phrasing.
Why it attracts. it looks like teaching.
Why it fails. it adds token overhead, covers only demonstrated phrasings, and does not generalize to novel edge cases where descriptions remain ambiguous.
- - A variant contains a broad rule such as whenever a user says find, search, locate, or look up, call a lookup tool. The mutation teaches to replace it with a narrower rule tied to whether live data inspection is needed. - A variant contains conflicting sources where a system prompt says critical only but a tool description says any severity. The mutation teaches that the description wins at selection time and must be corrected, linking to Rule 10.
Tool descriptions are the primary selection signal and must carry pairwise boundaries
Descriptions drive discrimination. When two tools share verbs or accept similar identifiers, the model has no reliable basis to choose and misroutes roughly a third to two fifths of the time in the tested material. The fix is to make each description unambiguous about purpose, input type, expected data source, output shape, and explicit guidance on when to use it versus the similar tool. The durable pattern is a boundary sentence that names the sibling.
Additional fields that recur as description enhancements include input formats, example queries, edge cases, and when to prefer an MCP tool over a built-in like Grep. When tools legitimately overlap in scope, the tested material emphasizes sharpening every overlapping description, not only the most confused pair, so no description reads as a superset of another.
{
"name": "get_rate",
"description": "Returns the sellable nightly rate for a date and room type. Input date, room_type. Use for pricing questions, not allotment counts. Do NOT use for raw availability counts, use get_room_allotment for that."
}{
"name": "get_room_allotment",
"description": "Returns raw allotment counts for a date and room type. Input date, room_type. Use for inventory counts, not sellable price. Do NOT use for pricing, use get_rate for that."
}Tool choice reasoning depends on description content before any output is observed. Names alone are insufficient, and reasoning steps outside the tool definition cannot reliably patch ambiguous contracts because the underlying ambiguity remains.
Boundary. Rewriting descriptions is the correct first fix when the toolkit is small enough to reason about and the symptom is overlap, for example five tools where get_customer and lookup_order both accept a simple identifier. The opposite principle applies when the toolkit is large enough that no wording quality rescues selection, such as eighteen to twenty two tools where the overloaded count itself is the disease. There description quality is still worth improving after scoping, but it cannot substitute for reducing decision complexity through role split or consolidation.
Recurring specifics. Pairs that recur include fetch_account versus retrieve_contract, search_flights versus search_hotels, open_ticket versus update_ticket, search_candidates versus search_employees versus find_talent, and built-in Grep versus mcp__github__search_issues. Keys that recur as description content include input formats, example queries, edge cases, trigger conditions, output shape, and explicit boundaries using phrasing such as use this instead of sibling when. Tool name fixes that recur include renaming query_inventory to get_room_allotment to remove superset wording.
Proposal. add a pre-routing classifier that inspects keywords before the model reasons.
Why it attracts. it looks deterministic.
Why it fails. it adds a brittle component that bypasses native language understanding and faces the same ambiguity the coordinator faces, while one edit per description solves it without machinery.
Proposal. merge overlapping tools into one manage_ticket or lookup_entity tool.
Why it attracts. it removes the two-way choice.
Why it fails. it pushes routing inside the implementation, removes distinct input and output contracts, and complicates validation.
When it would be right. when the family is homogeneous with one shared shape, which is Rule 4.
- - A variant keeps only
lookup_orderandfetch_accountminimal and asks for the least effort fix. The mutation teaches description expansion as the highest leverage first step. - A variant keepssummarize_reportandanalyze_reportboth described as processing a report. The mutation teaches that ambiguous contracts cause persistent misrouting to the sibling and require rewrites, not a system prompt list.
Forced selection of a named tool guarantees a mandatory first step
The tool_choice value {"type": "tool", "name": "<specific>"} forces the model to emit a tool_use block for exactly that tool in this turn. It also suppresses leading conversational text, as the response goes straight to the tool call. The typical workflow is to force the prerequisite on turn one and then return to auto for subsequent steps.
Inline requirements include exact name matching between the forced name and the definition name field. A mismatch produces a validation error. Tool-level details such as input_schema or strictness remain independent.
// turn one forces ordering regardless of user phrasing
{
"tool_choice": {"type": "tool", "name": "extract_metadata"},
"tools": [
{"name": "extract_metadata", "description": "Extracts document type, date, and identifiers"},
{"name": "lookup_citations", "description": "Looks up citations by DOI"},
{"name": "verify_doi", "description": "Verifies a DOI against the registry"}
]
}// turn two restores flexibility after the prerequisite result is in history
{
"tool_choice": {"type": "auto"},
"tools": [
{"name": "lookup_citations", "description": "Looks up citations by DOI"},
{"name": "verify_doi", "description": "Verifies a DOI against the registry"}
]
}Dependencies such as enrichment needing a document type, citation lookup needing a DOI, or triage needing an assessment must happen before downstream reasoning can succeed. Prompt instructions to always run a tool first are probabilistic and can be overridden by how the user phrases the request, while tool_choice is a configuration gate the model cannot ignore in that turn.
Boundary. Forcing is correct for a mandatory first turn. The nearby opposite case is when the workflow requires that some tool is called but the correct tool varies by input, for example unknown document types where invoice, receipt, or purchase order schemas each fit. There any is preferred per Rule 12, because forcing one named extraction tool applies the wrong schema to some documents.
Recurring specifics. Tool names that recur as forced prerequisites include extract_metadata, extract_fields, extract_invoice_data, login_check, assess_symptoms, and log_review_decision. Pattern descriptions that recur include forced on turn one then auto thereafter, sequential steps that each force their specific tool, and multi-step chains such as acknowledge then diagnose then conditionally page. Caveats that recur include that the search tool itself must not be forced away, and that forced calls can reduce visible reasoning which may affect parameter quality unless the call is truly mandatory.
Proposal. set tool_choice to any so some tool is always called.
Why it attracts. it guarantees a tool before text.
Why it fails. it leaves choice free, so lookup_citations can still be selected before extract_metadata when the dependent identifier is missing.
When it would be right. when any tool among several schemas is acceptable and text must be prevented, which is Rule 12.
Proposal. keep auto and describe ordering in the system prompt.
Why it attracts. no configuration change.
Why it fails. ordering guidance is probabilistic and has been shown to be bypassed when the enrichment request is prominent in the user message.
- - A variant forces
extract_metadataon every turn rather than just the first. The mutation teaches that forcing on every turn locks subsequent turns out of their intended enrichment calls, so the pipeline stalls. - A variant asks how to prevent a single step from issuing two tools when only extraction is wanted. The mutation teaches that forcing the named extraction tool in that turn prevents a same-turn combination like extraction plus rejection email, while later turns can be unrestricted.
The any mode guarantees a tool call while preserving choice among schemas
{"type": "any"} requires that the model emit at least one tool_use block. It does not specify which tool. This makes it the right choice when structured output is mandatory but the correct schema varies by input, for example unknown document types where invoice, receipt, and contract schemas each exist, or knowledge base versus action tools where both are plausible.
Inline values such as any, extract_invoice_data, and extract_receipt_data record that the call is mandatory while selection remains flexible. Lowering temperature or adding more prompt wording does not substitute.
{
"tool_choice": {"type": "any"},
"tools": [
{"name": "extract_invoice_data", "description": "Extracts invoice fields"},
{"name": "extract_receipt_data", "description": "Extracts receipt fields"},
{"name": "extract_contract_data", "description": "Extracts contract clauses"}
]
}auto permits a conversational answer, which breaks parsers that expect structured output. Forcing a single named tool would guarantee output but would also force the wrong schema on some inputs, which is structurally valid but semantically incorrect.
Boundary. The opposite case is when the workflow must run a specific tool no matter what, for example a mandatory audit log on every turn even when the user greets the system. There the tested material prefers forcing the named tool rather than any, because any could still select a peer tool. Another boundary is when the workflow should remain conversational on simple greetings.
Recurring specifics. Settings that recur include tool_choice values auto, any, and named tool, plus pipeline contexts such as Code review posting to issue trackers, extraction on unknown document types, and audit logging. Signals that recur include stale results or cached values when any triggers unneeded retrieval, and parser crashes when auto returns text.
Proposal. keep auto and add a system prompt such as always respond using a tool.
Why it attracts. it looks like it enforces behavior without mode change.
Why it fails. prompt compliance remains probabilistic, so occasional text responses persist and parsers still crash.
Proposal. define only one generic extraction tool to force schema convergence.
Why it attracts. it looks like it eliminates the choice.
Why it fails. it trades selection error for schema dilution, either dropping fields or carrying many nulls, and losing the tailored constraints that make each tool valuable.
- - A variant removes
anyand leavesautowhile observing occasional text responses. The mutation teaches that probabilistic prompt wording does not fix an automation guarantee. - A variant forces one named schema while document type is unknown. The mutation teaches that correctness requires flexible choice among schemas.
The auto mode preserves discretion to answer or to call and to branch conditionally
{"type": "auto"} lets the model decide on each turn whether to call a tool, to answer directly, or to chain further tool calls after a tool_result. This is the default and the correct mode for conditional workflows where action depends on prior output, such as diagnostics that decide whether to page on call, classification that decides which department handles the ticket, or orchestration that decides the next subagent after inspection.
Inline decision points such as classify_ticket, run_diagnostics, and the following auto selection illustrate that branching stays with the model's judgment. Adding a system prompt note to always consider tools can bias toward tool use without removing autonomy.
{
"tool_choice": {"type": "auto"},
"tools": [
{"name": "classify_ticket", "description": "Returns department label for a ticket"},
{"name": "acknowledge_incident", "description": "Acknowledges an incident"},
{"name": "run_diagnostics", "description": "Runs diagnostics and returns severity"}
]
}Conditional workflows require reasoning that spans results. Forcing a tool can bypass that reasoning by going straight to a tool_use block without leading text and without integrating prior output. Forcing any can trigger useless retrieval on greetings. auto preserves the model's ability to synthesize prior tool results before choosing the next step, which is why the reference material presents it as the general operation mode.
Boundary. auto is the choice when discretion matters. When every turn must produce a tool result, the opposite pattern applies and any or forced selection is preferred as in Rules 11 and 12. the tested material shows an operations assistant stalling after hello because tools were required when they were not needed, and a compliance pipeline crashing because tools were optional when they were needed. The boundary is whether conversational autonomy is part of the desired behavior in this turn.
Recurring specifics. Terms that recur include auto as default, discretion, conditional branching, and multi call sequences that remain possible because the model can return several tool_use blocks in one response and then continue after tool_result blocks. Notes that recur include that increasing tokens or lowering temperature does not change whether the mode permits a choice, and that parallel calls do not depend on a special tool_choice value.
Proposal. use any for incident response to guarantee diagnostics before paging.
Why it attracts. it guarantees a tool.
Why it fails. it does not guarantee the correct conditional judgment and may force a tool when the next step should be a conversational answer combining results.
Proposal. use repeated forced calls for each step in a multi step workflow.
Why it attracts. it looks orderly.
Why it fails. it fragments a single conversational context into canned steps and removes the model's ability to skip unnecessary actions based on prior output.
- - A variant forces
classify_ticketon every turn for audit logging. The mutation teaches that when every turn must have a specific classification tool, forcing is preferred overauto, which is the opposite boundary. - A variant triggerstranslate_to_germanon hello underautobecause the description is overbroad. The mutation teaches that description scope drivesautochoices, which links back to Rule 10, not to a mode change.
Deferred loading with a search tool enables progressive discovery for large catalogs
Every tool's full definition is still sent in the top-level tools array on each request. Marking a tool with defer_loading: true keeps its definition out of the model's initial context.
Inline controls include defer_loading, tool_search, tool_reference, and the distinction between what the API receives and what the model sees in its context window. Configuration notes that recur include leaving about six high-frequency tools eager and deferring the long tail that is used occasionally, and using the search variants that match on description and arguments.
{
"tools": [
{"name": "tool_search", "description": "Search the tool catalog by keywords over names and descriptions"},
{"name": "ticket_lookup", "description": "Lookup ticket by identifier"},
{"name": "technician_dispatch", "description": "Dispatch a technician to a site", "defer_loading": true},
{"name": "appointment_scheduling", "description": "Schedule a field appointment", "defer_loading": true},
{"name": "fleet_routing", "description": "Route a fleet vehicle", "defer_loading": true}
],
"tool_choice": {"type": "auto"}
}Monolithic loading trades a small per-task search step for a dramatically smaller startup context, which both saves tokens and improves selection accuracy on large catalogs. Large headers of three hundred or five hundred definitions consume latency and accurate selection headroom before work begins, while progressive disclosure loads only task relevant definitions.
Boundary. Deferred loading pays when the catalog is large, unpredictable, or dominated by occasional tools. When the catalog is small and every definition is used most turns, the opposite pattern in Rule 2 applies and immediate loading is preferred because search overhead is paid almost every turn while savings are small.
Recurring specifics. Sizes that trigger progressive discovery include around one hundred eighty, around two hundred fifty, around three hundred fifty, and five hundred plus tools. Two search variants recur by name as a regex variant and a retrieval variant. Details that recur include that hundreds of schemas on every request counts as avoidable context cost, and that the team must remain able to test from request through execution with deterministic escalation or result.
Proposal. remove deferred tools from the tools array and only add them after a search hit.
Why it attracts. it looks like it shrinks payload.
Why it fails. the API search index is built from definitions in the tools array, so a tool that is not in the request does not exist for that turn and cannot be discovered.
Proposal. set defer_loading: true on every tool including the search tool to maximize savings.
Why it attracts. it maximizes deferred surface.
Why it fails. a deferred search tool cannot be invoked, so discovery is blocked entirely. See Rule 16.
- - A variant with three hundred internal tools shows definitions dominating the header. The mutation teaches the two-layer model of payload versus context and the correct pattern of keeping all definitions server side while deferring the long tail. - A variant with twelve tools used almost every turn shows that enabling search adds overhead without meaningful savings, teaching the opposite boundary.
Search discoverability depends on rich names and descriptions, not on payload tricks
Search matches over names, descriptions, argument names, and argument descriptions. When deferred tools carry terse or code style identifiers such as svc_ops_417 and one line inherited registry text, searches return no relevant matches even though the capability exists, and the model reports that it lacks the tool. The fix is to enrich each deferred definition's metadata so human phrasing maps to the tool.
Inline fields such as name, description, equipment_type, and defer_loading show that discoverability lives in prose quality even though the tool is initially hidden from context.
{
"name": "check_equipment_availability",
"description": "Checks live availability of fleet equipment by type, region, and date range. Returns quantity, location, and reservation windows. Use for availability checks, not for dispatch or routing.",
"input_schema": {
"type": "object",
"properties": {
"equipment_type": {"type": "string", "description": "Category such as crane, forklift, or excavator"},
"region": {"type": "string", "description": "Depot region code"},
"date_range": {"type": "string", "description": "Inclusive start and end dates in ISO format"}
},
"required": ["equipment_type", "region"]
},
"defer_loading": true
}Progressive discovery changes where definitions live during the conversation, not how they reach the API. The full catalog is always present in the tools array for search indexing, and expansion into context happens only after a hit.
Boundary. The opposite case is when payload size itself is the concern. Truncating or omitting definitions from the request does not preserve discoverability, as covered in Rule 14, and diluting descriptions to save tokens is counterproductive. The boundary case where description enrichment is not the fix is when the tool is already eager and misrouting stems from pairwise overlap rather than discoverability. That case reverts to Rule 10 rather than further enrichment for search.
Recurring specifics. Failure symptoms that recur include searches returning nothing and the model claiming a capability does not exist. Metadata fields that recur as search surface include tool names, descriptions, argument names, and argument descriptions. Anti-patterns that recur include opaque identifiers, one line inherited descriptions, and attempts to shrink payload by withholding deferred definitions.
Proposal. stop sending deferred definitions in the tools array and submit only names.
Why it attracts. it looks like payload relief.
Why it fails. there is then nothing for search to index, and expansion has no schema to load.
Proposal. remove defer_loading from all two hundred fifty tools so every definition loads upfront.
Why it attracts. it restores visibility.
Why it fails. it reverts to monolithic cost and selection noise that Rule 14 was meant to solve.
- - A variant keeps two hundred fifty tools deferred with terse names and one-line descriptions and sees discovery failures. The mutation teaches rewriting all three layers of metadata before changing the deferral pattern. - A variant proposes loading all definitions to fix discovery. The mutation teaches that metadata quality is the cheaper correct lever.
At least one discovery path must remain non-deferred or nothing is discoverable
Deferred loading governs visibility in context, not existence in the tools array. At least one tool must remain visible in context at the start so the model can invoke discovery. In practice that tool is the search tool itself. Once discovery succeeds, the matched deferred definition is appended inline as a tool_reference block without invalidating the cached prefix. Deferred definitions remain searchable precisely because their full definitions are present server side.
Inline elements such as tool_search, defer_loading, and tool_reference encode the invariant: the discovery capability cannot itself be deferred.
{
"tools": [
{"name": "tool_search", "description": "Search deferred tools by name and description"},
{"name": "contract_search", "description": "Search contracts by scope and clause type", "defer_loading": true},
{"name": "fleet_maintenance", "description": "File a fleet maintenance request", "defer_loading": true}
],
"tool_choice": {"type": "auto"}
}Deferral without an eager discovery entry point leaves the model with no way to list or search. The configuration is coherent only if the entry point is present in context. the tested material calls deferring every tool including the search tool an explicit anti-pattern. The same invariant rules out request designs that try to maximize savings by hiding every definition. Up to hundreds of deferred tools can be hidden, but the search affordance must remain visible.
Boundary. The opposite case is when no discovery is needed at all because the catalog is small and every tool is routinely needed. There the correct choice is to leave every definition eager, which is Rule 2's opposite boundary, and discovery overhead is avoided entirely.
Recurring specifics. Controls that recur include defer_loading, tool_search, tool_reference blocks appended inline, and prompt caching that is preserved because discovery appends after the cached prefix. Anti-pattern phrasing that recurs includes defer every tool including the search capability itself and omitting deferred definitions from the request to shrink payload, both called out as failures.
Proposal. defer every tool including the search tool to leave zero definitions in initial context.
Why it attracts. maximal savings.
Why it fails. no discovery path remains and the catalog becomes unreachable.
Proposal. omit deferred definitions from the request and inject them after discovery.
Why it attracts. it looks like progressive payload.
Why it fails. search is built from definitions in the tools array. Without them the catalog is empty.
- - A variant defers every tool and asks why discovery never happens. The mutation teaches the eager entry point invariant. - A variant withholds payload and asks why hits never occur. The mutation teaches the payload versus context layer distinction.
Generic or multipurpose tools blur boundaries and should be replaced by purpose-specific tools
A single tool that handles many distinct operations through a free text instruction or a multipurpose operation argument hides responsibilities behind one name. the tested material shows analyze_document with a free text instruction parameter producing inconsistent narrative versus table outputs, manage_account handling profile, password, subscription, and deactivation under one name with frequent argument conflation, and process_file handling codec conversion, thumbnail extraction, audio normalization, and container remuxing under one description.
Inline members such as data_types, focus_area, summary_length, and returned shapes like data_points illustrate that each purpose has a distinct contract rather than a shared free text field.
{
"name": "extract_data_points",
"description": "Extracts structured data points for named types such as financial metrics or percentages. Returns {data_points: [{value, unit, context, location}]}.",
"input_schema": {
"type": "object",
"properties": {
"document": {"type": "string"},
"data_types": {"type": "array", "items": {"type": "string", "enum": ["financial_metrics", "percentages", "dates"]}}
},
"required": ["document", "data_types"]
}
}{
"name": "summarize_content",
"description": "Summarizes content for a focus area with a bounded length. Returns {summary, key_points}.",
"input_schema": {
"type": "object",
"properties": {
"document": {"type": "string"},
"focus_area": {"type": "string"},
"summary_length": {"type": "string", "enum": ["concise", "detailed"]}
},
"required": ["document", "focus_area"]
}
}Purpose-specific contracts give the model a structured slot for intent instead of an ambiguous natural language phrase, and they enforce output shape through schema rather than leaving format to interpretation. Generic tools also violate the core lesson from Rule 4's legitimate consolidation: sharing a true common shape is consolidatable, but distinct operations with different side effects or output shapes are not.
Boundary. The opposite case is genuine homogeneity covered in Rule 4, where many variants share inputs and outputs and the parameter is a true enum rather than a catch-all operation string. For example nineteen transformations with dataset in and dataset out are consolidatable, while user_ops with get_profile versus deactivate_account or analyze_codebase with location versus summarization versus tracing are not.
Recurring specifics. Split candidates that recur include analyze_document into extract_data_points, summarize_content, verify_claim_against_source; manage_account into update_profile, reset_password, change_subscription, deactivate_account; and process_file into convert_codec, extract_thumbnail, normalize_audio, remux_container. Single tool failures that recur include parameter conflation between update_email and reset_password, and misrouting between description variants that differ only by free text instructions.
Proposal. add an enum constraint on the operation parameter inside the generic tool.
Why it attracts. it looks like scoping.
Why it fails. it keeps unrelated side effects behind one name, so authorization and least privilege remain blurred and misrouting persists within the enum.
When it would be right. when the operation set shares a true common shape, which is Rule 4, not this case.
Proposal. add a system prompt with a one line description per operation.
Why it attracts. it teaches boundaries outside the schema.
Why it fails. it leaves contracts implicit and selection probabilistic, while split schemas make contracts explicit and enforceable.
- - A variant keeps
process_filewith a vague permission isError contract and asks whether fixing the error contract fixes misrouting. The mutation teaches that error handling fires after the call and cannot prevent choosing the wrong operation. The split into purpose-specific tools is required for prevention. - A variant keeps one genericanalyze_codebasefor location, summarization, and tracing and sees ambiguous requests mix behaviors. The mutation teaches that splitting intofind_definition,summarize_file,trace_call_chainremoves ambiguity at the description layer.
Scoped cross-role tools handle high-frequency simple cases locally without coordinator round trips
When an agent routinely needs a capability that nominally belongs to another role, the cheapest correct design is to give the needing agent a constrained version of that capability directly. Routing every verification or lookup through the coordinator adds two to three hops and large latency, especially when around eighty five percent of requests are simple single source lookups.
Inline elements such as verify_fact, claim, source_hint, and the description's stated escalation path define the scope. Tooling notes that recur include that read-only or narrowly scoped access is the exception, not the norm, and that sensitive writes remain coordinator mediated.
{
"name": "verify_fact",
"description": "Quick lookup for a single simple fact from one source. Results are constrained to a short passage with provenance. For multi-source or cross-referenced verification, escalate to the coordinator.",
"input_schema": {
"type": "object",
"properties": {
"claim": {"type": "string"},
"source_hint": {"type": "string", "description": "Single source identifier, for example docs or tickets"}
},
"required": ["claim"]
}
}The synthesis agent's job is to combine upstream findings. If every quick brand glossary lookup or date check requires a hop to the coordinator, then to the search agent, then back, simple turns pay the cost of the complex pipeline.
Boundary. Scoping is correct for a small number of high-frequency simple lookups with limited blast radius. The opposite principle dominates when the capability is sensitive or irreversible, or when the need is genuinely cross-concern. For example publish_campaign and adjust_ad_budget must remain outside copy and localization agents even at high frequency, and true cross-domain tickets are handled by a coordinator that delegates to the smallest necessary set after decomposition.
Recurring specifics. Signals that recur include eighty five percent simple lookups, two to three round trips per request, and forty percent latency increase when routing through the coordinator. Tool names that recur as scoped cross-role examples include verify_fact, check_brand_terms, and simple lookup_user variants that stay narrow. Description guidance that recurs states to explicitly limit the tool to simple lookups and to name escalation conditions.
Proposal. keep every tool in every agent so each agent is self sufficient.
Why it attracts. it avoids coordination.
Why it fails. it recreates the overloaded set from Rules 1 and 3, degrades selection, and tempts agents into off-script work.
Proposal. never share any tool and route every lookup through the coordinator.
Why it attracts. it preserves purity.
Why it fails. it wastes round trips on the dominant simple case.
- - A variant routes simple lookups through the coordinator and sees a forty percent latency penalty. The mutation teaches the scoped local tool. - A variant gives the synthesis agent a narrow lookup as the only exception and preserves sound isolation. The mutation teaches that one constrained addition can be sound while three broad additions are unsound.
Content catalogs belong in resources, not in exploratory tool calls
MCP distinguishes primitives. Tools are actions that the model can invoke. Resources are readable content and metadata the model can inspect, such as issue summaries, documentation hierarchy, and database schemas, and prompt templates are a separate primitive. When agents lack visibility into what content each server contains, they issue exploratory calls such as listing every project or fetching every schema before they can make a targeted call. Exposing the catalog as a resource eliminates that exploration.
Inline artifacts such as resource://, hierarchy, and schema_catalog encode visibility without action. After reading a resource, a single targeted tool call such as run_query or get_page can be issued with correct parameters.
// resource advertisement, not a tool call, visible at connection time
{
"resources": [
{"uri": "resource://issue_tracker/issue_categories", "description": "Catalog of issue labels and recent issue summaries"},
{"uri": "resource://docs_wiki/hierarchy", "description": "Documentation hierarchy by space and page"},
{"uri": "resource://db_explorer/schema_catalog", "description": "Database and table catalog with column types"}
]
}Exploratory tool traffic exhausts context before useful work completes, especially for cross-system questions such as which database tables are affected by an issue tracker change that span multiple servers. Turning eight to ten exploratory calls into one or two reads reduces turns, preserves tokens for synthesis, and removes the guesswork of which project or table to try.
Boundary. Resources are appropriate for static or slowly updated metadata such as catalogs and hierarchies. They are not the surface for actions, and converting every resource access into a tool such as discover_everything keeps the round trip the problem was meant to eliminate. The nearby opposite is when the workflow truly needs live computation or parameterized action, for example run_query against a specific predicate.
Recurring specifics. Server counts that recur include three servers for issues, docs, and databases, and ten plus servers in initialization pressure. Resource examples that recur include issue summaries and categories, documentation hierarchy and spaces, and database schema catalogs. Framing notes that recur include read once, inform many decisions versus consume context with each call.
Proposal. consolidate all three servers into one unified server.
Why it attracts. it appears to reduce integration surface.
Why it fails. it does not provide structural visibility. Nine combined tools still require exploratory calls to discover which issues, pages, or databases exist.
Proposal. add a prepare_investigation tool on each server.
Why it attracts. it promises relevant summaries.
Why it fails. it still requires one tool call per server as preparation and adds a natural language interpretation layer that can fail, while resources provide visibility with zero tool calls.
- - A variant gives the agent many resources in a flat list and sees wrong URI requests. The mutation teaches hierarchical organization, clear naming conventions, and resource templates with parameterized access such as
analytics/{metric}/{period}to collapse the visible set. - A variant shows a catalog that updates infrequently and asks whether to expose it as a tool. The mutation teaches that slowly changing content is suited to resource access by URI.
Configuration defines what is possible, prompt shapes when it is chosen, dependency gates enforce order
Three layers cooperate. Configuration such as allowedTools, mcp_toolset allowlists, and the tools array defines which capabilities exist in this request.
Inline elements such as verification_token, verify_identity, get_account_details, and extract_metadata show that the gate is typed and enforced, not merely suggested. In SDK style the same gate can be expressed as staged allowedTools where later tools are not present until the prerequisite result exists.
{
"tools": [
{"name": "verify_identity", "description": "Verifies the user and returns a verification_token"},
{"name": "get_account_details", "description": "Returns account detail. Requires verification_token from verify_identity"}
],
"tool_choice": {"type": "auto"}
}{
"tool_choice": {"type": "tool", "name": "extract_metadata"},
"tools": [
{"name": "extract_metadata", "description": "Mandatory first step for all enrichment flows"}
]
}Prompts alone cannot guarantee that a step will happen because they are advisory. Configuration alone cannot guarantee that steps happen in the right order because it defines presence, not sequencing. Dependency gates bridge the gap by making the next step technically impossible until the prior step produces the needed value.
Boundary. The nearby opposite case is when strict ordering is not needed and conditional branching with auto is preferred per Rule 13. For example incident response that acknowledges and diagnoses before conditionally paging must remain flexible, and forcing a single tool every turn breaks that branching.
Recurring specifics. Gate patterns that recur include a verification_token prerequisite, a doi dependency between extraction and enrichment, and a Task or Agent delegation gate where the coordinator cannot emit delegation blocks until that tool is present in allowedTools. Inspection artifacts that recur include transcripts that narrate intent to delegate but contain no Task blocks, which indicates a permission gap rather than a prompt quality problem, and error contracts that conflate empty results with access failures, which must be typed separately.
Proposal. lengthen the description of verify_identity or mark it urgent.
Why it attracts. it looks like stronger steering.
Why it fails. it improves selection but does not enforce a mandatory prerequisite. The skip remains possible.
Proposal. add a negative constraint such as never call details before verification.
Why it attracts. it looks like prohibition.
Why it fails. it is still probabilistic and can be bypassed in edge cases, while a token gate cannot.
- - A variant allows the model to choose
verify_identityfreely underautoand sees occasional skips. The mutation teaches the hard dependency gate. - A variant forces the prerequisite on every turn and then cannot reach enrichment. The mutation teaches the staged pattern of forcing on turn one only.
Distinctions that decide answers
| This | Not this | How to tell them apart |
|---|---|---|
| Scoping by role at four to five tools per agent | Packing all tools onto one agent | Scoping keeps each turn's choice among four or five role specific tools. Packing eighteen tools onto one agent raises decision complexity and misrouting irrespective of description quality. |
| Consolidating near duplicate tool variants into one parameterised tool | Splitting by role across agents | The consolidated form replaces nineteen tools that share a shape with one enum driven tool. Splitting separates tools that do different jobs, such as query versus transform, into different agents. |
| Scoped cross role verify_fact | Routing every verification through the coordinator | The scoped tool handles the eighty five percent simple cases locally in one hop. Coordinator routing for every verification adds two to three round trips regardless of complexity. |
| tool_choice auto | tool_choice any | auto leaves the model free to return text without a tool call, suitable for general turns. any forces at least one tool call, suitable where structured output from one of several schemas must be produced. |
| tool_choice any | Forced tool type tool with name | any forces some tool but lets the model choose which. Forced named selection enforces that a specific mandatory first step such as extract_metadata executes on this turn. |
Traps
The tempting answer. Centralise all fact verification at the coordinator because centralisation feels cleaner and more observable.
Why it fails. Centralisation for the simple single site share adds two to three hops per task and inflates latency by roughly forty percent for that share, where measurement shows eighty five percent of checks are simple lookups that a scoped verify_fact handles locally while complex checks remain centralised.
What is correct. Add a scoped verify_fact to synthesis for the simple path with explicit escalation language for multi source verification, keeping complex verification on the coordinator pipeline.
The tempting answer. Leave the turn on auto because auto feels safe and non coercive.
Why it fails. auto lets the model decide not to call a tool even when a structured extraction step is mandatory, so pipelines that require structured output must enforce it via any where the tool identity can vary or via forced named selection where the step is fixed.
What is correct. Use any to guarantee some structured call or forced type tool with name for a mandatory first step, reserving auto for conversational turns where a tool call is legitimately optional.
The tempting answer. Improve wording alone for a single agent with eighteen tools because the descriptions read well individually and tool count is only a number.
Why it fails. Decision complexity scales with tool count in the prompt, and the four to five per agent guidance is the tested anchor, so description work on a bloated agent leaves misrouting in place where distribution is the correct lever.
What is correct. Distribute via role scoping or enum consolidation before investing further in prose, measuring selection on the merged flat list rather than per server.
The tempting answer. Keep the generic fetch_url because flexibility feels valuable and avoids maintaining a narrower variant.
Why it fails. That flexibility enables misuse, muddies the tool's purpose for selection, and creates unintended side effects such as fetching non document resources, while the constrained load_document variant implements least privilege and makes description sharply indicative.
What is correct. Replace generic fetch with load_document that validates document URL shape and allowlist, keeping the narrower contract on the allowlist.
The tempting answer. Create a second MCP server and place half the tools there, assuming the model will see two smaller lists.
Why it fails. Server boundaries are not visible to the model, which receives one flat list from every connected server, so the overload persists after the move and only per agent allowlist trimming and consolidation actually reduce selection pressure.
What is correct. Evaluate overload on the merged prompt and apply scoping or enum consolidation rather than a server topology change.
MCP Server Integration
Scope MCP servers correctly, keep the shared file credential free with expansion, prefer community servers for standard systems, and make MCP tools selectable via enriched descriptions and Resources.
What you need to know
The Model Context Protocol, or MCP, is the standard open protocol that defines how hosts such as Claude Desktop and Claude Code connect to external systems such as databases, search APIs, version control hosts, and internal services through Servers that expose Resources, Tools, and Prompts over a JSON-RPC 2.0 transport. Because MCP decouples the language provider's request response so that tool discovery, invocation, and management happen over persistent connections with initialization, negotiated capabilities, and ordered shutdown, configuration scoping determines whether a team shares a consistent toolset or drifts into configuration chaos.
The scoping hierarchy is two tiered, and the line between the tiers is where most setup problems start. Project level configuration lives in dot mcp dot json at the repository root, is version controlled, and is shared automatically with every teammate who clones or pulls the repository. It is used for servers the entire team needs, such as Jira for issue tracking, GitHub for source control, or internal connectors that represent the shared workflow. A representative entry declares an mcpServers object mapping names to server specs with command, args, and env, for example command npx, args with dash y and at modelcontextprotocol slash server-github, and an env block that references a token variable rather than containing it. User level configuration lives in tilde slash dot claude dot json in the home directory, is personal, is not version controlled, and is not shared with teammates, and is used for experimental servers or personal integrations that are being evaluated before being proposed to the team. At connection time every tool from every configured server on either tier is discovered and available simultaneously with no manual activation step, and if a server is configured and reachable its tools appear in the agent's toolkit.
Credential handling follows from that bridge. The shareable file must not contain secrets, and MCP's variable expansion for dollar brace VARIABLE_NAME in dot mcp dot json keeps the shared file safe to commit because each developer sets their own values locally in a shell profile, dot env file, or secret manager, and rotation then requires no file change and leaves no secret in the repository history. A common vendor of this failure is a dot mcp dot json committed with the literal GITHUB_TOKEN value, which exposes a secret to every clone and every continuous integration log, whereas the correct file contains the reference syntax and every member authenticates with their own locally scoped token. The protocol itself does not define credential management beyond transport, so host level expansion plus local secret storage is the convention that makes shared configuration safe.
MCP Resources are the catalogue complement to Tools, and choosing to expose them well reduces wasted exploratory calls. Where a Tool lets the model take an action that may mutate the world, a Resource exposes read only, host controlled data such as issue summaries with titles and statuses, documentation hierarchies, or database schemas with table names, column types, and relationships, which the host can fetch and inject as context before the model is asked to choose Tools. Without that catalogue the model must discover what exists by calling a sequence like list_tables then describe_table for every table, burning turns purely to get bearings, while with an exposed schema Resource it knows immediately and can call the substantive Tools correctly. The read only distinction matters for the exam. Resources are not write capable, so examining a catalogue cannot create the side effects a misuse of a Tool call would.
Build versus use is the integration choice the exam tests most directly in this task, and it is consistently framed as a pragmatic first step before bespoke work. For standard integrations such as Jira, GitHub, Slack, Linear, or Notion, mature community MCP Servers already exist, are maintained and tested by the community, receive updates without local build cost, and handle the common use cases correctly. The correct response when a team needs one of those systems is therefore to evaluate existing community Servers first and only build custom when there are team specific workflows, custom business logic at the tool layer, or proprietary internal systems with no community equivalent. Building a custom Jira integration as the first response when a standard server would have sufficed trades maintained code for a one off fork that will diverge over time, which the exam consistently marks as the wrong priority.
Tool selection quality inside MCP also depends on the model being able to read what an MCP Tool is capable of at selection time, not merely that it exists. When two tools share a name stem or an overlapping capability, the agent may prefer a built in tool such as Grep over a semantically richer MCP search, because the built in tool's description is detailed while the MCP entry is sparse, even when the MCP implementation is more capable for the task at hand. The correction is to enhance MCP Tool descriptions to the same five part standard that governs API tools generally, stating the capability, the expected inputs, the returned shape and example outputs, edge cases, and explicit when to use this tool versus built in or sibling MCP tools, for instance calling out that AST aware indexing is more accurate than text search for intent, so the model's choice under tool_choice auto is guided by substance rather than by length of description. This pattern illustrates how MCP design feeds back into tool design, where the same description standard that removed ambiguity for get_customer versus lookup_order also determines whether an MCP catalogue is actually chosen in practice.
Scoping hierarchy and discovery without manual activation
Project dot mcp dot json at the repository root is version controlled and shared, carrying the mcpServers map for team wide dependencies. User tilde slash dot claude dot json is personal and never shared, carrying experimental or personal servers that are evaluated before being proposed to the team. The runtime property that bridges both tiers is that at connection time every tool from every configured server on either tier is discovered simultaneously with no manual activation step. If a server is configured and reachable its tools appear, which is why placing shared servers in the personal file or requiring a local activation command is the incorrect model of how discovery works.
That simultaneity interacts with distribution. Because the client merges all tools from all servers into one flat list, a team that fixes overload by moving tools to a second server has not reduced the model's option count. The correct fix remains per agent allowlist trimming and enum consolidation, while scoping decisions remain about sharing versus personal experimentation rather than about choice complexity.
Credential hygiene via expansion
The shareable dot mcp dot json must not contain literal secrets. MCP's variable expansion for dollar brace VARIABLE_NAME in that file keeps the shared file credential free because each developer sets values locally in a shell profile, dot env file, or secret manager, and rotation then requires no file change and leaves no secret in repository history. A common failure is a dot mcp dot json committed with the literal GITHUB_TOKEN value, which exposes a secret to every clone and every build log. The correct file contains the reference syntax and every member authenticates with their own locally scoped token, with continuous integration injecting the same variable from the secret manager.
This hygiene is convention rather than protocol, because MCP itself does not define credential management beyond transport. Host level expansion plus local storage is the boundary that makes shared configuration safe and reviewable, and it is the detail the exam probes when a file diff shows a literal token versus a reference.
Resources as catalogues that eliminate exploratory chains
Resources expose read only, host controlled data such as issue summaries, documentation hierarchies, or database schemas with table names, column types, and relationships, which the host can fetch and inject as context before the model chooses Tools. Without that catalogue the model must discover what exists by calling a sequence like list_tables then describe_table for every table, burning turns purely to get bearings, while with an exposed schema Resource it knows immediately and can call the substantive Tools correctly.
The read only distinction matters for exam reasoning. Resources are not write capable, so examining a catalogue cannot create the side effects a misuse of a Tool call would. Rendering schema or issue lists as Resources under a db colon slash slash style URI with declared mimeType therefore moves orientation cost from turns to one context injection, preserving budget for the substantive workflow.
Build versus evaluate and description enrichment for selection
For standard external systems such as Jira, GitHub, Slack, Linear, or Notion, mature community MCP Servers already exist, are maintained and tested, receive updates without local build cost, and handle the common use cases correctly. The correct response when a team needs one of those systems is to evaluate existing community Servers first and only build custom when there are team specific workflows, custom business logic at the tool layer, or proprietary internal systems with no community equivalent. Proposing custom Jira work as the opening move trades maintained code for a one off fork that will diverge, which the exam marks as the wrong priority.
Even after a server is chosen, selection quality depends on how selectable its tools appear. When a sparse MCP entry named search_codebase carries the description Searches code, the agent prefers Grep over that server even where semantic search would be better, because the built in tool has richer text. The correction is to enhance MCP Tool descriptions to the same five part standard, stating the capability as AST aware semantic search, the returned shape as functions and methods with file path and line number, and the scope advice such as Use this instead of Grep when searching by intent rather than by exact string, after which selection stabilises.
Authoritative mechanism reference
The sections below expand the tested rules into the complete documented surface: ownership, lifecycle, version differences, production boundaries, and adjacent mechanisms.
Mechanism reference: Surface separation: protocol, Claude Code client, API connector
Candidates lose points when they carry a rule from one surface to another. This section keeps the three surfaces explicitly separate.
The protocol specification is the open standard that defines hosts, clients, and servers; three primitives; JSON-RPC 2.0 envelopes; capability negotiation; and two standard transports. It is host-agnostic and versioned by date strings such as 2025-06-18. It does not define where files live, how credentials are stored, or which JSON keys a particular product uses for configuration.
The Claude Code client is one host implementation. It defines the configuration files .mcp.json and ~/.claude.json, the mcpServers object, the per-server fields command, args, env, url, and headers, environment variable expansion in command, args, env, url, and headers, the three logical scopes stored in two physical files, the precedence rule that selects one entry when names collide, lifecycle management for stdio and Streamable HTTP servers, tool discovery via tools/list, and aggregation into a unified toolkit. It also defines the ownership map for security at the process boundary.
The Messages API connector is a platform feature that removes the client layer. The application does not run an MCP client at all. Instead it declares mcp_servers and mcp_toolset inside the API request, and the platform's infrastructure acts as the client, performs discovery and calls, and returns tool_result blocks inline with normal model turns. It supports remote HTTP servers only, only tools, and requires a publicly reachable HTTPS endpoint. Local stdio servers, prompts, and resources are outside its scope and require a client-side host such as Claude Code or the Agents SDK.
A question that asks about mcp_servers is an API connector question. A question that asks about .mcp.json versus ~/.claude.json is a Claude Code question. A question that asks about tools/list versus resources/list or initialize is a protocol question. Naming the surface is the fastest way to eliminate distractors that borrow a detail from the wrong surface.
Mechanism reference: Protocol foundations: JSON-RPC 2.0, lifecycle, capabilities
Every MCP message is a JSON-RPC 2.0 payload with three shapes. A request carries jsonrpc with value 2.0, an id, a method, and optional params. A response carries the same id and either result or error. A notification carries method and optional params with no id and expects no response. Multiplexing is by id, so multiple requests may be in flight.
The connection lifecycle is four phases in order. The client sends initialize with protocolVersion, capabilities, and clientInfo. The server answers with its protocolVersion, capabilities, and serverInfo. The client sends the initialized notification to enter operational mode. Operational messages such as tools/list, tools/call, resources/list, resources/read, prompts/list, and prompts/get are only valid after initialized. Shutdown is a shutdown request followed by transport close. A server that accepts operational requests before initialized is non-compliant.
Capability negotiation is the intersection of what the client offers and what the server advertises. A client may advertise roots and sampling, a server may advertise resources with subscribe and listChanged, tools with listChanged, and prompts. Both sides must not invoke operations outside the negotiated set. Declaring tools: {} without a handler violates the contract and produces missing-tool symptoms identical to a server that failed to start, but the cause is negotiation, not transport.
The current protocol version strings are date-based. The reference version used in our lessons for examples is 2025-06-18. Version mismatch during initialize must either negotiate a mutually compatible version or fail the connection. A forward-looking note documents a stateless revision that replaces the explicit handshake with per-request _meta and adds server/discover, but current implementations still use the handshake described here.
Errors use the JSON-RPC error object with numeric code, message, and optional data. Standard codes include -32700 parse error, -32600 invalid request, -32601 method not found, and -32603 internal error. Servers should use -32000 to -32099 for application tool failures, and callers may return a successful payload with isError: true to carry partial data alongside the error.
Pagination uses cursor-based nextCursor. Methods tools/list, resources/list, and prompts/list may return nextCursor, and the client passes cursor on the next request. Cursors are opaque, server-defined, and may expire. A forward-looking note adds ttlMs and cacheScope to list responses for TTL-based caching, but that is not yet in the operational path most candidates will encounter.
Mechanism reference: The three primitives and who controls each
MCP defines exactly three primitives, each controlled by a different party. Understanding control is the single most important primitive concept for correct resource versus tool decisions.
Resources are application-controlled. The host fetches them and injects them as context before the model generates. They are read-only, addressed by URI, and have no side effects. They are analogous to GET. The model never creates or mutates them.
Tools are model-controlled. The model decides during generation to emit a tool_use block, the host routes that to the owning server via tools/call, and the result returns as a tool_result block in the next user turn. Tools are analogous to POST and are the only primitive with expected side effects.
Prompts are user-controlled. The user selects a prompt template with arguments via prompts/get, and the server returns pre-built messages. Prompts are macros for repetitive workflows and are how a server can standardize team procedures into shareable commands.
The decision heuristic our lessons teach is precise. When the application knows up front what context the model needs, expose it as a resource. When the model must decide what to fetch based on conversation content, expose it as a tool. When the user repeatedly needs the same workflow with varying parameters, expose it as a prompt. A database schema is a resource, a parameterized query is a tool, a reusable code-review workflow is a prompt.
Mechanism reference: Transports: stdio and Streamable HTTP
The specification defines exactly two standard transports. An older standalone HTTP plus SSE transport existed in protocol version 2024-11-05 and is now deprecated, superseded by Streamable HTTP, which carries a single POST endpoint and may stream the response as text/event-stream when the server needs to send progress before the final result. SSE is therefore not a transport choice to make for new servers, it is the streaming detail inside Streamable HTTP.
Stdio runs the server as a child process of the host. The host writes JSON-RPC requests to the server's stdin line-delimited by newline and reads responses from stdout. Stderr carries diagnostics and never participates in the protocol. Characteristics that matter on the exam: zero network configuration, no ports or TLS, inherited environment including variables expanded from env, automatic security isolation because no network peer can reach the pipe, single client only, stateful process lifetime tied to the host session, and fast startup expectation.
Streamable HTTP runs the server as an independent HTTP service. The client sends JSON-RPC requests as HTTP POST to a single MCP endpoint, and the server responds either with a single JSON body or with an SSE-framed stream when it has multiple messages for one request. Characteristics that matter on the exam: remote reachability across machines and networks, support for multiple simultaneous clients, stateless at the HTTP level so any instance can handle any request without affinity, standard HTTP authentication, standard observability, session management via Mcp-Session-Id and stream resumption via Last-Event-ID, requirement for TLS in production, and two routable headers Mcp-Method and Mcp-Name for gateway routing without body inspection.
The correct selection rule follows from the deployment shape. Local to the current machine plus single client plus tightly coupled lifecycle is stdio. Shared across developers, multi-client, horizontally scaled, or behind a team API gateway is Streamable HTTP. Using stdio for a shared team service is an anti-pattern, and choosing standalone SSE for a new remote server is choosing a deprecated path.
The legacy two-endpoint SSE design used a GET to /events for a persistent SSE stream plus a separate POST to /messages. The current design collapses both onto one POST. Clients that still support the legacy path probe Streamable HTTP first and fall back to the older flow, but new servers should not expose the older pair.
A key design guarantee is transport independence. The same tool, resource, and prompt handlers can be bound to stdio for local testing and to Streamable HTTP for production from the same codebase, with the transport selected at startup. Handlers must not branch on transport.
Mechanism reference: Claude Code configuration: files, scopes, keys, and precedence
Claude Code reads MCP server definitions from configuration that declares a single logical object name mcpServers whose keys are server names and whose values describe how to reach each server. Verification status of each field is noted explicitly.
Verified keys and shapes that appear in lessons and in product documentation, and may be asserted without qualification:
- Top-level key
mcpServersis an object mapping server name strings to server entry objects. - For stdio entries,
commandis a string naming the executable to spawn,argsis an array of strings, andenvis an object mapping environment variable names to string values. All three are verified. - For Streamable HTTP entries,
urlis a string with an HTTPS URL, andheadersis an object mapping header names to string values. Both are verified. - Environment variable expansion
${VAR}and${VAR:-default}is verified as supported incommand,args,env,url, andheaders. - The configuration is resolved at session start for all servers and the resulting tools are aggregated. The aggregated toolkit is presented to the model as one flat set.
Claimed scope and storage rules that are verified at the conceptual level by our lessons but whose physical-file implementation is refined by current product behavior:
- Project scope is the project root file
.mcp.json, version-controlled and shared via the repository. This is verified. - Local scope and user scope are both stored in
~/.claude.jsonunder different sections. Local is keyed to the current project's path and loads only for that project. User is keyed for all projects and loads everywhere. The popular two-file description.mcp.jsonversus~/.claude.jsonremains exam-correct as a conceptual model, but an answer that treats local and user as separate physical files would be imprecise against current product behavior. - Plugin-provided servers and hosted connectors are additional sources beyond the three scopes, and participate in precedence.
Verified precedence when the same server name appears in more than one source: local first, then project, then user, then plugin-provided servers, then hosted connectors. The winning entry is used as a whole, fields are not merged across sources.
Per-server tool visibility controls are verified in their intent but partly not independently confirmed in exact key spelling because lessons describe the capability in prose without a normative TypeScript interface in scope. Specifically, the ability to keep a server configured but inactive and the ability to restrict which tools from a server are visible to the model via a qualified allow-list such as mcp__server__tool are taught as real controls, but the precise configuration keys disabled or allowedTools as written in community material are not independently confirmed against the two resolution URLs above and should be asserted only as the described capability, not as the exact key name, unless confirmed by fetching the specific product reference in the exam environment. This marking satisfies the requirement to verify each key before asserting it.
Behavior verified about tool aggregation:
- All configured and reachable servers have their tools surfaced together. No manual per-turn activation exists.
- Tool names are namespaced by server in allow-list contexts as
mcp__<serverName>__<toolName>, which keeps identically named tools from colliding. The underlying conflict rule our lessons teach is to prefix tool names with the server domain to prevent confusion. - Built-in Claude Code tools such as
Read,Write,Edit,Bash,Grep, andGlobremain alongside MCP tools in the combined toolkit, and the model sees both sets together.
Lifecycle verified by transport:
- Stdio servers are spawned on session start and terminated when the host exits.
- Streamable HTTP servers are connected on demand to an already running endpoint, and the connection is closed while the service keeps running.
- A missing
commandon the system path for a stdio entry causes spawn to fail and the server never connects, so its tools never appear. This is a hard startup failure, not a silent degradation, and no fallback to HTTP occurs. - Horizontal scaling for Streamable HTTP is stateless. Multiple instances behind a load balancer are functionally equivalent, and externalized state is required if the server needs shared caches.
Environment handling details that matter for correctness:
- Expansion runs at spawn or request time, never at file write time, so the value never lives in configuration history.
- A missing variable expands to an empty string or the declared default from
${VAR:-default}, and an auth check then fails with a clear error. - Secrets should be supplied from shell environment,
.envignored by version control, or a system keychain or vault, never inline in.mcp.json. .envmust be added to.gitignorebefore any secret is ever committed, because history preserves the earlier commit.
Mechanism reference: API connector surface: mcp_servers and mcp_toolset
The Messages API connector is configured inside the API request itself, not in any file. Each entry describes a remote server, and a separate tool declaration enables its tools. This is the most confused boundary on the exam, so the verified shape is given explicitly with markings for unconfirmed details.
Verified core shape against lessons and the protocol note in our architecture lesson:
- The request carries an
mcp_serversarray. Each element hastypewith valueurl, anamethat uniquely identifies the server within the request, and aurlwith a publicly reachable HTTPS URL. This is verified. - The request's
toolsarray carries a correspondingmcp_toolsetentry that references the server bymcp_server_nameand is what surfaces that server's tools. Each server needs exactly one toolset entry to expose its tools. Without it, declaration alone does not enable tools. This is verified. - Individual tools can be allowlisted or denylisted via the toolset configuration. The exact field names for that allow-list inside the toolset are not independently confirmed beyond the described capability until the specific API reference is fetched in the exam environment.
- An OAuth bearer token can be supplied for a server as
authorization_tokenwith valueBearer .... The exact field name is marked not independently confirmed against the two resolution URLs cited, because our lessons describe the capability but the API reference uses evolving beta header naming. The ownership rule is verified: the application obtains and refreshes the token, the platform uses it on the MCP connection. - The connector targets remote HTTP servers only. A local stdio command cannot be attached through this surface because there is no HTTP endpoint for the platform to connect to.
- The connector supports tools only. Prompts and resources are not surfaced through this path and require a client-side host that implements those primitives.
- A beta header is required in some API forms to enable the connector. The exact header name is not independently confirmed in this file because it is a forward-looking detail documented in the live API reference rather than in the two cited product URLs.
The consequence of these verified constraints is a clean selection rule. When the scenario is a serverless product calling a partner API over the network, the connector is the right fit. When the scenario involves a local development server, a filesystem resource, a prompt workflow, or fine-grained connection control, a client-side MCP client such as Claude Code or the Agents SDK is required.
Mechanism reference: Resources: URI, templates, mimeType, subscriptions, pagination
Every resource is identified by a URI following scheme://path. The scheme is server-defined and acts as a handler selector. Common schemes in our lessons are file://, db://, docs://, and api://, but a server may define any scheme that fits its domain. Consistency within a server matters more than the literal scheme string.
Resource metadata is advertised via resources/list and carries uri, name, description, and mimeType. The two required server handlers are resources/list and resources/read. Without resources/list, discovery is broken. Without authorization checks in resources/read, the server exposes an open read endpoint.
Content typing uses MIME. text/plain for unstructured text, text/markdown for documentation, application/json for structured data, text/csv for tabular data, image/png and image/jpeg for multimodal visual context, and application/octet-stream for binary data which must be base64-encoded. Structured types are preferred over plain text when the data has inherent structure, because the model reasons more reliably over structured input.
Templates handle the open-ended case. The server advertises resources/templates/list with entries carrying uriTemplate such as file://{path} or db://table/{table_name}/sample. Templates follow RFC 6570 with {variable} placeholders. The client resolves a template to a concrete URI and calls resources/read with it. Templates are the only correct answer for unbounded sets such as per-customer or per-file resources, registering a static resource per customer does not scale.
Subscriptions remove the need to poll. A client sends resources/subscribe with a URI, the server acknowledges, and later sends notifications/resources/updated with the same URI when content changes. The client then re-reads. The notification carries no content, only the signal. A server that supports push also supports resources/unsubscribe. For a static catalog with few entries, static registration suffices, but for live metrics, build status, or collaborative edits, subscriptions are the correct mechanism.
Pagination for resource and template lists uses nextCursor exactly as for tools and prompts. Each response may carry nextCursor, and the next request passes cursor. Cursors are opaque and server-scoped.
The decision framework our lessons use for resource versus tool is tested directly. A mostly static, slowly changing catalog that the host can provide up front is a resource. A parameterized, per-query, side-effectful, or model-driven operation is a tool. Many production servers expose both: a schema or catalog as a resource and a query or mutation as a tool, so the model knows the shape from the resource before it crafts the tool call.
Mechanism reference: Build versus use, tool descriptions, and least privilege
For standard integrations, evaluate community servers first. Building custom duplicates tested, maintained work and adds permanent maintenance with no functional gain for a standard need. The correct build justifications are team-specific workflows, custom business logic that must live in the tool layer, or integration with a proprietary system that has no community server. A hybrid scenario where a team needs one standard system plus one proprietary workflow is the textbook combined case: use a community server for the standard system and a thin custom server for the proprietary one. Wrapping the existing system rather than forking a community codebase is the expected thin-server pattern.
Tool descriptions are the highest-leverage field for tool selection. The model chooses among tools based almost entirely on name, description, and inputSchema properties. A sparse description loses to a rich built-in description for the same job. The lesson anti-pattern is a description of Searches code, the strong pattern is a multi-sentence description that states what the tool does, what it returns, when to use it, and how it compares to the built-in alternative, for example noting that an AST-aware semantic search returns functions, classes, and methods with file path, line numbers, and surrounding context and is more accurate than text grep for intent search. Every inputSchema property must carry its own description so the model knows what value to provide, enum must be used when values are fixed, and required must be minimal so the model can actually construct a valid call.
Least privilege must be enforced at the server, deterministically. A filesystem server should allow specific directories and deny everything else by default. A server offering both read and write operations should expose only the read tools to a read-only session. System-prompt instructions and output-pattern monitors are not substitutes, because an injection can override them or encode leaked data. Docker read-only prevents writes but still permits sensitive reads, TLS protects in transit but does not bound authorization, and both need server-level path and tool scoping alongside them. The exam's correct answer for injection containment is always the server-level allow-list or path boundary, never output filtering.
Roots are the related boundary primitive. They let the client communicate to the server which directories or operational boundaries are permitted. They are distinct from prompts, sampling, and stop sequences, and the correct distinction the exam probes is that roots scope where a server may act, not which tools it offers.
Sampling is the optional capability where a server can request an LLM completion via the host through sampling/createMessage. It is negotiated in capabilities and carries circular dependency risk when the host, model, tool, and server form a loop. Most servers should not use it. The protocol's forward note deprecates it in a later revision and prefers server-side model access or direct API calls. For the exam, the testable point is that sampling is client-mediated, optional, and a risk when used in tool-call loops.
Ownership map
Which layer owns which guarantee determines where the exam expects a fix to be applied. A fix in the wrong layer is always a distractor.
| Layer | Owns | Does not own | How failure looks |
|---|---|---|---|
| Protocol specification | JSON-RPC envelope shape, initialize and initialized ordering, tools/list versus resources/list versus prompts/list versus tools/call, URI and template semantics, nextCursor, notification names notifications/tools/list_changed and notifications/resources/updated, transport definitions for stdio and Streamable HTTP | File paths, environment handling, product configuration keys, host product UX | Non-compliant ordering or missing capability advertisement, method not found, pagination contract violation |
| Claude Code host process | Configuration files and scope resolution, ${VAR} and ${VAR:-default} expansion in command, args, env, url, headers, precedence selection across scopes, spawning stdio processes and connecting to Streamable HTTP endpoints, discovery aggregation into one flat toolkit, namespaced allow-list references such as mcp__server__tool, process isolation and approval gates | Authenticating a remote HTTP server beyond passing the configured header or token, enforcing per-tool authorization beyond what the server checks, inventing an auth JSON-RPC method | Missing tools because the command is not on the path, because the server omitted tools from capabilities, or because environment variables were not set locally |
| Claude Code server process | Exposure of capabilities it actually implements, input validation in each handler via JSON Schema, handler correctness for tools/list and tools/call or resources/list and resources/read, scoped directory and tool exposure for least privilege, sanitization of tool results and error messages before they enter context, audit logging | Deciding which servers the host should connect to, choosing which description the model prefers, passing an approval gate by itself | Prompt-injection via tool result, sensitive read because path allow-list was open, vague description losing to a built-in |
| Messages API connector | Carrying mcp_servers and mcp_toolset in the API request, validating that the URL is publicly reachable HTTPS, performing discovery and calls as the platform client, returning tool_result blocks inline | Running or reaching a local stdio command, exposing prompts or resources, inventing local filesystem access for a remote API call | A URL pointing at localhost that the platform cannot resolve to the developer machine, a missing mcp_toolset so no tools surface |
| Model | Choosing which tool to call and with what arguments based on description and schema, deciding whether to use a resource for context or a tool for action, reformulating after a resources/updated notification or a tool error | Transport, authentication mechanism, file storage, precedence | Sparse description causing preference for a built-in, confirmation bias in scenario reasoning that assumes transport rather than description is the cause |
| Infrastructure | TLS termination for Streamable HTTP, OAuth authorization server for user-delegated flows, load balancer with Mcp-Method and Mcp-Name awareness, Kubernetes health and ready endpoints, connection pools, rate limiters with Retry-After | Protocol ordering, MCP message semantics, prompt templates | 401 or 403 because the token was not issued for resource indicated server, 429 because bursty parallel calls exceeded budget, 503 readiness because the database pool is exhausted |
The host is the security boundary. It decides which servers are connected and what scope each has, mediates every tool invocation and resource read so it can enforce approval, and isolates each server in its own process so a crash or compromise cannot traverse to peers. For stdio the process boundary is the isolation. For Streamable HTTP the transport boundary is TLS plus the header or token that the host forwards on behalf of the developer.
Version and terminology currency
This task sits at a point where the protocol, the product, and the exam vocabulary use overlapping but not identical terms. Keeping them straight is part of passing.
Protocol version. The stable specification version referenced throughout our lessons is 2025-06-18. Previous protocol version 2024-11-05 defined the standalone HTTP plus SSE transport that is now deprecated. Protocol version 2025-03-26 replaced it with Streamable HTTP. Every lesson that teaches stdio versus Streamable HTTP is teaching the 2025-03-26 and later view, and every older tutorial that says to pick SSE is teaching 2024-11-05. The correct new-server choice is Streamable HTTP.
Upcoming changes flagged in our lessons but still flagged as forward-looking notes: a stateless revision that removes the explicit initialize handshake in favor of _meta on every request with server/discover, TTL-based caching fields ttlMs and cacheScope on list responses, routable transport headers Mcp-Method and Mcp-Name on Streamable HTTP POST, and deprecation of sampling. These are explicitly called out as upcoming, not current operational requirements, and the exam distinguishes them as notes, not as the current handshake to implement for a new server built with the SDK today.
Product terminology. The config key is mcpServers with command, args, env, url, headers. Environment expansion is ${VAR} and ${VAR:-default} in command, args, env, url, and headers. The popular exam phrasing of two files, .mcp.json for project and ~/.claude.json for user, is conceptually accurate. Current product behavior that refines it without contradicting it is that ~/.claude.json holds both local and user scopes under distinct sections, and that plugin-provided servers and hosted connectors are additional sources. An answer that asserts three named scopes plus those additional sources with precedence local, project, user, plugin, connector is the current product answer.
Transport terminology. The exam now treats SSE as an internal streaming detail of Streamable HTTP, not as a peer transport to pick. A choice that offers SSE transport versus Streamable HTTP should be read as legacy SSE equals deprecated predecessor, Streamable HTTP equals correct remote transport. There is no WebSocket, gRPC, raw TCP or UDP, or GraphQL transport in the specification, and any answer offering one is a transport distractor.
API connector terminology. The connector is a product feature on the Messages API, not a protocol primitive. Its surface is mcp_servers with typed entries and mcp_toolset in the tools array, not a file-based mcpServers registration. A response that puts a local stdio command inside mcp_servers is a surface-confusion trap.
Credential terminology. Stdio transport authenticates via environment credentials read from env, not via OAuth. Streamable HTTP authenticates via OAuth 2.1 with PKCE and resource parameter validation or via a proxy that terminates bearer tokens or mutual TLS. There is no JSON-RPC auth or authenticate method. The exam tests this negative fact directly.
Governance note. MCP was donated in December 2025 to the Agentic AI Foundation, a Linux Foundation directed fund co-founded with Anthropic, Block, and OpenAI founding contributions. Day-to-day protocol direction stays with maintainers, the foundation oversees strategic investment, not protocol decisions. This is background knowledge, not a configuration trap, but it frames the open standard claim.
Official versus community divergence
Where our internal documentation, current Anthropic product documentation, and broader ecosystem writing diverge, the candidate answers with the documentation position and can note the divergence.
The two scopes versus three scopes divergence is the most consequential. The reference page teaches project .mcp.json versus user ~/.claude.json as the complete scoping model. Our mcp-security lesson and the live Claude Code reference teach three named scopes, local, project, and user, plus plugin and hosted sources, with ~/.claude.json holding both local and user sections. The divergence is one of granularity, not contradiction. The correct exam answer is still project versus user for team versus personal, and the correct product answer when asked for complete scope enumeration includes local as project-private but not shared. A candidate who answers team servers belong in .mcp.json is correct under both framings.
The transport divergence is SSE as a transport. A large body of tutorials and older servers still describe SSE transport as the remote option. Current product documentation and the 2025-06-18 specification teach that standalone SSE is deprecated and Streamable HTTP is the standard remote transport, with SSE surviving only as the framing of a streamed response. The candidate answers with the deprecated framing for historical identification and with Streamable HTTP for any new server.
The tool-poisoning divergence is the most dangerous community misconception. Informal material sometimes claims that because MCP is a protocol, any MCP server is safe by definition. Our security lessons and the platform guidance state the opposite: a server is executed code that carries the host's permissions, and tool descriptions and results are untrusted text that can carry prompt-injection payloads directing exfiltration. The documentation position is vet, scope, treat output as untrusted. The community shortcut fails security review and is the wrong answer.
The build versus use divergence is low severity but recurrent. Informal advice sometimes recommends building custom for full control even when a standard integration exists, or equivalently shelling out via Bash to an API instead of adopting a server. The documented position is community first for standard integrations, custom only for proprietary or team-specific needs, and community adoption plus thin wrapping rather than forking. A direct Bash call is correct only for a genuine one-off, not for a recurring workflow that benefits from schema discovery.
The resource versus tool divergence shows up as over-tooling. Ecosystem examples sometimes model every access as a tool because tools feel familiar. The specification and our lessons maintain the distinction on control axis: resources for application-fetched read-only catalogs, tools for model-driven actions. Lean exam answers eliminate pure read-catalog-as-tool options when the goal is reducing exploratory calls.
The server lifecycle divergence appears as stale advice to restart transiently failing servers with host flags. The lesson position is to distinguish three missing-tool causes: process never started due to missing command, server started but omitted tools capability in initialize, and a server that declared tools but returned empty or error from tools/list. Only the first is a path or installation issue. The others require code changes, not restarts. Informal material that lumps all missing-tool cases under restart is imprecise.
Where documentation and ecosystem writing conflict, documentation wins. For this task that means: answer .mcp.json for team servers, use ${VAR} expansion, choose Streamable HTTP for remote, use resources for catalogs, enhance descriptions to win selection, and scope every server to least privilege at the server, not in the prompt.
Beyond the task statement
The reference page covers scoping, expansion, discovery, resources, build versus use, and description quality. Our lesson set covers substantial adjacent material that the reference page omits but that supports the same task and appears in scenario follow-ons.
mcp-architecture in full provides the three-layer diagram, the four-phase lifecycle sequence, the capability negotiation contract, JSON-RPC 2.0 fundamentals, the API connector note as the host-managed alternative, and the governance donation. It is the foundation for every other lesson and the source for the surface separation that prevents category errors. Without it, a candidate cannot distinguish protocol from product.
mcp-transports supplies the deprecation narrative, the Streamable HTTP details including Mcp-Session-Id, Last-Event-ID, Mcp-Method, Mcp-Name, horizontal scaling without affinity, and the legacy two-endpoint SSE pattern that older servers still expose. The reference page mentions no transport by name, so transport questions draw entirely on this lesson.
mcp-security deepens credential handling beyond the single ${GITHUB_TOKEN} example. It defines the five-layer defense stack of authentication, authorization, input validation, output sanitization, and audit logging; the exact scope-storage behavior with precedence; OAuth 2.1 for HTTP with PKCE and resource indicators; and why stdio skips OAuth and reads from environment. The prompt-injection and least-privilege sections are unique to this lesson.
building-mcp-servers gives the concrete SDK shapes in Python and TypeScript, the two-handler-per-primitive pattern, lifecycle table, Inspector workflow, error handling guidance, and transport binding code. The minimal server example candidates write by hand is drawn from here.
mcp-production covers what running in production actually requires: transport-level authentication choices from API key to mutual TLS, per-client and per-tool rate limiting with 429 and Retry-After, connection pooling, health and ready endpoints, observability at the tool level, horizontal scaling with Kubernetes deployment and autoscaler examples, and versioned endpoints. None of this appears on the reference page, but every operational scenario does.
mcp-tools provides the full execution flow from tools/list discovery through tool_use and tool_result round-trip, schema design guidance with name, description, inputSchema, enum, required, dynamic registration via notifications/tools/list_changed, and idempotency for side-effectful tools. The reference page's sparse-description trap draws its resolution from here.
mcp-resources expands the payoff claim into templates, subscription push notifications, pagination, MIME choices, and the resource versus tool decision framework that the reference page summarizes in one paragraph. It is the source for every resource-shaped exam answer.
mcp-prompts is the third primitive the reference page barely mentions. It defines prompts/list and prompts/get, argument substitution for prompt templates, and slash-command integration in Claude Code. Team-standardization scenarios that mention reusable workflows or slash commands draw on this lesson.
mcp-integration is the Claude Code integration narrative that turns configuration into behavior: .mcp.json wiring, secret management, aggregation, dynamic updates, lifecycle per transport, troubleshooting table, and the built-in tool selection guidance distinguishing Read versus Grep versus Glob and similar. The exam's internal-search versus remote-api distinction is drawn from this lesson.
configuration from the Claude Code domain supplies the broader picture of Claude Code configuration beyond MCP: CLAUDE.md, .claude/settings.json, and the related .claude.json scope file that MCP shares with other settings. Candidates who conflate CLAUDE.md prose with MCP registration fail items that depend on the registration boundary.
Worked production examples
The examples below form a coherent progression. A team needs an internal knowledge search plus a proprietary approval workflow, wants Jira for the standard tracker, needs a database catalog to avoid exploratory calls, and runs a partner booking service over the network. Each example is substantial, runnable, and tied to its failure boundary and observable output.
Worked production examples: Example 1: Project shared servers with environment expansion
The project file configures the standard systems every engineer must receive. Credentials are referenced but never stored.
{
"mcpServers": {
"github": {
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-github"],
"env": {
"GITHUB_TOKEN": "${GITHUB_TOKEN}"
}
},
"jira": {
"command": "npx",
"args": ["-y", "@community/mcp-server-jira"],
"env": {
"JIRA_URL": "${JIRA_URL}",
"JIRA_TOKEN": "${JIRA_TOKEN}"
}
},
"internal-search": {
"command": "node",
"args": ["./scripts/mcp-search-server.js"],
"env": {
"SEARCH_API_KEY": "${INTERNAL_SEARCH_KEY}",
"SEARCH_INDEX": "company-docs"
}
}
}
}What it proves: mcpServers is the correct top-level key, command plus args drives stdio spawn, env carries a map of names to expansion strings, and ${INTERNAL_SEARCH_KEY} keeps the actual key out of history. Every developer clones this file identically, sets the three variables in their shell or vault, and receives the same three servers without further configuration.
Failure boundary: a teammate who cannot list GitHub issues has almost certainly not set GITHUB_TOKEN locally. The symptom is auth failure after discovery succeeds, not missing-tool discovery.
Observable output: after git pull, both servers appear in the host's server list without any per-developer registration step. Running git diff before commit shows no secret, and rotating a token changes no file.
Worked production examples: Example 2: User file for an experimental server
The developer is testing a personal semantic code search server before proposing it to the team. It belongs in personal scope, not in the project file.
{
"mcpServers": {
"semantic-search-experiment": {
"command": "node",
"args": ["./experiments/semantic-search-server.js", "--index", "${HOME}/.cache/semantic-index"],
"env": {
"EXPERIMENTAL_SEARCH_TOKEN": "${EXPERIMENTAL_SEARCH_TOKEN:-}",
"INDEX_MODE": "${INDEX_MODE:-incremental}"
}
}
}
}Placement context: this block lives in personal scope. In current product behavior that is the user or local section of ~/.claude.json, not in the repository's .mcp.json. In the popular two-file mental model it is the content of ~/.claude.json. The path-valued args entry demonstrates expansion inside args, and the env block shows ${VAR:-default} falling back to empty or incremental when the developer has not exported the variable.
What it proves: args and env both expand, defaults avoid hard failures for optional variables, and availability is scoped to the single developer's sessions. No teammate is affected.
Failure boundary: placing this under .mcp.json with a commented toggle does not prevent loading, every cloner would load it. The only correct boundary is scope.
Observable output: the experimental server appears locally when the developer opens any project that loads personal scope, or only the current project for local scope. Teammates who pull the project repo see no change in their server lists.
Worked production examples: Example 3: Minimal server exposing a tool and a resource with scoped transport binding
The team builds a thin wrapper around an internal documentation service. It exposes a semantic search tool and a catalog resource that lists all available documentation collections. Transport is selected at startup so the same handlers run locally via stdio and in staging via Streamable HTTP.
import { Server } from "@modelcontextprotocol/sdk/server/index.js";
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
import {
ListToolsRequestSchema,
CallToolRequestSchema,
ListResourcesRequestSchema,
ReadResourceRequestSchema,
} from "@modelcontextprotocol/sdk/types.js";
const server = new Server(
{ name: "internal-docs-server", version: "1.0.0" },
{
capabilities: {
tools: {},
resources: {},
},
}
);
server.setRequestHandler(ListToolsRequestSchema, async () => ({
tools: [
{
name: "search_internal_docs",
description:
"Performs semantic search across the company's internal documentation using AST-aware indexing for code-adjacent prose and hybrid keyword matching for handbook content. Use when the engineer asks how an internal system works, what parameters an internal API accepts, or what a specific platform feature does. Returns up to 5 matching sections with title, hierarchy path, last-updated date, and excerpt. More accurate than text-based Grep for finding concepts by intent rather than exact string. Do not use for general web knowledge, use web search instead.",
inputSchema: {
type: "object",
properties: {
query: {
type: "string",
description: "Specific search query using technical terms from the internal platform. Use 3 to 8 keywords for best precision.",
},
collection: {
type: "string",
description: "Optional collection filter. Valid values: 'handbook', 'api-reference', 'runbooks'.",
enum: ["handbook", "api-reference", "runbooks"],
},
limit: {
type: "number",
description: "Maximum sections to return. Default 5. Range 1 to 20.",
minimum: 1,
maximum: 20,
},
},
required: ["query"],
},
},
],
}));
server.setRequestHandler(CallToolRequestSchema, async (request) => {
const { name, arguments: args } = request.params;
if (name !== "search_internal_docs") throw new Error(`Unknown tool: ${name}`);
const { query, collection, limit = 5 } = args as {
query: string;
collection?: string;
limit?: number;
};
const results = await searchDocs(query, { collection, limit });
return { content: [{ type: "text", text: JSON.stringify(results, null, 2) }] };
});
server.setRequestHandler(ListResourcesRequestSchema, async () => ({
resources: [
{
uri: "docs://catalog/collections",
name: "Documentation Collections",
description: "Hierarchy of all documentation collections with section counts and last update dates",
mimeType: "application/json",
},
],
}));
server.setRequestHandler(ReadResourceRequestSchema, async (request) => {
const { uri } = request.params;
if (uri === "docs://catalog/collections") {
const catalog = await fetchDocsCatalog();
return {
contents: [{ uri, mimeType: "application/json", text: JSON.stringify(catalog, null, 2) }],
};
}
throw new Error(`Resource not found: ${uri}`);
});
async function searchDocs(query: string, opts: { collection?: string; limit: number }) {
return { query, opts, hits: [] };
}
async function fetchDocsCatalog() {
return { collections: [] };
}
if (process.env.MCP_TRANSPORT === "stdio") {
await server.connect(new StdioServerTransport());
}What it proves: declaring capabilities with both tools and resources is required for discovery, the name plus multi-sentence description steers selection over a built-in, inputSchema carries per-property description, enum, and bounds, and the two-handler pattern per primitive holds. The catalog resource gives the model hierarchy visibility without an exploratory tool call, and the tool executes the model-driven query.
Failure boundary: a sparse description of Searches code would cause the model to choose Grep even when this tool is strictly more accurate, and omitting tools from capabilities would make discovery fail even though handlers are registered. Running the Inspector against the stdio command would surface either immediately.
Observable output: resources/list returns one collection catalog entry, resources/read on docs://catalog/collections returns the JSON hierarchy, and tools/list returns a richly described search tool. The host aggregates both without additional wiring.
Worked production examples: Example 4: Streamable HTTP transport with health, readiness, and rate limiting
The same server logic can be bound to an HTTP endpoint for staging or team-shared deployment. This example adds production gates that the lessons require for any shared service.
import express from "express";
import { Server } from "@modelcontextprotocol/sdk/server/index.js";
import { StreamableHTTPServerTransport } from "@modelcontextprotocol/sdk/server/streamableHttp.js";
import { Pool } from "pg";
const app = express();
app.use(express.json());
const pool = new Pool({
connectionString: process.env.DATABASE_URL,
max: 20,
idleTimeoutMillis: 30_000,
connectionTimeoutMillis: 5_000,
});
function authenticate(req: express.Request, res: express.Response, next: express.NextFunction) {
const token = req.headers.authorization?.replace("Bearer ", "");
if (!token) return res.status(401).json({ error: "Missing authentication token" });
try {
const payload = verifyJwt(token);
(req as unknown as { user: unknown }).user = payload;
next();
} catch {
return res.status(401).json({ error: "Invalid or expired token" });
}
}
const limiters = new Map<string, { count: number; resetAt: number }>();
function checkRateLimit(clientId: string): boolean {
const now = Date.now();
const entry = limiters.get(clientId);
if (!entry || now > entry.resetAt) {
limiters.set(clientId, { count: 1, resetAt: now + 60_000 });
return true;
}
if (entry.count >= 100) return false;
entry.count += 1;
return true;
}
const server = new Server(
{ name: "internal-docs-http", version: "1.0.0" },
{ capabilities: { tools: {}, resources: {} } }
);
app.get("/health", (_req, res) => res.json({ status: "ok", timestamp: new Date().toISOString() }));
app.get("/ready", async (_req, res) => {
try {
await pool.query("SELECT 1");
res.json({ status: "ready" });
} catch (e) {
res.status(503).json({ status: "not ready", reason: (e as Error).message });
}
});
app.post("/mcp", authenticate, async (req, res) => {
const clientId = (req as unknown as { user: { sub: string } }).user.sub;
if (!checkRateLimit(clientId)) {
res.setHeader("Retry-After", "60");
return res.status(429).json({ error: "Rate limit exceeded" });
}
const transport = new StreamableHTTPServerTransport({ request: req, response: res } as unknown as never);
await server.connect(transport);
});
app.listen(8080, () => console.log("MCP server listening on :8080"));
function verifyJwt(_token: string): { sub: string } {
return { sub: "local" };
}Corresponding client-side configuration for Claude Code to reach this server is:
{
"mcpServers": {
"remote-docs": {
"url": "https://mcp.internal.example.com/mcp",
"headers": {
"Authorization": "Bearer ${REMOTE_DOCS_TOKEN}"
}
}
}
}What it proves: Streamable HTTP requires explicit authentication on every request, rate limiting with 429 and Retry-After, liveness versus readiness separation, and header-based credentials supplied via ${VAR}. The url plus headers shape is the verified remote shape, complementary to the stdio command shape. Mcp-Method and Mcp-Name routable headers can be added at the gateway from this foundation.
Failure boundary: serving the same logic over plain HTTP with no authenticate middleware is equivalent to an unauthenticated database. Skipping the ready check that verifies the pool lets the load balancer send traffic to a server that cannot handle a tool call.
Observable output: curl against https://mcp.internal.example.com/health returns ok, ready returns 503 when the pool is down, and a burst of 101 calls in one minute from one client receives 429 with Retry-After: 60 while other clients are unaffected.
Worked production examples: Example 5: Server-level authorization and scoped exposure
A data server must enforce authorization per action, not just authentication per connection. This example enforces a tool allow-list derived from the authenticated principal and a path boundary for file-oriented operations.
import { Server } from "@modelcontextprotocol/sdk/server/index.js";
import {
CallToolRequestSchema,
ListToolsRequestSchema,
} from "@modelcontextprotocol/sdk/types.js";
const server = new Server(
{ name: "scoped-data-server", version: "1.0.0" },
{ capabilities: { tools: {} } }
);
const ALLOWED_TOOLS_BY_ROLE: Record<string, string[]> = {
viewer: ["query_analytics", "list_reports"],
editor: ["query_analytics", "list_reports", "create_report"],
admin: ["query_analytics", "list_reports", "create_report", "manage_sources"],
};
const ALLOWED_SCHEMAS = new Set(["analytics", "reporting"]);
function isReadOnlyQuery(sql: string): boolean {
return /^\s*select\b/i.test(sql) && !/\b(insert|update|delete|drop|create|alter)\b/i.test(sql);
}
function referencesAllowedSchema(sql: string): boolean {
return [...ALLOWED_SCHEMAS].some((s) => new RegExp(`\\b${s}\\b`, "i").test(sql));
}
server.setRequestHandler(ListToolsRequestSchema, async (_req, _context: unknown) => {
const role = (globalThis as unknown as { currentRole?: string }).currentRole ?? "viewer";
const allowed = new Set(ALLOWED_TOOLS_BY_ROLE[role] ?? ALLOWED_TOOLS_BY_ROLE.viewer);
const allTools = [
{
name: "query_analytics",
description:
"Execute a read-only SELECT against the analytics or reporting schemas. Use for questions about product metrics, funnel conversion, or retention cohorts. Returns tabular JSON. Do not use for mutations.",
inputSchema: {
type: "object",
properties: {
sql: { type: "string", description: "Read-only SQL statement. Only SELECT is permitted." },
},
required: ["sql"],
},
},
{
name: "list_reports",
description: "List available report templates with their required parameters and refresh cadence.",
inputSchema: { type: "object", properties: {}, required: [] },
},
{
name: "create_report",
description: "Create a new report instance from a template. Requires editor role or above.",
inputSchema: {
type: "object",
properties: {
template: { type: "string", description: "Template identifier from list_reports" },
},
required: ["template"],
},
},
];
return { tools: allTools.filter((t) => allowed.has(t.name)) };
});
server.setRequestHandler(CallToolRequestSchema, async (request, context: unknown) => {
const ctx = context as { user?: { role: string; id: string } } | undefined;
const role = ctx?.user?.role ?? "viewer";
const allowed = new Set(ALLOWED_TOOLS_BY_ROLE[role] ?? ALLOWED_TOOLS_BY_ROLE.viewer);
const { name, arguments: args } = request.params as { name: string; arguments: Record<string, string> };
if (!allowed.has(name)) throw new Error(`Access denied for tool '${name}' with role '${role}'`);
if (name === "query_analytics") {
const sql = args.sql;
if (!isReadOnlyQuery(sql)) throw new Error("Only SELECT queries are permitted");
if (!referencesAllowedSchema(sql)) throw new Error("Query references unauthorized schema");
return { content: [{ type: "text", text: JSON.stringify({ rows: [], sql }) }] };
}
return { content: [{ type: "text", text: JSON.stringify({ ok: true, tool: name }) }] };
});What it proves: fine-grained authorization is per tool call, not per connection, and per-tool exposure is filtered in tools/list based on the principal's role, so a viewer never sees create_report or manage_sources. SQL-level guards enforce that mapping even if a tool call bypasses the listing.
Failure boundary: instructing the model in the system prompt to use only read operations is not deterministic. An injection can override it, while this server rejects the unauthorized CallToolRequestSchema even if the model attempts it.
Observable output: as viewer, tools/list returns two tools. The same session, after authenticating as editor, returns three. A query_analytics call with DELETE FROM analytics.events is rejected at the handler before any downstream execution.
Worked production examples: Example 6: Messages API connector for a remote partner service
An application that does not run its own MCP client can attach the partner booking service through the API connector. The request carries the server declaration and the toolset that enables it, and the platform performs discovery and invocation.
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });
const response = await client.messages.create({
model: "your-model-id",
max_tokens: 1024,
mcp_servers: [
{
type: "url",
name: "partner-bookings",
url: "https://api.partner.example.com/mcp",
authorization_token: "Bearer ${PARTNER_MCP_TOKEN}",
},
],
tools: [
{
type: "mcp_toolset",
mcp_server_name: "partner-bookings",
},
],
messages: [
{
role: "user",
content: "Find available bookings for next Tuesday near the base village and hold one cabin if available.",
},
],
});
for (const block of response.content) {
if (block.type === "tool_use") console.log("tool use", block.name, block.input);
if (block.type === "text") console.log(block.text);
}What it proves: mcp_servers declares the remote, HTTPS URL and bearer token, mcp_toolset is what surfaces its tools, and no command or args appears because this surface cannot spawn a local process. The platform owns the MCP client state, the application owns only token lifecycle. Local stdio servers and non-tool primitives are correctly excluded.
Failure boundary: a value of https://localhost:3000/mcp in url fails because the platform resolves localhost against its own network, not the developer workstation. A missing mcp_toolset entry leaves the server declared but its tools invisible. Supplying a stdio command here is a surface confusion trap.
Observable output: the model's response includes one or more tool_use blocks for the partner's tools and their tool_result blocks inline, interleaved with text blocks that explain the held cabin, all without any client-side MCP transport code in the application.
Worked production examples: Example 7: Resource template with parameterized scope and subscription signal
A per-customer ledger that cannot be enumerated at startup must be addressed per customer.
import { Server } from "@modelcontextprotocol/sdk/server/index.js";
import {
ListResourceTemplatesRequestSchema,
ReadResourceRequestSchema,
} from "@modelcontextprotocol/sdk/types.js";
const server = new Server(
{ name: "ledger-server", version: "1.0.0" },
{ capabilities: { resources: {} } }
);
server.setRequestHandler(ListResourceTemplatesRequestSchema as never, async () => ({
resourceTemplates: [
{
uriTemplate: "ledger://customer/{customer_id}/summary",
name: "Customer Ledger Summary",
description: "Summary of the ledger for the given customer. Provide the customer_id from the authenticated context.",
mimeType: "application/json",
},
],
}) as unknown as never);
server.setRequestHandler(ReadResourceRequestSchema, async (request) => {
const uri = request.params.uri as string;
const match = uri.match(/^ledger:\/\/customer\/([^/]+)\/summary$/);
if (!match) throw new Error(`Unknown resource: ${uri}`);
const customerId = match[1];
const summary = await fetchLedgerSummary(customerId);
return {
contents: [{ uri, mimeType: "application/json", text: JSON.stringify(summary, null, 2) }],
};
});
async function fetchLedgerSummary(_customerId: string) {
return { customerId: _customerId, open: 0, total: 0 };
}What it proves: uriTemplate with RFC 6570 placeholders is the open-ended alternative to static uri registration, the client resolves {customer_id} to a concrete URI at read time, and the handler enforces existence before returning structured application/json rather than text/plain.
Failure boundary: registering one static ledger://customer/ACME/summary per customer at startup does not scale to tens of thousands of customers and mints URIs that are never read.
Observable output: resources/templates/list advertises the template shape, and resources/read on ledger://customer/ACME/summary returns that customer's summary while another customer's URI returns a different payload.
Build exercise material
Reproducible steps with the observable outcome that proves each step worked. Perform them in order, they build on one another. All file paths are from the project root unless noted as the home directory.
Step 1. Create the project file with one community server and a known-variable reference.
Create docs/tmp-mcp-2-4/.mcp.json for sandbox testing, or edit the real project root .mcp.json:
{
"mcpServers": {
"github": {
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-github"],
"env": {
"GITHUB_TOKEN": "${GITHUB_TOKEN}"
}
}
}
}Verify shape: run python3 -m json.tool .mcp.json to confirm valid JSON with no trailing commas. Verify no secret was written: run git diff and git diff --cached and confirm no literal token appears. The only string that should appear is ${GITHUB_TOKEN}.
Step 2. Supply the variable from the local environment, not from the file.
Export locally with export GITHUB_TOKEN=ghp_local_dev_token_from_your_vault or via your shell profile or .env ignored by version control. Restart or reload the host session. If the variable is missing, the server will be discovered but every call fails with an auth error, which is the correct diagnostic that expansion produced an empty value. The fix is to set the variable and restart, not to edit the committed file.
Observable outcome: at host startup, the server appears in the connected server list without any per-developer file edit. Running env | grep GITHUB_TOKEN on the spawning machine shows the value, while cat .mcp.json shows only the variable name.
Step 3. Add a personal server in personal scope.
Edit the personal section of ~/.claude.json to add a server visible only to your sessions:
{
"mcpServers": {
"personal-notes": {
"command": "node",
"args": ["./scripts/personal-notes-server.js"],
"env": {
"NOTES_TOKEN": "${PERSONAL_NOTES_TOKEN}"
}
}
}
}Verify scope: ls -la ~/.claude.json confirms the file is in the home directory and is not under repository path tracking. Run git status from the project root and confirm this edit produces no repository change. That proves scope separation.
Observable outcome: the personal server appears alongside the project servers in your sessions, but a teammate who clones the repository without that home-directory file sees only the project servers.
Step 4. Expose a catalog as a resource.
In the minimal server from Example 3, ensure both handlers are present, then exercise them outside Claude Code via the Inspector:
npx @modelcontextprotocol/inspector --server-command "node ./scripts/mcp-search-server.js"
Visit the listed resources panel. The catalog at docs://catalog/collections should list available collections by name and hierarchy path with mimeType application/json. Click read on that single URI and confirm the returned contents array contains valid JSON with at least collections and lastUpdated style fields. If the resource appears in list but read returns Resource not found, the URI routing inside the handler is incorrect.
Observable outcome: resource discovery takes one call, and a separate tool call loop is not needed to learn what collections exist. That is the payoff the resource primitive provides.
Step 5. Enhance the tool description so the search MCP tool wins over Grep.
Replace any single-sentence description with the multi-sentence shape from Example 3. The strong description must include: what the tool does, what it returns and in what shape, when to use it, and an explicit comparison naming the built-in it competes with. For a code search tool, that comparison names Grep, for a table-extraction tool it names Read. Redeploy the server.
Verification ritual: ask the host model two probes in the same file tree. First, try a conceptual request such as find where we validate checkout totals and confirm the trace shows the MCP search tool being called over Grep. Second, try a literal-string probe such as find the literal line containing "TODO FIXME" and confirm Grep remains preferred for exact-string cases. The first probe validates the enhanced description, the second validates that built-ins are not hidden, only properly routed.
Observable outcome: tool choice shifts from generic text search to the MCP search tool for intent-shaped questions while preserving Grep for literal cases, which is exactly the behavior the sparse-description trap targets.
Step 6. Confirm precedence and disabled behavior.
Add a server named internal-search to both scopes, with a distinguishing flag in env such as VARIANT: "project" in .mcp.json and VARIANT: "user" in personal scope. Restart the host and fetch the effective configuration via the host's server info panel or by observing which variant's log message appears. The local-then-project-then-user precedence predicts the project variant wins over the user variant, and local wins over both if present.
If you intend a server to stay configured but inactive, use the host's configured inactive control rather than deleting the entry. The block should keep the same server fields but surface as inactive so the remaining active servers still aggregate cleanly.
Observable outcome: one single server instance runs, and its VARIANT value proves which source won, confirming that fields do not merge across scopes.
Mechanism and API surface
- Two tier configuration map
- Project dot mcp dot json at the repository root with mcpServers holding command, args, and env, version controlled and shared, versus user tilde slash dot claude dot json in the home directory, personal and never shared, with every tool from every server on either tier discovered simultaneously with no manual step.
- ENV_VAR expansion for secrets
- Write dollar brace VARIABLE_NAME in dot mcp dot json env blocks such as GITHUB_TOKEN, JIRA_URL, and JIRA_TOKEN, with each developer and CI injecting values locally via shell profile or secret manager, keeping the shared file credential free and rotation requires no JSON edit.
- JSON-RPC 2.0 lifecycle and pagination
- Every connection sequences initialize to exchange supported primitives and version, initialized to confirm readiness, then discovery calls tools slash list and resources slash list, and invocation via tools slash call with pagination via nextCursor as an opaque string the client must not parse.
- Transport choice without behaviour change
- Stdio wraps a child process with stdin and stdout for local development and fast iteration, Streamable HTTP exposes MCP over HTTP as stateless endpoints scaling across users, with legacy SSE superseded, selection does not change tool semantics but determines sharing scope.
- Resources versus Tools versus Prompts
- Resources are read only host controlled catalogues with uri, name, description, and mimeType, Tools are model controlled actions with name, description, and inputSchema that may mutate the world, Prompts are user task templates consumed as slash commands or skills, with transport security and sampling governed at the host.
- Enriched MCP description for selection
- Expand sparse entries such as Searches code into multi sentence statements that name AST aware capability, returned shape with file path and line number, and when to prefer this tool over Grep or sibling servers, so model choice under auto is guided by substance rather than description length.
The decision rules in play
Each rule below is a distinct decision the exam tests, stated with its mechanism, the boundary where the answer flips, and the wrong answers written against it.
Configuration scope is two-level
An agent host reads server definitions from two locations. A project file lives at the repository root and travels with the codebase through version control. A user file lives in the developer home directory and never enters version control. At session start the host reads both and merges their declared servers into one set available to the agent.
The project file is the team's single source of truth for shared tooling; the user file lets an individual experiment without editing that shared file. The two scopes are additive: a developer gets the union of project and user servers, while a teammate with only the project file sees only the team-wide servers.
Boundary. The deciding question is "who must receive this server automatically?" If the answer is every person who clones the repository, the entry belongs in the project file. If the answer is only the one developer testing it, the entry belongs in the user file. The flip case makes this obvious: a personal experimental server placed in the project file would load for everyone who clones, which defeats the user file's whole purpose of keeping experiments private.
Recurring specifics. - Project file at repository root, committed. - User file in home directory, never committed. - Experimental servers never belong in the shared project file. - A subagent's tool visibility is controlled by allowedTools, not by which servers are connected.
Proposal. put shared servers in the user file so each developer controls them.
Why it attracts. it keeps everything "per machine."
Why it fails. it defeats the purpose of a single shared source of truth and causes drift across twenty machines.
When it would be right. a purely personal server, never a team-wide one.
Proposal. put the experimental server in the project file with a flag like beta: true.
Why it attracts. it looks like a tidy toggle.
Why it fails. any entry in the project file is loaded for every cloner; flags do not prevent loading.
When it would be right. never for a personal experiment; the user file is the correct home.
Proposal. write the server into CLAUDE.md prose.
Why it attracts. CLAUDE.md is the familiar per-project config surface.
Why it fails. CLAUDE.md is instruction text, not an MCP registration surface.
When it would be right. never for server registration.
- - Mutation: team of six with personal tokens versus one shared token. Correct answer unchanged: project file plus per-developer environment variables. - Mutation: experimental server must be testable locally without touching the repo. Correct answer unchanged: user file. - Mutation: system-wide scope mentioned. the tested material never treats a system-wide file as the canonical answer; the two real scopes are project and user.
Environment variable expansion keeps credentials out of version control
Inside the project file, a server entry can reference a variable with ${NAME} syntax inside its env block. At launch the host resolves that reference from the developer's own shell environment. The committed file stores only the variable name, never the value.
Version control history is permanent. Once a literal secret is committed, it remains extractable even after later edits. Expansion breaks the link between the shared configuration and any individual's actual credential, so the file is safe to commit and to share.
Boundary. Use expansion whenever the credential must be secret yet the server definition must be shared. The opposite becomes correct only when there is no shared file at all, such as a purely personal server in the user file where the value could in principle be inlined. Even there, expansion is still preferred.
Recurring specifics. - Syntax: env: { "GITHUB_TOKEN": "${GITHUB_TOKEN}" }. - Each developer sets the variable locally (shell profile, secrets manager, SSO-provisioned environment). - Token rotation needs no config change. - A developer who has not set the variable gets an empty or literal expansion and an auth failure.
Proposal. hardcode the token directly in the project file.
Why it attracts. zero local setup.
Why it fails. the secret enters git history and is compromised.
When it would be right. never for a shared file.
Proposal. move the whole file to the user scope so the token is per-user.
Why it attracts. keeps it off the repo.
Why it fails. it also removes the shared source of truth and causes drift.
When it would be right. only if the server itself is personal.
Proposal. commit a shared .env with all tokens.
Why it attracts. centralizes credentials.
Why it fails. still commits secrets to version control.
When it would be right. never.
Proposal. encrypt the token with a team key and commit the ciphertext.
Why it attracts. "it is encrypted."
Why it fails. still commits a secret artifact and shares the decryption key, violating per-developer credential policy.
When it would be right. never.
- - Mutation: the token fails for one new hire though the file is identical. Correct answer: that hire never set the variable locally. - Mutation: pool size and timeout differ per environment. Correct answer: expand
${POOL_SIZE}and${POOL_TIMEOUT}rather than hardcode or fork files. - Mutation: a secrets manager is suggested. Correct answer: it adds infrastructure and may still imply a shared account; expansion stays canonical.
All configured servers discovered at connection time, presented simultaneously
When the host initializes, it opens each configured server, performs capability negotiation, and collects the tool, resource, and prompt inventories. The aggregate is offered to the model as one combined toolkit in a single request. There is no per-turn opt-in.
MCP is designed for composability: a host aggregates capabilities from many servers. Discoverability at connection time is what makes multi-server setups useful. Manual per-turn activation would defeat the purpose.
Boundary. The rule holds for tool visibility across connected servers. The nearby opposite is that a specific subagent can be restricted by allowedTools, so the coordinator may itself hold all tools while a delegated subagent sees only a subset. That restriction is applied at the agent layer, not by the connection logic.
Recurring specifics. - Tools from project and user scope coexist; user scope does not replace project scope. - Two servers exposing identically named tools remain individually addressable. - The combined total of built-in tools plus several servers can become unwieldy.
Proposal. only the first server listed is loaded.
Why it attracts. "first wins" is a common config pattern.
Why it fails. hosts load all configured servers.
When it would be right. never.
Proposal. each server must be enabled per turn with a flag.
Why it attracts. sounds like safe opt-in.
Why it fails. no such flag exists in the model.
When it would be right. never.
Proposal. resources take priority over tools so the wrong tool is invoked.
Why it attracts. confuses the two primitives.
Why it fails. tools and resources are independent.
When it would be right. never.
- - Mutation: a tool from one server silently overwrites another. Correct answer: it does not; namespacing preserves both. - Mutation: the host refuses to load until a rename. Correct answer: it does not refuse.
Tools are namespaced by server name
Each server's tools are exposed under a qualified name that incorporates the server identifier. In one SDK the convention is mcp__<server-name>__<tool-name>. This lets two servers each expose get_pull_request without collision because the qualified names remain distinct.
Without namespacing, two independently authored servers could produce ambiguous tool names. The qualified scheme preserves addressability while still presenting a flat combined toolkit to the model.
Boundary. Namespacing prevents collision in the combined toolkit. The nearby opposite is misrouting: when two servers expose similarly named tools and the model picks the wrong one for a given request, the cause is ambiguous description or weak differentiation, not a naming clash.
Recurring specifics. - Convention: mcp__server__tool. - Allow-list patterns reference these qualified names, e.g. mcp__playwright__*. - Built-in tools remain alongside MCP tools.
Proposal. the second server's tool overwrites the first.
Why it attracts. naive map semantics.
Why it fails. namespacing preserves both.
When it would be right. never.
Proposal. the host refuses to load both.
Why it attracts. collision avoidance by refusal.
Why it fails. the host merges, it does not refuse.
When it would be right. never.
- - Mutation: asked whether identically named tools collide. Correct answer: they remain addressable. - Mutation: asked how to restrict a server's tools. Correct answer: an allow-list on qualified names, not renaming.
Evaluate community servers before building custom
For a widely used external system, a maintained community server almost always exists and covers standard operations. The correct first step when integrating such a system is to evaluate that existing server rather than start a build.
A community server is already tested, maintained, and kept current. Building from scratch duplicates that effort and adds a permanent maintenance burden for no current benefit when the standard integration meets the need.
Boundary. The rule holds for "standard integration." The nearby opposite is a team-specific workflow or proprietary system that no community server covers; there, building custom is justified. The flip is triggered by the phrase "community server cannot handle our specific workflow" or "no community equivalent exists."
Recurring specifics. - Named exemplar systems: issue trackers, source-hosting platforms, chat platforms, documentation tools, linear-task trackers. - The decision tree: does a suitable server exist, does it expose what the workflow needs, else build. - Community first is "always correct" for standard integrations.
Proposal. build custom first to guarantee full control.
Why it attracts. control feels safer.
Why it fails. premature; the community server already covers the need.
When it would be right. only when control is required for team-specific logic.
Proposal. call the REST API directly from a shell tool.
Why it attracts. no server setup.
Why it fails. loses structured schema, discovery, and error handling.
When it would be right. a genuine one-off query, not a recurring workflow.
Proposal. put it in the user file so each developer configures independently.
Why it attracts. personal control.
Why it fails. a team-wide integration belongs in the project file.
When it would be right. a personal server only.
- - Mutation: community server covers exactly what is needed. Correct answer: adopt it. - Mutation: community server covers most but team wants two field remappings. Correct answer: use it plus a lightweight post-processing step, not a from-scratch build. - Mutation: speculative future fields. Correct answer: still adopt community; do not build for hypothetical needs.
Build custom only for team-specific or proprietary needs
When the integration requires embedded business logic, allow-list validation, row-level access control, data masking, or a proprietary internal system with no community equivalent, a custom server is justified because the community option cannot meet the requirement.
The decision turns on whether required logic can live only in a server the team controls. Compliance controls, custom workflows, and proprietary backends are exactly that case.
Boundary. Custom is correct when community cannot. The opposite is correct when community can: standard integrations should not be rebuilt. The flip is triggered by the presence or absence of "team-specific workflow" or "no community server exists."
Recurring specifics. - HIPAA masking and department-credential checks justify custom. - Allow-list validation plus row-level access control justify custom for a database. - Proprietary internal approval workflow justifies custom while PostgreSQL and chat platforms stay community. - A custom server should remain thin, wrapping the existing system.
Proposal. fork the community server and maintain it.
Why it attracts. reuses code.
Why it fails. still a maintenance burden and usually unnecessary when a wrapper suffices.
When it would be right. only if the community server's internals must change.
Proposal. rely on built-in sandboxing to hide PII.
Why it attracts. "the platform handles it."
Why it fails. masking must be enforced at the server layer before data reaches the conversation.
When it would be right. never for compliance.
- - Mutation: community server exists but needs two field remaps. Correct answer: post-process, do not build. - Mutation: community server exists and covers needs. Correct answer: adopt, do not build. - Mutation: required control is compliance-critical. Correct answer: build, because community cannot enforce it.
MCP is an open, vendor-neutral standard
The protocol is an open specification defining a client-server contract for tools, resources, and prompts. Any compliant host can consume any compliant server.
Standardization is the entire value proposition: build or reuse one server per system and any capable host can consume it, replacing N bespoke integrations with one common protocol.
Boundary. The rule holds that the protocol is open and reusable. The opposite becomes correct only in the narrow sense that a specific product may ship its own client; the protocol itself is not owned by any single model vendor.
Recurring specifics. - Open standard, model-agnostic. - Replaces custom per-integration glue. - Not a hosted service, not a prompt framework, not a caching layer.
Proposal. MCP is proprietary to one vendor.
Why it attracts. a product popularized it.
Why it fails. the spec is open.
When it would be right. never.
Proposal. MCP replaces the model API entirely.
Why it attracts. conflates layers.
Why it fails. the model API still carries the conversation.
When it would be right. never.
Proposal. MCP requires a cloud endpoint.
Why it attracts. "servers are remote."
Why it fails. local stdio servers are first-class.
When it would be right. never.
- - Mutation: asked whether a server can be local. Correct answer: yes, stdio. - Mutation: asked to characterize MCP's value. Correct answer: open protocol standardizing tool and data access.
Three roles - Host, Client, Server
The Host is the application that initiates connections (an IDE plugin, a desktop app, a CLI). The Client is the protocol implementation inside the Host that maintains a one-to-one connection to a Server. The Server provides capabilities: resources, tools, prompts.
This separation lets the host application own the model loop while the embedded client speaks the protocol to each server. The client handles initialization, capability negotiation, and message routing.
Boundary. The rule holds that the host embeds the client and connects to servers. The opposite becomes correct when the application delegates the MCP client to infrastructure: in the Messages API connector pattern, the platform acts as the client, not the local harness.
Recurring specifics. - Host initiates; Client implements protocol; Server exposes capabilities. - The model is reached through the Host, not as an MCP role. - A user, a database, and a REST endpoint are not MCP roles. - Producer/Consumer/Broker and Frontend/Backend/API are different architectural patterns, not MCP roles.
Proposal. the integration is the server and the host is the client reversed.
Why it attracts. both touch the protocol.
Why it fails. the host application is the client side.
When it would be right. never.
Proposal. MCP is peer-to-peer with no client.
Why it attracts. "both are servers."
Why it fails. MCP is explicitly client-server.
When it would be right. never.
Proposal. it is just a plain REST call.
Why it attracts. the backend is HTTP.
Why it fails. registered as an MCP server, it speaks MCP.
When it would be right. never.
- - Mutation: a custom app uses the same transport but fails. Correct answer: capability negotiation is likely wrong, not the transport. - Mutation: asked which component has which role. Correct answer: host initiates, client implements, server provides.
Two transports - stdio for local, Streamable HTTP for remote
The protocol defines a local transport where the host spawns the server as a subprocess and exchanges JSON-RPC over standard input and output, and a network transport where the server is reached over HTTP. The older standalone SSE transport is treated as deprecated in favor of Streamable HTTP, which carries both request-response and server-push on one endpoint.
Local process communication needs no network stack, so stdio is simplest and lowest latency. Remote servers need an addressable endpoint, so HTTP is used. Both carry the same JSON-RPC envelope.
Boundary. Use stdio when the server runs on the same machine as the client. Use Streamable HTTP when the server is on another machine or behind a gateway. The opposite becomes correct when a "local" server is actually remote: then stdio cannot reach it and HTTP is required.
Recurring specifics. - stdio: command plus optional args and env; no URL. - Streamable HTTP: a url field; optional headers; supports push streaming. - Standalone SSE (/sse) described as the older remote option, now absorbed into Streamable HTTP. - gRPC, raw TCP/UDP, WebSocket, GraphQL are not MCP transports. - A stdio server unreachable from a remote connector because it exposes no HTTP endpoint.
Proposal. WebSocket is a standard transport.
Why it attracts. full-duplex sounds right.
Why it fails. not in the core spec.
When it would be right. never.
Proposal. gRPC is required for high-throughput remote.
Why it attracts. performance framing.
Why it fails. not an MCP transport.
When it would be right. never.
Proposal. Bluetooth or UDP are options.
Why it attracts. "any transport."
Why it fails. explicitly excluded.
When it would be right. never.
Proposal. SSE alone cannot push.
Why it attracts. confusion with the deprecated split.
Why it fails. Streamable HTTP includes push; standalone SSE is the deprecated piece.
When it would be right. never.
- - Mutation: asked best transport for a web IDE to a remote server. Correct answer: Streamable HTTP (or SSE in older framing). - Mutation: asked best for same-machine low latency. Correct answer: stdio. - Mutation: asked about a dropped SSE connection locally. Correct answer: switch to stdio for in-process. - Mutation: asked whether Streamable HTTP supports push. Correct answer: yes, on the same endpoint.
Authentication is transport-specific
For HTTP-based servers, the spec defines an optional OAuth 2.1 authorization framework with protected-resource metadata, PKCE, and resource indicators. A server may instead sit behind a reverse proxy doing bearer-token or TLS-client-cert auth. For stdio servers, the spec says to skip the OAuth flow and read credentials from the environment, because process ownership already provides the trust boundary.
A local subprocess launched by the user already runs with that user's privileges, so a browser-style redirect flow is pointless. A network server has no such implicit trust, so standards-based auth applies.
Boundary. HTTP servers may implement OAuth or sit behind auth-terminating proxies; stdio servers read env credentials. The opposite becomes correct if one asserts OAuth is mandatory for every transport including stdio: it is not, and the spec explicitly excludes stdio from the OAuth flow.
Recurring specifics. - HTTP: OAuth 2.1, bearer tokens in Authorization headers, PKCE, resource indicators. - stdio: credentials from environment; no auth JSON-RPC method. - Corporate SSO typically integrates at the authorization server or proxy. - No authenticate or auth JSON-RPC method exists; initialize only negotiates version and capabilities.
Proposal. OAuth is mandatory for stdio too.
Why it attracts. "always authenticate."
Why it fails. spec excludes stdio from the flow.
When it would be right. never for stdio.
Proposal. embed secrets in tool descriptions.
Why it attracts. "the model needs them."
Why it fails. descriptions are not a credential channel; untrusted text is an injection surface.
When it would be right. never.
Proposal. no authentication since servers are trusted.
Why it attracts. internal network assumption.
Why it fails. remote servers need scoped auth.
When it would be right. never.
- - Mutation: asked how a remote server authenticates many users. Correct answer: OAuth-based authorization. - Mutation: asked how stdio authenticates. Correct answer: environment credentials, no OAuth flow. - Mutation: asked whether an
authJSON-RPC method exists. Correct answer: no.
A stdio server that cannot launch fails to start
For a stdio entry, the host spawns the executable named by command with the given args. If that executable is not resolvable on the system PATH, the spawn fails and the server never connects, so its tools never appear.
The host can only connect to a process it successfully started. A missing executable is a hard startup failure, not a silent degradation.
Boundary. The rule holds for local stdio spawn failures. The nearby opposite is a server that starts but fails capability negotiation or omits the tools capability; there the process runs yet tools are still invisible. Both produce "tools missing," but the cause differs.
Recurring specifics. - command plus optional args required for stdio. - A url is not used for stdio. - Failure yields an error; the server is unavailable. - Restarting the host may help but does not fix a missing command.
Proposal. the client falls back to HTTP transport.
Why it attracts. "it will find another way."
Why it fails. transport is fixed by config.
When it would be right. never.
Proposal. the client prompts for a path.
Why it attracts. interactive recovery.
Why it fails. not how stdio launch works.
When it would be right. never.
Proposal. the client launches a default shell.
Why it attracts. "something runs."
Why it fails. no such default.
When it would be right. never.
- - Mutation: server works for one engineer, not another with identical config. Correct answer: the server process is not running on the second machine. - Mutation: command not on PATH. Correct answer: spawn fails, server unavailable.
The Messages API MCP connector attaches remote servers without owning the client
The platform's API offers a connector that lets an application attach remote MCP servers without implementing its own MCP client. The request carries an mcp_servers array defining each server's URL and optional authorization token, plus an mcp_toolset entry in the tools array that references the server by name and controls which of its tools are enabled. During the turn, the platform infrastructure acts as the MCP client, discovers tools, calls them, and returns results inside the normal agentic loop.
The connector shifts transport and session management off the application. The application still performs the OAuth flow to obtain and refresh the token, but it never speaks the MCP protocol itself.
Boundary. The connector fits remote, HTTP-reachable, tools-only integrations. The opposite becomes correct when the server is a local stdio process or when resources and prompts are needed: then a client-side MCP client (or SDK helpers) is required, because the connector cannot reach a local process and supports only tools.
Recurring specifics. - mcp_servers array, currently url type, optional authorization_token. - mcp_toolset entry in tools references the server; each server needs exactly one toolset to surface its tools. - Requires a beta header in some API forms. - Local stdio cannot be attached; only Streamable HTTP or SSE over public HTTP. - Only tools, not prompts or resources, are supported.
Proposal. declare the server in mcp_servers alone and rely on the connector to enable all tools.
Why it attracts. "declaring is enough."
Why it fails. the toolset entry is what surfaces tools, and each server needs exactly one.
When it would be right. never.
Proposal. point a url entry and toolset at an internal stdio server's address.
Why it attracts. "it is an address."
Why it fails. a stdio process exposes no HTTP endpoint the connector can reach.
When it would be right. never.
Proposal. run a local stdio mirror of the partner server and connect the API to it.
Why it attracts. "avoid network round trips."
Why it fails. the API still cannot reach a local process.
When it would be right. never.
Proposal. re-implement the partner's operations as user-defined tools with client-executed handlers.
Why it attracts. "we control enforcement."
Why it fails. only justified when the app must be the enforcement point, not when a trusted remote server already encapsulates the integration.
When it would be right. precisely that enforcement-point case.
Resources are read-only catalogs, tools are actions
A resource is content the server exposes for the client to read: files, records, schemas, summaries, hierarchies. A tool is a callable action that performs work or queries with parameters. Surfacing a content catalog as resources gives the model visibility into what data exists before it acts.
Without resources, the model must call discovery tools (list_tables, describe_table) just to orient itself, burning calls and context. A resource catalog makes that structure available upfront, in one read.
Boundary. Use resources for static or slowly changing reference content the model reads; use tools for actions and parameterized queries. The opposite becomes correct when the content is genuinely dynamic per query: then a tool that fetches on demand may be appropriate, but a static catalog still reduces orientation cost.
Recurring specifics. - Resources read via resources/read with a URI; listed via resources/list. - Resource objects carry name, description, mimeType, and a URI (or URI template). - Tools invoked via tools/call with an inputSchema. - A product catalogue, quarterly reports, database schema, documentation hierarchy are resource-shaped. - Submitting a ticket, running a query, deploying a service are tool-shaped.
Proposal. expose everything as tools because the model calls uniformly.
Why it attracts. uniformity.
Why it fails. misuse of the abstraction; loses the catalog benefit.
When it would be right. never.
Proposal. expose everything as resources because they come from one server.
Why it attracts. single primitive.
Why it fails. actions are not readable content.
When it would be right. never.
Proposal. dump the whole catalog from one tool.
Why it attracts. "one call gets all."
Why it fails. defeats the catalog pattern and bloats context.
When it would be right. never.
Proposal. a single search_catalog tool instead of resources.
Why it attracts. detailed description helps.
Why it fails. still requires a call to gain visibility; resources give it at connection time.
When it would be right. only if the data is truly dynamic per query.
- - Mutation: asked to reduce exploratory calls. Correct answer: expose content catalog as resources. - Mutation: asked whether a static catalogue should be a resource or tool. Correct answer: resource. - Mutation: asked whether a submit action should be a resource. Correct answer: tool.
Sparse tool descriptions lose to built-in tools
When an MCP tool has a thin description, the model may prefer a built-in tool it understands better (for example a generic search versus a purpose-built search), even when the MCP tool is more capable. Rich, specific descriptions let the model select the MCP tool appropriately.
The model selects among available tools using the descriptions it has. If the MCP tool's description is vague while a built-in's is detailed, the built-in wins by familiarity.
Boundary. Expand descriptions when an MCP tool competes with a built-in for the same job. The opposite becomes correct when the MCP tool is unique with no built-in competitor: then a minimal description may suffice, though detail still helps.
Recurring specifics. - Description should state what the tool does, what it returns, when to use it, and how it compares to built-ins. - Example weak: "Searches code." Example strong: a multi-sentence statement of AST-aware semantic search returning file path, line numbers, and surrounding code, more accurate than text grep for intent-based search. - A table-extraction tool described only as "Extracts tables" loses to the Read tool.
Proposal. restrict allowedTools to exclude the built-in so the model cannot fall back.
Why it attracts. forces MCP use.
Why it fails. hides a generally useful built-in and is blunt.
When it would be right. only as a narrow guardrail.
Proposal. raise the server's priority in config so its tools load first.
Why it attracts. "order matters."
Why it fails. loading order is not the selection mechanism; description quality is.
When it would be right. never.
Proposal. add a hook that re-runs extraction after every Read.
Why it attracts. automates correction.
Why it fails. addresses the symptom, not the description gap.
When it would be right. never.
- - Mutation: asked the most direct fix for fallback behavior. Correct answer: expand the tool description. - Mutation: asked whether priority ranking in config helps. Correct answer: no, description quality is the lever.
Least privilege scopes paths and tool exposure
Each server should receive only the access it needs. A filesystem server should allow specific directories and deny everything else. A server offering both read and write on sensitive data should expose only read tools to a session that needs only reading. Restrictions are enforced at the server level, deterministically, not by hoping the model complies.
A compromised or prompt-injected server acts with whatever access it has. Scoping at the server means even a successful injection is blocked because the server rejects the unauthorized action.
Boundary. Enforce at the server or a shared policy layer. The opposite becomes correct when someone proposes output monitoring or system-prompt instructions as the control: those are unreliable because data can be encoded or split across responses, and the model is not a trusted enforcement point.
Recurring specifics. - Allowed paths: configure specific directories, deny all others by default. - Scoped tool exposure: expose only the read tools to a read-only session. - Docker read-only prevents writes but not reads of sensitive files; path restrictions still needed. - Transport encryption protects in transit but does not limit what the server may read.
Proposal. instruct the model in the system prompt to use only read operations.
Why it attracts. "tell it."
Why it fails. not deterministic; injection can override.
When it would be right. never as the sole control.
Proposal. monitor Claude's output and block sensitive patterns.
Why it attracts. catch leaks.
Why it fails. unreliable; data can be encoded.
When it would be right. only as defense in depth.
Proposal. run in Docker read-only.
Why it attracts. isolation.
Why it fails. still permits reads of sensitive files.
When it would be right. as a complement, not a replacement.
- - Mutation: asked what prevents an injection from reading sensitive files. Correct answer: least-privilege path scoping at the server. - Mutation: asked how to give read-only access. Correct answer: expose only read tools server-side. - Mutation: asked whether output filtering suffices. Correct answer: no, server-level control is deterministic.
Third-party servers are an injection surface
A server from an untrusted source may carry hidden instructions in its tool descriptions or return poisoned content in its results, steering the model to exfiltrate data or take unintended actions. The correct posture is to vet the server, restrict its scope and permissions, and treat its output as untrusted.
Tool descriptions and results are text the model ingests. If an adversary controls that text, they control a prompt-injection channel ("tool poisoning"). Trust must be earned by review, not assumed from the protocol.
Boundary. Treat untrusted servers as hostile input. The opposite becomes correct only for a server you have reviewed and scoped: there, you may rely on it within its least-privilege bounds, but you still do not grant it more than needed.
Recurring specifics. - Review source code, especially what directories and APIs it touches. - Configure minimal permissions. - "It speaks MCP" is not evidence of safety. - Hidden instructions in a description telling the model to exfiltrate is the canonical example.
Proposal. MCP servers can never be malicious.
Why it attracts. protocol feels safe.
Why it fails. the protocol is neutral; the implementation is not.
When it would be right. never.
Proposal. grant full access to save setup time.
Why it attracts. speed.
Why it fails. maximizes blast radius.
When it would be right. never.
Proposal. raise model temperature to ignore instructions.
Why it attracts. "confuse it."
Why it fails. temperature is not a security control.
When it would be right. never.
- - Mutation: asked the concern and mitigation. Correct answer: tool-poisoning risk; vet and restrict. - Mutation: asked whether "it is MCP" implies safety. Correct answer: no.
Tools must be advertised during capability negotiation
After connection, the client sends initialize; the server replies with its protocol version and a capabilities object declaring which primitives it supports (tools, resources, prompts). If the server does not include tools in capabilities, or fails to start, the model never sees its tools.
Discovery is explicit. The model can only call tools it has been told about through the negotiated capability set.
Boundary. If a server runs but omits the tools capability, tools are invisible. The opposite becomes correct when the server is simply not running: then no negotiation occurs at all. Both yield missing tools; diagnosis differs.
Recurring specifics. - initialize exchanges version and capabilities. - A server advertises tools by including "tools": {} (or a non-empty object). - resources and prompts are separate capability groups; a server can offer tools without prompts. - Restarting the host may help but does not fix a server that errors during init or omits capabilities.
Proposal. tools are auto-detected without negotiation.
Why it attracts. "it just works."
Why it fails. the server must advertise them.
When it would be right. never.
Proposal. authentication happens in initialize.
Why it attracts. first message.
Why it fails. initialize negotiates version and capabilities, not user auth.
When it would be right. never.
Proposal. prompts are discovered inside tools/list.
Why it attracts. combined listing.
Why it fails. prompts are a separate capability advertised at init.
When it would be right. never.
- - Mutation: asked why tools are missing though config is correct. Correct answer: server error or missing tools capability. - Mutation: asked where prompts capability is set. Correct answer: initialize handshake. - Mutation: asked what
initializedoes. Correct answer: exchange capabilities and protocol version.
Resource templates use parameterized URIs
A server can expose a resource template with a URI pattern containing placeholders, for example invoice://{customer_id} or logs://{app_id}. The client resolves the template to a concrete URI at read time and the server returns the matching content.
Not all resources are known at startup. Parameterized templates let a server address an unbounded set of resources (per customer, per document) without registering each one.
Boundary. Use templates for parameterized, resolved-at-read content. The opposite becomes correct when the set is fixed and small: then static resources registered at startup suffice. Templates are for the open-ended case.
Recurring specifics. - RFC 6570-style URI templates. - Example: files://documents/{docId}, invoice://{customer_id}. - The host resolves the template to a concrete URI when reading. - A static catalog cannot represent per-customer data without templates.
Proposal. resources cannot take parameters and must be fully static.
Why it attracts. "resources are data."
Why it fails. templates exist for parameterization.
When it would be right. never.
Proposal. use a tool whose URI argument the model supplies.
Why it attracts. tools take args.
Why it fails. that is a tool, not a resource template, and loses the catalog benefit.
When it would be right. only if the data is action-shaped.
Proposal. register a separate static resource per customer at startup.
Why it attracts. "fixed URIs."
Why it fails. does not scale to unbounded sets.
When it would be right. only for a known small set.
- - Mutation: asked how to address per-customer data. Correct answer: resource template with parameterized URI. - Mutation: asked whether resources are always static. Correct answer: no, templates parameterize.
Resource subscriptions push change notifications
A client sends resources/subscribe with a resource URI. The server adds the client to a subscriber list and, when that resource changes, sends a notifications/resources/updated JSON-RPC notification. The client can then re-read. This eliminates polling.
Push notifications let the model learn about changes without repeatedly querying. The notification is one-way and requires no response.
Boundary. Subscriptions require server support for resources/subscribe. The opposite becomes correct when the server does not support subscriptions: then the client must poll resources/list or re-read on a schedule, accepting staleness or overhead.
Recurring specifics. - Subscribe via resources/subscribe with the URI. - Server sends notifications/resources/updated. - Used for live metrics, document-edit notifications, theme changes. - Not a financial "subscription"; not bookmarking; not offline caching.
Proposal. increase cache TTL to reduce polling.
Why it attracts. fewer reads.
Why it fails. introduces staleness; does not provide push.
When it would be right. never as the primary mechanism.
Proposal. convert metrics to tools for callbacks.
Why it attracts. "tools are active."
Why it fails. tools are for actions, not streaming updates.
When it would be right. never.
Proposal. there is a batch endpoint returning all resources.
Why it attracts. "one call."
Why it fails. no such endpoint; resources/list returns metadata, not content.
When it would be right. never.
- - Mutation: asked how to get notified of a change. Correct answer: subscribe, receive notification. - Mutation: asked what a
resources/updatednotification means. Correct answer: content changed, refresh.
Roots define directory boundaries
Roots let the client communicate to the server the boundaries within which it may operate, typically permitted directories. A filesystem tool constrained to authorized directories expresses those boundaries as roots.
Roots make the server's operating scope explicit and client-communicated, supporting least privilege without hardcoding paths inside the server.
Boundary. Roots scope where a server may act. The opposite becomes correct when the boundary is expressed instead as an allow-list of tool names: that controls which tools, not which directories. The two are complementary, not the same concept.
Recurring specifics. - Roots are distinct from tools, prompts, and sampling. - Used to limit filesystem tool operation to authorized directories. - Communicated from client to server.
Proposal. prompts express boundaries.
Why it attracts. "prompts guide."
Why it fails. prompts are templates, not boundaries.
When it would be right. never.
Proposal. sampling expresses boundaries.
Why it attracts. confusion of terms.
Why it fails. sampling requests a completion.
When it would be right. never.
Proposal. stop sequences express boundaries.
Why it attracts. "they halt."
Why it fails. unrelated to MCP scope.
When it would be right. never.
- - Mutation: asked which concept bounds directory access. Correct answer: roots. - Mutation: asked to distinguish from prompts. Correct answer: roots are boundaries, prompts are templates.
Sampling lets a server request a model completion
MCP defines a sampling feature where a server can request an LLM completion through the client. This lets server-side logic leverage the model while keeping model access mediated by the host, rather than the server calling a model API directly.
Sampling enables server-side agentic behavior without the server needing its own model credentials or a direct API relationship.
Boundary. Sampling is the server-requested completion. The opposite becomes correct when the model is the one initiating a tool call: that is ordinary tool use, not sampling. The direction of the request distinguishes them.
Recurring specifics. - Distinct from roots, resources, transport. - Mediated by the host client. - Enables server-side reasoning steps.
Proposal. sampling is the same as roots.
Why it attracts. similar vocabulary.
Why it fails. different concepts.
When it would be right. never.
Proposal. sampling is the transport channel.
Why it attracts. "sampling over transport."
Why it fails. transport carries messages; sampling is a capability.
When it would be right. never.
- - Mutation: asked what lets a server use the model. Correct answer: sampling. - Mutation: asked to distinguish from resources. Correct answer: sampling is a completion request, resources are readable data.
Messages are JSON-RPC 2.0
MCP encodes every client-server message as JSON-RPC 2.0: request/response pairs with method names and IDs, plus notifications. The transport (stdio or HTTP) only carries the envelope.
A single, simple message format keeps the protocol implementation uniform across transports and languages.
Boundary. JSON-RPC is the envelope; the transport is separate. The opposite becomes correct if one conflates them: saying "MCP uses REST" or "MCP uses gRPC" describes the wrong layer.
Recurring specifics. - JSON-RPC 2.0 for all messages. - Methods like tools/call, resources/read, initialize, notifications/resources/updated. - Not HTML forms, not raw binary opcodes, not CSV.
Proposal. MCP uses REST with GET/POST.
Why it attracts. HTTP is involved remotely.
Why it fails. the message format is JSON-RPC, not REST semantics.
When it would be right. never.
Proposal. MCP uses gRPC with protobufs.
Why it attracts. performance.
Why it fails. not the format.
When it would be right. never.
Proposal. MCP uses GraphQL.
Why it attracts. "queries."
Why it fails. not the format.
When it would be right. never.
- - Mutation: asked the format. Correct answer: JSON-RPC 2.0. - Mutation: asked whether transports change the format. Correct answer: no, format is constant; transport varies.
Tool results return as tool_result blocks
The client sends a tool call to the MCP server via JSON-RPC. The server executes and returns the result. The client wraps that result as a tool_result content block carrying the matching tool_use_id, and includes it in the next Messages API request so the model can read it.
The model never talks to the server directly; all communication passes through the client, which bridges MCP results into the model's tool-result convention.
Boundary. Results flow client-mediated as tool_result. The opposite becomes correct if one claims the server sends results straight to the model API: it cannot, because the server has no direct API connection.
Recurring specifics. - Client sends call; server returns result; client wraps as tool_result with tool_use_id. - A resource can be returned inline with a uri and text or blob. - Results are synchronous via JSON-RPC response, not a shared database.
Proposal. the server sends the result directly to the API.
Why it attracts. "faster."
Why it fails. no direct connection; violates architecture.
When it would be right. never.
Proposal. the server writes results to a shared database the client polls.
Why it attracts. "decoupled."
Why it fails. not how MCP works; adds latency.
When it would be right. never.
Proposal. the server converts the result to a resource URI for separate fetch.
Why it attracts. "resource-like."
Why it fails. unnecessary round-trip; the result is the synchronous return.
When it would be right. never.
- - Mutation: asked how results reach the model. Correct answer: client wraps as
tool_result. - Mutation: asked whether the server talks to the API directly. Correct answer: no, through the client.
Many servers at startup inflate initialization
When a host initializes, it may start every configured server concurrently. With many servers, that is many process spawns and initialization handshakes, producing a slow startup.
Each server is a process with startup cost. Multiplying that cost by the server count dominates startup time.
Boundary. Consolidate related functionality or lazy-load. The opposite becomes correct if one asserts a hard protocol limit on server count: there is none; the issue is overhead, not a cap.
Recurring specifics. - Fifteen servers took startup from seconds to nearly a minute in one framing. - No hard maximum in the protocol. - Fix: consolidate related tools into fewer servers, or lazy initialization. - Config-file parsing is negligible; process startup is the bottleneck.
Proposal. MCP supports at most five servers.
Why it attracts. "performance implies a limit."
Why it fails. no such limit; the issue is overhead.
When it would be right. never.
Proposal. the config file is too large to parse.
Why it attracts. file size.
Why it fails. parsing is negligible versus spawn cost.
When it would be right. never.
Proposal. move all to localhost.
Why it attracts. "network is the cause."
Why it fails. local spawns were already the cause.
When it would be right. never.
- - Mutation: asked the likely cause of slow startup. Correct answer: each server starts on load. - Mutation: asked the fix. Correct answer: consolidate or lazy init. - Mutation: asked whether a cap exists. Correct answer: no, overhead is the issue.
Too many tools degrade selection
When an agent has many simultaneously available tools, the context consumed by tool schemas grows and the model has more options to confuse, lowering selection accuracy. the tested material cites a measurable drop beyond roughly five or six tools from a single server.
Selection quality degrades as the tool set grows because the model must distinguish more schemas and choose correctly under that load.
Boundary. Curate tool exposure. The opposite becomes correct when the combined set is genuinely needed and scoped per workflow: then selective loading by query domain, not blanket loading, is the mitigation. The principle is "relevant only," applied across multiple sources as well as within one.
Recurring specifics. - A single server exposing fifteen tools caused selection errors. - Fix: a curated server exposing only relevant tools. - The principle extends to combined built-in plus multiple servers. - allowedTools and toolset entries control exposure.
Proposal. authentication is failing intermittently.
Why it attracts. errors appear.
Why it fails. auth failures are explicit, not subtle selection mistakes.
When it would be right. never.
Proposal. protocol version incompatible.
Why it attracts. "something is wrong."
Why it fails. incompatibility prevents connection, not selection.
When it would be right. never.
Proposal. tool names must follow a strict convention.
Why it attracts. "naming fixes it."
Why it fails. no such requirement; it is a count problem.
When it would be right. never.
- - Mutation: asked the likely cause of wrong tool selection. Correct answer: too many tools. - Mutation: asked the fix. Correct answer: curate to relevant tools. - Mutation: asked whether the principle applies across servers. Correct answer: yes, relevant-only loading applies broadly.
The disabled flag keeps config but inactive
A server entry can carry a disabled field set to true. The host then keeps the configuration present but does not start or connect to that server.
This supports temporarily disabling a server without deleting its definition, useful for debugging or staged rollout.
Boundary. disabled preserves config while preventing connection. The opposite becomes correct when one wants the server active: then omit or set false. It is not a tool-hiding mechanism for individual tools; that is the allow-list.
Recurring specifics. - "disabled": true prevents start and connection. - Config remains in the file. - Distinct from allow-list tool restriction.
Proposal. disabled removes the server from configuration.
Why it attracts. "disabled means gone."
Why it fails. config remains, just inactive.
When it would be right. never.
Proposal. disabled hides the server from the UI permanently.
Why it attracts. "hidden."
Why it fails. it is an operational toggle, not a UI preference.
When it would be right. never.
An allow-list restricts visible tools
A server entry can specify an allow-list of tool names, optionally with patterns like dev_*. Only those tools appear in the tools/list response, so the model never sees the others and cannot call them.
Allow-listing gives precise control over which capabilities a server exposes to the model, supporting least privilege at the tool level.
Boundary. Allow-list hides tools from the model. The opposite becomes correct when the goal is to bound directories: that is roots, not an allow-list. The two controls address different surfaces.
Recurring specifics. - Property name cited as toolAllowList with pattern support. - Only listed tools appear; unlisted ones are invisible to the model. - No standard toolDenyList; prefer allow-list. - Patterns like mcp__playwright__* in SDK allowedTools.
Proposal. an unlisted tool is blocked at call time but still visible.
Why it attracts. "it fails when called."
Why it fails. with an allow-list the model never sees it, so it will not try.
When it would be right. never.
Proposal. the server rejects the call.
Why it attracts. "server enforces."
Why it fails. the client hides it before listing.
When it would be right. only if the server independently enforces.
Proposal. a toolDenyList is the standard property.
Why it attracts. "deny is natural."
Why it fails. not the cited standard; allow-list is preferred.
When it would be right. never.
- - Mutation: asked what happens to a tool not in the allow-list. Correct answer: the model never sees it. - Mutation: asked the property name. Correct answer:
toolAllowList(flag uncertain exact key; verify against current spec).
Remote SSE requires CORS headers
When a browser-based client connects to a remote MCP server over SSE, the server's SSE responses must include CORS headers permitting the client origin, or the browser blocks the connection.
Browsers enforce CORS for cross-origin requests. An SSE stream is still an HTTP response subject to that policy.
Boundary. CORS is needed for browser-based remote SSE. The opposite becomes correct for a non-browser client or a stdio server: there, CORS does not apply. A firewall failure produces timeout or refused errors, not a CORS error, which is the diagnostic tell.
Recurring specifics. - CORS headers required on SSE responses. - CORS error distinguishes misconfiguration from network failure. - HTTPS is recommended but does not fix CORS. - stdio has no CORS concern.
Proposal. switch to stdio for all deployments.
Why it attracts. "avoid CORS."
Why it fails. stdio is local only; remote needs HTTP.
When it would be right. only for local.
Proposal. a firewall causes the CORS error.
Why it attracts. "connection blocked."
Why it fails. firewall yields timeout/refused, not CORS.
When it would be right. never.
Proposal. add HTTPS to fix CORS.
Why it attracts. "secure fixes it."
Why it fails. HTTPS is separate from CORS.
When it would be right. never.
- - Mutation: asked the cause of a CORS error. Correct answer: missing CORS headers. - Mutation: asked whether HTTPS fixes it. Correct answer: no, CORS is separate.
Missing environment variables fail fast
When a required environment variable referenced by expansion is unset, the server should fail at startup with a clear message identifying the missing variable, rather than starting in an invalid state that fails unpredictably later.
Silent defaults or ignored variables create hard-to-debug production incidents. A clear early failure points directly at the misconfiguration.
Boundary. Fail fast with a clear diagnostic. The opposite becomes correct only if a genuine default is safe and intended: then a default may be acceptable, but the tested material frames silent invalid starts as wrong.
Recurring specifics. - ${DATABASE_URL} unset should abort with a clear error. - Avoids runtime failures with no diagnostic. - Expansion resolves from the developer's environment; if unset, empty or literal value causes auth failure.
Proposal. start successfully ignoring missing variables.
Why it attracts. "keep running."
Why it fails. hides misconfiguration.
When it would be right. never.
Proposal. use silent placeholder defaults leading to invalid runtime state.
Why it attracts. "no crash."
Why it fails. produces confusing later failures.
When it would be right. never.
Proposal. hardcode variables to avoid expansion.
Why it attracts. "guaranteed value."
Why it fails. commits secrets and removes per-developer flexibility.
When it would be right. never.
- - Mutation: asked how to handle a missing variable. Correct answer: fail fast with a clear message. - Mutation: asked why a token fails for one dev. Correct answer: they never set the variable.
Keep the server thin and stateless
An MCP server should be an interface layer over existing systems, translating their capabilities into tools, resources, and prompts, rather than becoming a stateful application tier that owns business logic and its own datastore.
The protocol's value is standardized access. Embedding substantial business logic and state turns the server into yet another application to maintain, contradicting the "thin wrapper" intent.
Boundary. Keep it thin. The opposite becomes correct only when the required control (masking, allow-lists, credential checks) genuinely cannot live in the wrapped system and must be in the server: there, the server carries that logic, but still as a focused wrapper, not a general stateful tier.
Recurring specifics. - Server handles auth, transformation, error handling once. - Exposes clean tool interfaces. - Avoids caching each client session centrally. - Wraps existing systems rather than reimplementing them.
Proposal. make the server fully stateful to cache sessions.
Why it attracts. "performance."
Why it fails. protocol expects it as an interface layer, not a stateful tier.
When it would be right. never as a general rule.
Proposal. embed all logic and expose one tool.
Why it attracts. "fewer tools."
Why it fails. fewer tools is not the goal; a monolithic tool loses structured selection and the wrapper stays bloated.
When it would be right. never.
Proposal. move all logic client-side, server forbidden from computing.
Why it attracts. "thin client."
Why it fails. misreads the division; the server does the integration work.
When it would be right. never.
- - Mutation: asked the recommended design. Correct answer: thin, mostly stateless wrapper. - Mutation: asked whether to embed logic. Correct answer: no, wrap existing systems.
Slash commands can invoke MCP tools
A slash command's prompt can instruct the agent to use any tool already available in its environment, including those from a configured MCP server. As long as the server is loaded globally or per project, the command can rely on its tools.
Slash commands orchestrate the agent's available capabilities; MCP tools are part of that capability set.
Boundary. A command can use MCP tools if the server is configured. The opposite becomes correct if one claims MCP tools cannot appear in slash commands: they can, provided the server is loaded.
Recurring specifics. - Define the server via project settings or a config flag, then write the command to invoke the tool. - No need to embed the server binary in the commands folder. - MCP tools are not forbidden in commands.
Proposal. MCP tools cannot be used in slash commands.
Why it attracts. "separate systems."
Why it fails. commands leverage available tools.
When it would be right. never.
Proposal. embed the server binary in the commands folder.
Why it attracts. "self-contained."
Why it fails. not how configuration works.
When it would be right. never.
Proposal. define the server only in the command file frontmatter.
Why it attracts. "local to command."
Why it fails. configuration belongs in the MCP config surface.
When it would be right. never.
- - Mutation: asked how to make a command use an MCP tool. Correct answer: load the server, instruct the command. - Mutation: asked whether it is possible. Correct answer: yes.
System prompts and CLAUDE.md are not MCP config surfaces
MCP server registration happens in the MCP configuration files (project or user), not in instruction text such as a system prompt or a CLAUDE.md file. Putting server definitions in prose does not register them.
Registration requires a structured configuration the host reads at startup. Prose is instruction content, not a server registry.
Boundary. Server definitions go in config files. The opposite becomes correct only in the limited sense that a system prompt can guide the model to use an already-registered tool; it cannot register the server.
Recurring specifics. - CLAUDE.md is for instructions, not server registration. - System prompts carry credentials is wrong; credentials belong in server/env config. - .mcp.experimental.json is not a recognized filename; only .mcp.json is read.
Proposal. put the server in CLAUDE.md under a tools heading.
Why it attracts. CLAUDE.md is per-project.
Why it fails. not a registration surface.
When it would be right. never.
Proposal. put credentials in the system prompt.
Why it attracts. "model needs them."
Why it fails. credentials belong at the infrastructure layer; the model need not see raw keys.
When it would be right. never.
Proposal. a .mcp.experimental.json file.
Why it attracts. "named for experiments."
Why it fails. not a recognized file; only .mcp.json is read.
When it would be right. never.
- - Mutation: asked where to register a server. Correct answer:
.mcp.jsonor~/.claude.json. - Mutation: asked whether CLAUDE.md registers servers. Correct answer: no.
The Agent SDK attaches in-process custom servers
The Agent SDK can register MCP servers under an mcpServers option, where a server may be a stdio entry launched by command, an HTTP entry, or a custom in-process server implemented directly in the SDK. Its tools then plug into the same allowedTools pattern as built-in tools.
First-class MCP integration lets locally defined and MCP tools coexist in one agent configuration, giving a unified tool ecosystem.
Boundary. SDK supports stdio, HTTP, and in-process custom servers. The opposite becomes correct if one claims MCP tools require a separate SDK or only work with one model: they do not; integration is model-agnostic and unified.
Recurring specifics. - Register under mcpServers; tools plug into allowedTools. - mcp__<server>__<tool> naming applies. - Local and MCP tools coexist. - Model-agnostic.
Proposal. MCP tools need a separate SDK.
Why it attracts. "different system."
Why it fails. built-in MCP integration.
When it would be right. never.
Proposal. only local tools are supported.
Why it attracts. "SDK is local."
Why it fails. MCP is supported.
When it would be right. never.
Proposal. only one model family supports MCP.
Why it attracts. "specific model."
Why it fails. model-agnostic.
When it would be right. never.
- - Mutation: asked whether MCP and local tools coexist. Correct answer: yes. - Mutation: asked how tools become available. Correct answer: register under
mcpServers, plug intoallowedTools.
Remote localhost is resolved against the client machine
A server URL of http://localhost:3000 in a shared project file refers to the machine running the client. When a teammate clones the repo, the server must be running on their own machine; if it is not, the connection fails even though the config is correct.
localhost is always the local machine from the perspective of the process using it. The configuration is portable; the running process is not.
Boundary. localhost means the client's machine. The opposite becomes correct when the server is deployed on a remote host with a real network address: then the URL points there, not to localhost.
Recurring specifics. - localhost resolution is per client machine. - Config is fine; missing local process is the cause. - Re-committing or moving to user config does not fix a missing process.
Proposal. .mcp.json does not support localhost URLs.
Why it attracts. "maybe restricted."
Why it fails. localhost is a normal URL.
When it would be right. never.
Proposal. add to user config to activate.
Why it attracts. "scope issue."
Why it fails. scope is fine; the local server is not running.
When it would be right. never.
Resource MIME type declared in listing
When a server lists resources, each entry may include a mimeType field (for example application/json or text/markdown) that helps the client interpret the content. The MIME type is part of the resource metadata, not the tool schema or the config file.
Declaring the content type lets the client render or parse the resource appropriately without guessing.
Boundary. MIME type belongs in the resource listing. The opposite becomes correct if one looks for it in the tool definition or .mcp.json: it is not there.
Recurring specifics. - mimeType in the resource listing. - Examples: application/json, text/markdown. - A size field is not standardized; it may be added as custom metadata. - Resource read returns content plus mimeType.
Proposal. MIME type is in the tool definition.
Why it attracts. "tools carry schema."
Why it fails. resources are separate from tools.
When it would be right. never.
Proposal. size is a required field.
Why it attracts. "resources have size."
Why it fails. not standardized; optional custom metadata.
When it would be right. never.
Resources are inherently read-only
In the current protocol, resources are accessed only through resources/read and resources/list. There is no write method for resources, so they are conceptually read-only assets.
The protocol defines the resource primitive as readable content. Mutating content is the job of a tool that performs an action.
Boundary. Resources are read-only by construction. The opposite becomes correct when the goal is to change state: that is a tool, not a resource. The read-only nature is enforced by the absence of a write method, not by a flag.
Recurring specifics. - No write method for resources. - Read via resources/read; listed via resources/list. - Mutating content is a tool responsibility. - A readOnly flag is not the enforcement mechanism; absence of write is.
Proposal. set a readOnly flag to enforce read-only.
Why it attracts. "explicit flag."
Why it fails. enforcement is by lacking a write method, not a flag.
When it would be right. never.
Proposal. use a special URI scheme for read-only.
Why it attracts. "scheme signals it."
Why it fails. scheme does not enforce semantics.
When it would be right. never.
Authorize multi-path resources at a shared layer
When the same underlying content is reachable through more than one transport (for example MCP and direct REST), authorization must be enforced in the resource service or a shared trusted policy layer that every adapter uses, with each adapter propagating the effective principal. Per-adapter authorization creates bypass and policy drift.
If one path enforces access and another does not, the same confidential content is exposed through the unguarded route. A single enforcement point keeps decisions consistent.
Boundary. Enforce once at a trusted layer. The opposite becomes correct only if a single adapter is the sole access path: then authorizing there suffices, but the moment a second path exists, shared enforcement is required.
Recurring specifics. - Authorize in the resource service or shared policy layer. - Each adapter propagates the effective principal. - Do not let the model decide entitlement from conversation context; it is not a trusted source. - Per-adapter authorization leaves bypass routes.
Proposal. let the model decide from context whether the user should see the file.
Why it attracts. "it understands intent."
Why it fails. conversation context is not a trusted entitlement source.
When it would be right. never.
Proposal. authorize only in the MCP server.
Why it attracts. "model clients are riskiest."
Why it fails. the REST path bypasses that decision.
When it would be right. only if MCP is the sole path.
Proposal. authorize only in the REST gateway.
Why it attracts. "MCP uses HTTP anyway."
Why it fails. the MCP adapter is a different route; transport-specific auth drifts.
When it would be right. only if REST is the sole path.
- - Mutation: asked where to enforce. Correct answer: shared trusted layer. - Mutation: asked whether model context suffices. Correct answer: no.
The connector supports only tools, not prompts or resources
The Messages API MCP connector surfaces a remote server's tools through an mcp_toolset entry, but it does not surface that server's prompts or resources. The server must be publicly reachable over HTTP (Streamable HTTP or SSE). Local stdio servers cannot be attached.
The connector is scoped to the tool-calling portion of the protocol and shifts transport and session management to the platform. Prompts and resources are out of scope for this mechanism.
Boundary. Connector equals tools-only over public HTTP. The opposite becomes correct when prompts or resources are needed, or when the server is local: then a client-side MCP client (or SDK helpers) is required.
Recurring specifics. - Tools only; no prompts or resources via the connector. - Public HTTP required (Streamable HTTP or SSE). - Local stdio cannot be attached. - Each server referenced by exactly one toolset.
Proposal. the connector supports resources too.
Why it attracts. "it is MCP."
Why it fails. the connector is tools-only.
When it would be right. never.
Proposal. a local stdio server can be attached via url entry.
Why it attracts. "point at it."
Why it fails. no HTTP endpoint exists for a stdio process.
When it would be right. never.
Proposal. declaring the server alone enables its tools.
Why it attracts. "declaration suffices."
Why it fails. the toolset entry is required.
When it would be right. never.
- - Mutation: asked what the connector supports. Correct answer: tools only. - Mutation: asked whether local stdio works. Correct answer: no, public HTTP required.
Distinctions that decide answers
| This | Not this | How to tell them apart |
|---|---|---|
| Project dot mcp dot json | User tilde slash dot claude dot json | Project file is at the repository root, version controlled, and shared by the team. User file is in the home directory, personal, and never shared. |
| Dollar brace ENV_VAR expansion in dot mcp dot json | Literal secret in dot mcp dot json | Expansion keeps the shared file credential free and lets each developer authenticate locally. A literal secret leaks to every clone and build history. |
| Community MCP Server for Jira or GitHub | Custom Server as first step | Community options are already maintained, tested, and updated for standard systems and should be evaluated first. Custom is reserved for team specific logic or proprietary systems with no community equivalent. |
| Resources as content catalogues | Tools as actions | Resources expose read only data such as schema or issue lists that the host fetches for context before tool choice. Tools mutate or query with side effects and are model controlled. |
| Enhanced MCP description that states output shape | Sparse MCP description such as Searches code | The enhanced description supplies capability, input format, output shape, and when to prefer this tool over Grep or sibling servers. The sparse one leaves the model to prefer the better described built in tool. |
Traps
The tempting answer. Propose a custom integration for Jira or GitHub immediately because custom sounds more tailored to the team's sprint model.
Why it fails. Evaluated community Servers for Jira, GitHub, Slack, Linear, and Notion already cover standard workflows, receive updates, and remove build and maintenance cost, so the exam consistently prefers evaluation before build and marks custom first as the wrong priority.
What is correct. Evaluate the community Server against the required fields and operations first, adopt it when it covers the scope, and reserve custom for team specific workflow fields or proprietary systems with no community equivalent.
The tempting answer. Store team wide servers in tilde slash dot claude dot json so each developer maintains their own connection.
Why it fails. That file is personal and not version controlled, so teammates do not inherit the same toolset on clone and the team drifts, while the repository root file is the correct shared model with automatic inheritance on pull.
What is correct. Place shared servers in dot mcp dot json at the repository root and keep only experimental personal servers in the home file.
The tempting answer. Paste the GITHUB_TOKEN value directly into dot mcp dot json because pasting feels simpler than managing environment injection.
Why it fails. That value then lives in every cloned history and every CI log, exposing the credential, whereas expansion keeps the JSON credential free while each environment injects its own secret and rotation requires no JSON edit.
What is correct. Write dollar brace GITHUB_TOKEN in the JSON and inject the actual token via shell profile or secret manager at the developer and CI layer.
The tempting answer. Leave the MCP entry as Searches code because the name feels self documenting and longer prose feels redundant.
Why it fails. Model selection at tool_choice auto is driven by available text, so a sparse MCP entry loses to a well described built in such as Grep even when the MCP index is semantically richer, and selection only stabilises after an enriched description.
What is correct. Rewrite the description to state AST aware semantic capability, returned shape with file path and line numbers, and when to prefer this tool over Grep or sibling servers.
The tempting answer. Discover an unfamiliar backend by iterating list_tables then describe_table for every table because iterative calls feel explicit.
Why it fails. That burns multiple turns purely to get bearings, while a Resource catalogue exposes schema with table names, column types, and relationships in one read only fetch that the host injects before tool choice, preserving budget for substantive calls.
What is correct. Expose the catalogue as a db colon slash slash style Resource with declared mimeType and consume it as context before invoking domain tools.
Built-in Tools
Use Grep for inside, Glob for names, and Edit with a unique anchor before Read plus Write, all through incremental discovery rather than bulk reading.
What you need to know
Claude Code includes six built in tools for codebase work, Read, Write, Edit, Bash, Grep, and Glob, and using the right one for the task is a source of measured productivity rather than style. The workflow the exam probes is codebase comprehension and modification at scale, where choosing between Grep and Glob, or between Edit and Read plus Write, determines whether context is spent well or burned, and whether results are complete or truncated. The distinctions are small in name but large in consequence, because each tool emits a different shape of evidence and costs a different amount of the context budget.
The sharpest distinction the task tests is between Grep and Glob. Grep searches file contents for patterns such as function names, import strings, error messages, and variable assignments. It answers what is inside files. Glob matches file paths by naming patterns such as extension or directory shape, answering which files exist by name without opening them. The sentence that anchors the difference is simple. Grep finds what is inside files, Glob finds files by their names. The exam deliberately presents scenarios where the wrong choice produces a null result or an expensive misfire. Using Glob to find callers of a deprecated function fails entirely, because Glob can only match star star slash star processLegacyOrder star as a file name pattern, not the string processLegacyOrder inside OrderProcessor dot ts. Using Grep to find test files by naming suffix works technically by searching content for the literal test substring, but is the wrong tool because Glob is purpose built for star star slash star dot test dot tsx style path matching and the exam expects that choice.
The file write family has an analogous preferred order. Edit performs a targeted modification via unique text matching, where you supply the precise old_string and its replacement and the tool touches only that occurrence. When old_string appears multiple times in the file, Edit fails by design rather than guessing which occurrence was meant, which is a safety property rather than a bug. When that happens the documented recovery is to widen the anchor with more surrounding context until the match becomes unique, or to set replace_all true if every occurrence should be changed within the same turn. Both options keep you on Edit, which costs almost no additional context, while falling back to Read plus Write, loading the entire file and rewriting it, is a context heavy fallback that the exam treats as a last resort rather than the immediate next step, because it spends a file's worth of tokens to change a line. The ordering therefore matters. Attempt Edit with the shortest plausible unique anchor, widen until it is unique on a non unique match or set replace_all where a global change was intended, and only escalate to Read plus Write when neither narrowing path can disambiguate the target.
The discovery shape around these tools matters as much as the tools themselves. The correct orientation is incremental discovery rather than eager bulk reading. Grep first for entry points such as the function name, class name, or error string that anchors the investigation to learn which files are relevant, then Read those specific files to follow imports and trace flows, then Grep again for any wrapper or re export names to find indirect consumers, and only Read what each prior discovery step justified. The opposite shape, loading every file into context before knowing what is relevant, is the highest cost discovery failure in this domain, because a two hundred file codebase read upfront can exhaust the window largely on files with nothing to do with the task at hand.
Tracing through wrapper modules is a specific instance of that shape. When a function is defined in one module, re exported through a wrapper or a barrel file such as index dot ts, a naive Grep for the original name misses every consumer who imports the wrapper name, so the correct trace is to Grep for the definition to find the defining file, Read the defining file to identify exported and re exported names, Grep for each of those names across the codebase, and if a barrel file is involved, Grep for the barrel module's import path to include consumers that never mention the function directly. That layered trace is what the exam contrasts with Glob first or Bash for both steps when built ins would suffice, and with bulk Read before any selection.
The deprecation scenario brings these choices together into the canonical examined sequence. To find every file that calls a deprecated function and every test that covers those files, the correct order is Grep for the function name to locate callers, including tests that import it directly because call content search surfaces them, then Glob for the sibling test of each caller file, such as star star slash OrderProcessor dot test dot star for OrderProcessor dot ts, to capture tests that cover the function transitively through the source module without mentioning it by name, then a further Grep for any wrapper or alias name under which the caller exposes the function, catching remaining tests that covered it through that alias. That Grep then Glob then Grep again pattern, content for direct references, path for adjacent tests, content for indirect coverage, is a structured trace that the exam contrasts with Glob first, Bash for both steps when built ins would suffice, and bulk Read before any selection.
Grep versus Glob by evidence shape
Grep accepts a pattern and an optional path scope and emits matches with file path, line number, and line content, which is why search plus Read is sufficient to reconstruct the call sites for the deprecation scenario without an intermediate Bash pipeline. Glob accepts a pattern such as star star slash star dot test dot tsx or star star slash config dot star scoped to the working tree and emits matching paths without reading contents. The evidence shapes are therefore complementary. Grep tells you where a string occurs inside files, Glob tells you which files exist by naming convention, and the question wording itself tells you which you need, inside versus named.
That complementarity explains the two most common misfires. Using Glob to find callers searches path names for a function name that lives inside files and returns nothing, because the name is not in the path. Using Grep to find files by extension searches content for the literal substring test rather than matching the path pattern star star slash star dot test dot tsx, which is technically possible yet imprecise and slower compared with Glob which matches the extension directly without opening contents.
Edit with widened anchor versus Read plus Write
Edit accepts old_string and new_string plus an optional replace_all and returns success or a structured non unique match error naming the count of locations, which the caller handles by supplying a longer surrounding anchor that includes unique neighbouring lines, comments, or function signatures, and Write accepts a full path and content payload and replaces or creates the file, which is why its token cost scales with file size on every path. Read accepts a file path and an optional range and returns the requested window with no evaluation of contents beyond text delivery, requiring the caller to supply surrounding context when Edit needs a wider anchor.
The cost ordering is therefore attempt Edit with the shortest plausible unique anchor, on non unique failure widen until unique or set replace_all true where a global change was intended, and only then escalate to Read plus Write. The widened anchor returned success in the worked trace with only a few extra lines, while the fallback would have consumed the entire file length in tokens for what remained a single line edit. That ordering is what the exam marks as the expected workflow rather than Read plus Write as the default.
Incremental discovery as budget control
The correct discovery shape is Grep first for entry points such as the function name, class name, or error string that anchors the investigation to learn which files are relevant, then Read those specific files to follow imports and trace flows, then Grep again for any wrapper or re export names to find indirect consumers, and only Read what each prior discovery step justified. Each Read is therefore justified by a prior hit, and context spend is proportional to relevance rather than to repository size.
The opposite shape, loading every file into context before knowing what is relevant, is the highest cost discovery failure in this domain. A two hundred file upfront read is primarily a budget burn that the exam explicitly warns against, because most files have nothing to do with the task at hand and their inclusion crowds out the synthesis and validation budget needed later for correct integration.
Wrapper tracing and the canonical deprecation sequence
When a function is defined in one module and re exported through a wrapper or a barrel file such as index dot ts, a naive Grep for the original name misses every consumer who imports the wrapper name. The correct trace is to Grep for the definition to find the defining file, Read the defining file to identify exported and re exported names including the wrapper, Grep for each of those names across the codebase, and if a barrel is involved Grep for the barrel module's import path to include consumers that never mention the function directly. That reconstruction is necessary because no single Grep sees the full consumer set without the intermediate Read that reveals what was wrapped.
The deprecation scenario assembles the whole chain into the canonical order the exam expects. Grep for the deprecated function name to locate callers, including tests that imported it directly. Glob for the sibling test of each caller file such as star star slash OrderProcessor dot test dot star for OrderProcessor dot ts to capture transitive coverage. Grep for any wrapper or alias name to catch remaining tests that covered the function through that alias. That Grep then Glob then Grep again content for direct, path for adjacent, content for indirect, is the structured trace that beats Glob first, Bash for both steps when built ins would suffice, and bulk Read before any selection.
Authoritative mechanism reference
The sections below expand the tested rules into the complete documented surface: ownership, lifecycle, version differences, production boundaries, and adjacent mechanisms.
Mechanism reference
This section documents each mechanism at full depth. Every field, flag, and value space is described as shipped, including the variants the exam tends to compress into a single option.
Mechanism reference: Grep: content search
Grep searches file contents with regular expression support and returns matching files with line context. It answers the question what files contain a given string, not which file has a given name.
Ownership and execution model. In Claude Code, Grep runs locally against the project directory, honoring ignore rules such as .gitignore so that build artifacts and dependency directories do not flood results. Results are returned to the model as structured tool results that feed the next planning step in the perception-planning-tool use-execution-observation loop. In the API without Claude Code, the analogous operation is application-implemented search or retrieval, not a built-in server-side Grep. The distinction matters: Claude Code's Grep is a local filesystem tool, the API's retrieval patterns are a separate design choice.
Parameters and usage shape. The exam presentation keeps parameters implicit, but operationally Claude Code invokes Grep with a pattern string and an optional scope. The pattern is a regular expression, not a literal-only search. The implementation streams results path by path with line numbers and matching excerpts so the model can decide which files justify a follow-up Read. Because results are truncated or summarized when very large, a broad pattern can return a partial view that should be narrowed before reading.
Matching semantics. Grep matches inside file contents. It does not match on filename alone. A Grep for OrderProcessor will find OrderProcessor.ts if that file contains the string OrderProcessor, but it will not find a file that is named OrderProcessor yet contains no occurrence of the term. That case is Glob territory, not Grep.
Common call shapes by intent.
- Function caller discovery:
Grepfor the symbol name as a literal, then refine with word-boundary or import-aware regex when false positives appear. - Error surfacing:
Grepfor the error message substring. - Import tracing:
Grepfor the import path fragment, for examplefrom 'utils/auth'orfrom "@/lib/auth". - Wrapper and re-export tracing:
Grepfor the exported wrapper name after reading the defining file to extract that name.
Failure modes and recovery. An overly narrow pattern can miss callers that use a different import alias or a barrel name. An overly broad pattern can return excessive noise that burns context before the model can read. The recovery is to start with the literal symbol, then broaden to alias and barrel searches only after reading the defining file to enumerate the exported surface.
Cost and token behavior. Grep itself does not load full file contents, it returns match excerpts. That is why it is cheaper as an entry-point finder than reading every candidate file. The model should use Grep to earn its Read calls, not to replace them.
Mechanism reference: Glob: path matching
Glob discovers files by path and name pattern using glob syntax such as /.test.tsx or content/domains//.mdx.
Matching semantics. Glob operates on paths, not contents. A pattern like */config. matches src/config.json, config.yaml, and app/config.ts by name breadth, regardless of what those files contain. It will not find a file whose contents mention config if the path itself does not match the pattern.
Typical pattern shapes.
- Extension search:
/.tsor/.tsxto enumerate source files of a type. - Test adjacents:
/OrderProcessor.test.or the broader/.test.*to find sibling tests for a known source file. - Config discovery:
/config.or/.envfor configuration surfaces. - Domain material:
content/lessons//.mdxorcontent/domains//.mdxfor lesson inventories.
Interaction with Grep. The tested pairing is Grep then Glob then Grep again for the deprecation scenario. Grep establishes which source files call the deprecated symbol by content, Glob locates their test files by naming convention even when the test never mentions the symbol, and the final Grep covers wrapper-mediated indirect callers. Inverting the order to Glob first is not correct for that scenario because it presupposes knowledge of the caller set that only a content search can establish.
Ignore and scope handling. Like Grep, Glob respects ignore files and does not enumerate ignored paths unless explicitly instructed to look elsewhere.
Mechanism reference: Read: targeted reading
Read loads file contents from disk and is the perception step that lets the model observe the actual codebase state before planning edits.
Parameters. Read accepts a file path and optional offset and limit for partial reads by line. This is the mechanism that makes incremental discovery affordable: instead of loading a full large file, the model reads only the lines that Grep identified as relevant. The view command of the API's text_editor tool provides analogous partial-read capability with view_range.
Partial read discipline. The model should use Grep to find the line range that contains the target, then Read that range plus a modest window of surrounding context to understand surrounding imports, types, and call sites, rather than reading the entire file when only a function of interest is needed. For very large files, the implementation reads in chunks and may summarize output rather than retaining an unbounded raw dump.
Preconditions and follow-on. A targeted Read should normally precede any Edit to that file so that the anchor text for the edit is constructed from observed contents, not guessed from memory. Guessing the anchor is a common source of Edit failures when files have diverged from the conversation history.
Mechanism reference: Edit: anchored string replacement
Edit is the default modification tool for existing files because it changes only the specified text rather than rewriting the full file.
Parameters. Edit is invoked with old_string and new_string. old_string is the exact literal text currently in the file to be replaced, including whitespace and indentation. new_string is the replacement. The API server-side text_editor tool uses the same shape under the name str_replace with old_str and new_str plus path. The shape is intentionally literal: the tool does not interpret regular expressions for the old text, it matches the exact substring.
Uniqueness guard. If old_string appears in more than one location in the file, Edit fails. This is intentionally strict to avoid changing text the caller never meant to touch. If old_string matches exactly one location, the edit is applied atomically at that site.
Recovery options and ordering. On a non-unique match, the documented recovery in order is: first widen old_string with more surrounding context until it is unique to one site, or if every occurrence should change, set replace_all: true so the tool applies the replacement at each site in one call. Only when neither widening nor replace_all can safely express the intent should the caller fall back to Read plus Write of the full file. The ordering matters because Read plus Write costs roughly a full file's worth of tokens for what is usually a single-line change.
Atomicity and staleness. Edit validates against the file as observed. If the file has changed since the last Read so that old_string no longer matches exactly, Edit fails rather than editing the wrong text. This staleness check is the same mitigation the indentation drift and context staleness analysis calls out. The practical consequence is that the model must re-read when an edit fails due to a stale read.
When not to use Edit. Edit is not the right shape for new files, for changes that span more than about half the file, for repeated Edit failures where the content mismatch persists, or for generated implementations that have no prior content to match against. In those situations Write is appropriate.
Mechanism reference: Write: full-file creation and overwrite
Write creates a new file or overwrites an existing file with the provided full content.
Purpose-built cases. The product documentation calls out four cases where Write is preferred over Edit: creating a file that does not yet exist, applying a change that touches a large fraction of the file, handling repeated Edit failures due to persistent content mismatch, and generating an entirely new implementation. The API's text_editor tool provides the same capability under the create command, with distinct error handling for file-not-found, permission denied, and multiple-matches cases.
Risk profile. Write overwrites the entire file and therefore carries the highest risk of incidental changes: removed comments, lost formatting, or dropped imports that Edit would have preserved. The permission system treats full overwrites as a higher-risk class than targeted edits, so Write is more likely to surface an approval prompt or to require an explicit allow rule when running unattended.
Cost profile. Write requires Read to load the file first in the common scenario where the caller is overwriting an existing file after reasoning about its current contents. This pair costs a file's worth of context. The exam penalizes choosing Write as the default for every modification for exactly this reason.
Mechanism reference: Bash: guarded local shell execution
Bash is the local command execution tool that runs shell commands, waits up to a timeout, and returns stdout and stderr for observation.
Execution model. Bash runs against the local working directory, not inside the API's server-side sandbox. It captures output and exit status so the model can branch: type-check, test, lint, or build after edits and iterate on failures. Long-running commands respect the configured timeout and return what was produced up to that point, which is why output summarization is applied to large results.
Guarding and permission surface. Bash is the most powerful local tool and the most tightly permissioned. In settings.json, Bash rules are scoped by command pattern in parentheses, for example Bash(npm test) or Bash(npm run lint) for allowed cases and Bash(rm -rf *) or Bash(sudo *) for denied cases. The broader permission model distinguishes six baseline modes: default which prompts for anything not already allowed, plan which is read-only exploration with no edits or commands, acceptEdits which auto-accepts file edits and the common filesystem subset mkdir, touch, mv, cp inside the working directory, auto which uses a background classifier to reduce prompts and is constrained to user-level settings as of recent versions, dontAsk which auto-denies anything not already in permissions.allow or the built-in read-only set, and bypassPermissions which skips prompts entirely and can be disabled organization-wide.
Deny precedence. Rules in permissions.deny take precedence over allow and over permissive modes such as acceptEdits and bypassPermissions. A deny rule is the hard boundary; prose instructions in CLAUDE.md are advisory and can be misinterpreted, while a deny rule is enforced by the tool layer before the model's judgment. This is why exam scenarios about destructive accidents point at a missing deny rule, not at insufficient instructions.
Best practice for non-interactive use. When Claude Code runs with -p or --print and no human is present to answer approval prompts, any operation not already covered by an allow rule or by the chosen --permission-mode baseline aborts the run rather than executing silently. Pipelines should declare the smallest set of allowed tools for the job, typically Read, Grep, Glob, and a scoped Bash(npm test) or similar, rather than opening broad shell access.
Mechanism reference: Claude Code local versus API server-side: the full boundary
This is the central scope distinction the task requires and the exam deliberately tests. Claude Code's local tools and the Messages API's server-side tools share names in conversation, but they are distinct implementations running in distinct environments with distinct ownership.
Claude Code local toolkit. Read, Write, Edit, Bash, Grep, Glob, and WebFetch/WebSearch as exposed through Claude Code run locally against the developer's filesystem, ignore rules, shell, and network. They consume local context and local permissions. Their lifecycle is the terminal agent loop: perception through Glob/Grep/Read, planning, tool use, execution, and observation of tool output that loops until the task is complete or asks for clarification.
API server-side toolkit. The Messages API defines a separate set of Anthropic-provided tools that execute inside Anthropic infrastructure, not on the developer's machine.
code_executionwhich providesbash_code_executionandtext_editor_code_executionsub-tools forBashcommands and file operations inside a sandboxed container, plus structured results for file view, create, andstr_replace. Type identifiers are versioned: the current family iscode_execution_20250825, withcode_execution_20260120adding REPL state persistence and programmatic tool calling, andcode_execution_20260521which is the same runtime with an updated description that informs on the 90-second wall-clock limit per Python cell. The legacycode_execution_20250522Python-only beta lineage remains valid as an opt-in but is not the current path.
str_replace_based_edit_toolwhich is the standalonetext_editortool for view,str_replace,create, andinsertoperations against the API's file surface, configured withtypeandnameand optionallymax_characters. Versioned identifiers aretext_editor_20241022(initial),text_editor_20250124,text_editor_20250429(removesundo_editand updates the architecture name), andtext_editor_20250728(addsmax_characters).
web_searchandweb_fetchwhich give the model access to live web content with citation support and domain and location controls. Versioned identifiers includeweb_search_20250305for basic search,web_search_20260209which adds dynamic filtering through automatic code execution, andweb_search_20260318which addsresponse_inclusioncontrol.
Why the boundary matters for the task. A codebase operation that needs to touch local files under version control must use the local toolkit. An API request that needs to run untrusted Python or fetch live web content for grounding must use the server-side toolkit. Confusing the two leads to incorrect answers about who owns execution, where files live, what is audited, and which permission gate applies. The application owns local file writes and shell execution for Claude Code, Anthropic's sandbox owns server-side code execution with its 5 GiB workspace and network isolation, and the web retrieval service owns server-side search and fetch.
Interleaving and multicomputer awareness. When both a server-side execution tool and a client-provided local tool are present in the same request, the model is operating across two computers. Outputs and state do not persist between them, and the API handles the sequencing by returning server-side work and client-side tool_use blocks with stop_reason handling that depends on whether the model needs fresh client results before continuing server-side work.
Mechanism reference: Read and edit flow with the server-side text editor
The server-side text editor mirrors the local Read plus Edit shape with a distinct wire format: the model emits server_tool_use blocks with name set appropriately and input.command set to view, str_replace, create, or insert. The application is responsible for implementing the host side: reading the file for view, truncating to max_characters when configured, performing the replacement with a unique-match check for str_replace, and returning a tool_result with is_error when the file is missing, permission is denied, or the old string matches zero or multiple locations.
Mechanism reference: Web retrieval and citations
Server-side web_search is invoked by the model with query and optional max_uses, allowed_domains or blocked_domains, user_location, allowed_callers, and response_inclusion. Results arrive as web_search_tool_result blocks containing web_search_result entries with url, title, page_age, and encrypted_content for continuation, and citations of type web_search_result_location are emitted alongside text so that downstream display can attribute sources. web_fetch follows the same server-tool pattern for fetching specific URLs. In Claude Code, WebSearch and WebFetch are local built-ins that perform the network fetch from the developer's environment and are subject to the local permission model rather than the API's server-tool billing and domain filtering.
Ownership map
Clarity about who owns which guarantee is essential for both building a real system and for choosing the right exam answer. The five layers below correspond to where a defect surfaces and who can fix it.
Model. The model owns selection among the tool options it was offered and construction of parameters consistent with each tool's description and schema. It owns deciding to call Grep when the task is about contents and Glob when the task is about paths, choosing a narrow or broad anchor for Edit, requesting a str_replace versus a create, and interleaving reads with searches to keep context focused. The model does not own enforcement of permission policy, persistence of writes, or validation of filesystem paths. When the model hallucinates a dangerous system call or interpolates an untrusted string into a path, the layers above it must catch the error.
Application code. Application code owns tool construction, argument validation, and result handling. This includes defining tool schemas with adequate descriptions so the model selects correctly; validating and sanitizing every string parameter against an allowlist, length bound, or format before it reaches a database, filesystem, or downstream API; enforcing parameterized queries rather than string interpolation; implementing the host side of text_editor operations with correct unique-match checks and backup discipline; and returning structured tool results the model can reason from. Ownership of sequencing also lives here: the application implements the tool loop that appends the assistant turn, executes all tool_use blocks, and returns tool_result blocks with matching tool_use_id values in order.
SDK. The SDK provides the thin transport and schema layer that carries tools, tool_choice, stop_reason, and tool_result blocks between the application and the API without business logic of its own. It owns faithful serialization of tool definitions and tool results and faithful dispatch of callbacks such as canUseTool in the Agent SDK, but it does not own authorization decisions, which remain with application hooks, allow and deny rules, and the permission mode gate.
CLI. The Claude Code CLI owns the local permission mode baseline, the --allowedTools and --permission-mode per-invocation overrides, the --add-dir directory exposure, and the --output-format json structured output contract for pipelines. It also owns the memory and settings precedence that determine which rules and instructions are loaded: managed-settings.json or MDM policy at the top, then CLI --settings JSON for the session, then local settings.local.json, then project settings.json, then user settings.json, with CLAUDE.md concatenation ordered root to working directory and subdirectory CLAUDE.md files loading lazily when touched.
Infrastructure. Infrastructure owns sandbox isolation, resource limits, and network policy for server-side execution. For code_execution this includes the Linux container with 5 GiB RAM and 5 GiB workspace, one CPU, no outbound network, 30-day container retention with checkpoint after about five minutes of idle, and pre-installed Python libraries such as pandas, numpy, matplotlib, and scipy that the model can use without installing packages at runtime. It also owns the web retrieval infrastructure that enforces domain controls and returns encrypted_content for continuation, and the container reuse and file download surfaces via the Files API.
Version and terminology currency
The exam guide and community writeups describe an earlier snapshot of the product surface than what is documented today. The gaps are small enough to cause distractor errors and large enough to matter for the tool-type identifier that appears verbatim in API examples.
Claude Code local tool names versus API server tool type strings. The exam guide speaks about six built-in codebase tools by short name: Read, Write, Edit, Bash, Grep, Glob. The Messages API speaks about versioned tool type strings that include a date suffix: code_execution_20250825 and later, text_editor_20250728 and earlier siblings, web_search_20250305, web_search_20260209, and web_search_20260318, and web_fetch_20260209 and later. A candidate who memorizes only the short local names will miss questions that show a wire-format type and ask which tool is present. Conversely, a candidate who thinks code_execution is interchangeable with local Bash will misplace where execution happens and what is isolated.
Text editor version progression. The text_editor lineage that the exam guide collapses into a single Edit is actually a progression: text_editor_20241022 introduced view, create, str_replace, insert, and undo_edit; text_editor_20250124 optimized for a newer generation without changing capabilities; text_editor_20250429 removed undo_edit and renamed the architecture around str_replace; text_editor_20250728 added max_characters. The changelog matters because exam distractors may show undo_edit as if it were current, or omit max_characters handling and expect truncation behavior.
Code execution version progression. The reference page does not mention server-side code execution. The current type family is code_execution_20250825 for Bash and file operations, code_execution_20260120 which adds REPL state persistence and programmatic tool calling from within the sandbox, and code_execution_20260521 which keeps the same runtime with a description that tells the model about the 90-second per-cell limit. The prior Python-only lineage code_execution_20250522 with the code-execution-2025-05-22 beta header remains supported as a legacy opt-in but is not the type shown in current examples.
Web search version progression. The reference page does not mention web retrieval at all, yet it is part of the server-side tool story. web_search_20250305 is basic search; web_search_20260209 adds dynamic filtering that runs search through sandboxed code execution with allowed_callers defaulting to code_execution_20260120; web_search_20260318 adds response_inclusion to drop nested server blocks from the response when consumed internally. A question that shows allowed_callers or response_inclusion is testing knowledge of the newer types, even if the exam guide text never mentions them.
Permission mode vocabulary. The exam guide recaps the permission ideas without enumerating every mode name. The current product enumerates default or ask, plan, acceptEdits, auto, dontAsk, and bypassPermissions with documented behavior differences for filesystem commands and network requests. Community posts sometimes speak about alwaysAllow or readOnly as if they were modes; those are not the shipped identifiers. The related CLI flags are --permission-mode and --allowedTools, not a catch-all --pref.
Settings and memory precedence vocabulary. Earlier exam summaries sometimes describe CLAUDE.md as the only instruction surface and settings.json as a single file. The current precedence is layered: CLI flags override session settings, which override local settings.local.json, which override project settings.json, which override user settings.json, with managed settings above all of them; CLAUDE.md files concatenate rather than override and subdirectory files load lazily. A distractor that treats CLAUDE.md layers as overriding, or that places permission rules in CLAUDE.md instead of settings.json, is wrong on current behavior.
Tool result block naming. The wire format distinguishes tool_use and tool_result for client tools from server_tool_use, bash_code_execution_tool_result, text_editor_code_execution_tool_result, and web_search_tool_result for server tools. An exam answer that matches a tool_use_id to a tool_result is testing the client loop; one that talks about server_tool_use with srvtoolu_ identifiers and pause_turn is testing the server-tool loop. The two loops have distinct continuation rules and should not be conflated.
Official versus community divergence
When community study material diverges from Anthropic's documentation, the examination expects the documentation position. Three divergences recur for this task.
Divergence 1: Grep versus Glob teaching collapses into Bash pipelines.
- Community position: use
Bashwithfind. -name "*.test.tsx" | xargs grep -l processLegacyOrderas a single shell pipeline for both file discovery and content search, since shell is universal across environments. - Documentation position: use the purpose-built Claude Code tools
Grepfor content search andGlobfor path matching because they natively respect ignore files, return structured tool results suited to the agent loop, and keep token cost proportional to what the model actually needs rather than dumping an unbounded shell transcript into context. - Candidate action: answer with
Grepfor contents andGlobfor paths. Treat theBashpipeline option as the distractor the reference page calls out, unless the prompt explicitly constrains the environment to excludeGrepandGlob.
Divergence 2: Read plus Write as the default modification path.
- Community position: always
Readthe whole file andWriteit back because it is conceptually simpler and avoids thinking about anchor uniqueness. - Documentation position: prefer
Editwithold_stringandnew_stringfor targeted changes, widen the anchor or usereplace_all: trueon non-unique matches, and reserveReadplusWritefor the fallback when those do not disambiguate. The preference is grounded in token cost, edit precision, and the permission system's treatment of full overwrites as higher-risk. - Candidate action: choose
Editas the first attempt and the widen-or-replace_allrecovery. ChooseReadplusWriteonly when the reference page says neither lighter option can disambiguate the target.
Divergence 3: Permission rules placed in CLAUDE.md versus settings.json.
- Community position: document
never run rm -rfas prose inCLAUDE.mdand rely on the model's instruction following. - Documentation position: enforce hard boundaries in
settings.jsonunderpermissions.denywith scoped patterns such asBash(rm -rf *)andRead(./.env), because deny rules are evaluated by the tool layer before the model's judgment and take precedence over permissive modes includingacceptEditsandbypassPermissions.CLAUDE.mdis for project knowledge, preferences live insettings.json. - Candidate action: choose the
settings.jsondeny rule answer when the scenario involves a destructive accident that prose instructions failed to prevent.
Divergence 4: Fabricated permission modes and config file formats.
- Community position gaining traction in third-party writeups: Claude Code has an
alwaysAlloworreadOnlymode, a--prefcatch-all flag, or a glob-matched.mdcfile format for directory rules. - Documentation position: the shipped modes are
default,plan,acceptEdits,auto,dontAsk, andbypassPermissions; the shipped per-invocation flags are--permission-mode,--allowedTools,--max-turns, and similar explicit flags with no single--pref; directory-scoped instruction is plainCLAUDE.mdfiles loaded lazily plus@path/to/fileimports, not a separate glob-matched file format. - Candidate action: reject answer options that mention
alwaysAllow,--pref, or.mdcas if they were product vocabulary. They are borrowings from adjacent tools that do not apply to Claude Code.
Beyond the task statement
The reference page scopes itself to Read, Write, Edit, Bash, Grep, and Glob with a strong emphasis on Grep versus Glob and on incremental discovery. Our lesson set captures a wider set of topics that are tested in the same domain and that make the difference between a narrow pass and a top score. Each item below names the lesson slug where full treatment lives.
Incremental discovery as a context-management discipline, not just a tool tip. The lessons frame Grep before Read as an instance of a broader principle: keep the context window lean by earning each file read with evidence that it matters, scoped reads with offset and limit, periodic CLAUDE.md pruning, and CLAUDE.local.md for ephemeral notes. The principle generalizes beyond the six tools to every context decision in a long session.
Diff safety and staleness, including the indentation drift family. The Edit uniqueness guard is paired with a catalog of file-operation pitfalls: indentation drift when full rewrites reformat unaffected lines, hallucinated imports when the model invents dependencies, context staleness when the model edits from an earlier read that has since changed, and over-application that touches unrelated code. Edit with a fresh Read plus type-check or test verification is the mitigation chain.
Permission modes, hooks, and the CLI gate for automations. Beyond choosing a tool, the lessons treat the permission mode baseline, the permissions.allow, permissions.ask, permissions.deny sets, hook execution order, and CI flags (-p or --print, --output-format json, --json-schema, --allowedTools, --permission-mode, --max-turns, --bare) as a coherent safety and scripting story. A session that can edit and execute locally must be gated differently from one that should only read and report, and a deny rule is the hard boundary where instructions are only advisory.
Memory hierarchy and lazy instruction loading. CLAUDE.md files are discovered by walking up the directory tree and concatenated, not merged with override, with subdirectory files loading lazily only when touched. This is why a project that puts package-specific rules in the right subdirectory file pays no context cost until the model actually enters that package.
Server-side versus local execution awareness. The code_execution, text_editor, and web_search plus web_fetch surfaces are server-side tools with sandbox isolation, versioned type strings, and server_tool_use block shaping that differ from the local Bash and Read or Write shapes. Lessons on tool design and on the sandbox isolate the principle that broadening a server-side tool's parameters should be done via a new parallel tool rather than a boolean switch on the existing one, and that new parallel tools preserve backward compatibility.
Output handling, summarization, and retry placement. The agentic loop guidance places retry and validation after observing tool results rather than preemptively in prompts, and distinguishes between result summarization before the model sees output and manual /compact or automatic compaction when the window fills. For Glob and Grep, this means the raw result list may be summarized transparently and the model should narrow the pattern rather than reading every hit.
Tool granularity and the action-perception feedback loop. The adjacent architecture lessons generalize the codebase tool choices into tool design principles: clear descriptions, structured results, graceful errors, and balanced granularity where read operations favor fine-grained control and writes favor coarse-grained batching. Each tool result becomes new perception for the next step, so result quality directly affects loop quality.
Security boundary for execution tools. Input validation, sanitization, least-privilege scoping, audit logging, and invocation limits apply to every execution tool, not only to the codebase six. The server-side sandbox restricts filesystem scope, network access, and resource consumption, while the application layer still owns format validation and per-user isolation.
Worked production examples: Example A: Retirement of processLegacyOrder across a wrapper and barrel boundary
Context. A team must retire a deprecated processLegacyOrder function. Callers live in src/orders/OrderProcessor.ts and src/billing/RefundHandler.ts. Tests live as sibling files OrderProcessor.test.tsx and RefundHandler.test.tsx by naming convention, but the test suite does not mention processLegacyOrder by string. One caller imports through a re-export barrel src/orders/index.ts under a wrapper name applyLegacyOrder. A single literal search will miss the wrapper consumer, and a pure filename pattern will miss the caller set entirely.
Reasoning chain.
First, establish the caller set by contents. Grep for processLegacyOrder finds the defining file and the direct callers that reference the deprecated symbol. This surfaces direct references and any tests that import the function by name.
Second, enumerate the exported surface that could hide indirect callers. Read the defining file with a focused window around the export declarations to determine whether the symbol is exported as processLegacyOrder, re-exported under a different name, or wrapped under a name such as applyLegacyOrder.
Third, find adjacent tests by path. Glob for /OrderProcessor.test. and /RefundHandler.test. locates each caller's sibling test file even when the test never mentions the deprecated name. This is the phase the exam describes as path matching for sibling tests and it cannot be replaced by Grep alone.
Fourth, close the indirect gap. Grep for each wrapper or barrel-exported name discovered in the defining file, for example applyLegacyOrder and the barrel import string from 'orders' or from '@/orders', to capture consumers that import through the barrel rather than from the defining module path.
Fifth, perform targeted reads only on the files identified above, then apply narrowly scoped edits. Each modification should use Edit with an anchor that includes the call site plus enough surrounding context to be unique, and verify with Bash type-check and test runs before moving on.
Failure that this avoids. Reading every file upfront would spend a whole file's worth of tokens on files unrelated to the retirement, and would dilute context before the model has a plan. Relying only on Grep would miss sibling tests by naming convention. Relying only on Glob would miss callers whose paths bear no relation to the function's name. Collapsing the search into a single Bash pipeline loses the structured signal that lets the model reason incrementally about direct versus indirect consumers.
Observable outcome. The result is a precise set of source files to modify, a paired set of test files to update or extend, and a set of wrapper consumers to migrate, with no unrelated files loaded and no silent missed callers.
Worked production examples: Example B: Fixing a validation gap in SignupForm.tsx with narrow reads and guarded execution
Context. A product issue reports that an email validation branch is missing in src/components/SignupForm.tsx. The repository is large and the form imports shared validators from src/lib/validators.ts. The fix must not disturb imports or unrelated formatting.
Reasoning chain.
First, narrow by path. Glob for src/*/signup* identifies the candidate files SignupForm.tsx and its test. This is the discovery step that prevents a blind full scan.
Second, observe. Read the identified form file. In a 200-line component the model does not need the entire handbook, a Read of the file, or a Read with offset around the submit handler, is sufficient after confirming the file is relevant.
Third, check for existing conventions before editing. Grep for validation in the file shows no prior branch. Read of src/lib/validators.ts and a Grep for import.validator across src//.ts surface existing patterns that the fix should reuse rather than invent.
Fourth, apply a targeted edit. Edit the validation site with an old_string that includes the unique anchor around the function call plus a few lines of surrounding context, replacing it with the validated variant. If Edit reports a non-unique match, widen the anchor with additional surrounding lines until it is unique, or use replace_all: true only if every call site should change. This preserves indentation and leaves unrelated imports untouched.
Fifth, verify locally with Bash type-check and the specific test suite. Bash with npx tsc --noEmit followed by npm run test -- --testPathPattern=SignupForm confirms the change compiles and does not regress existing tests. In a guarded session, those Bash patterns are allowlisted explicitly in settings.json, while mutations to unrelated paths remain denied.
Failure that this avoids. Using Read plus Write for a one-line validation addition would risk reformatting unaffected sections and would incur a full file's token cost. Using Bash without a scoping rule would allow an over-broad command to touch unrelated files. Skipping the initial Glob and Grep narrowing would force a broad read that wastes context and makes it harder for the model to attend to the relevant site.
Observable outcome. The form validates as intended, only the validated lines changed, imports and formatting elsewhere are untouched, and the gated Bash verification proves the fix is safe.
Worked production examples: Example C: Choosing server-side versus local execution gates for risky work
Context. A ticket asks for two things in the same request: run an untrusted data analysis snippet to classify customer sentiment, and then modify the repository's auth middleware at src/middleware/auth.ts to enforce a new rate limit. The correct plan must not conflate where each piece runs or which gate applies.
Reasoning chain.
First, route the analysis to the API's server-side sandbox. Add the versioned tool code_execution_20250825 with name code_execution to the Messages API request so the model can run Bash and file operations in a Linux container with 5 GiB RAM, 5 GiB workspace, no outbound network, and pre-installed libraries. The response arrives as server_tool_use and bash_code_execution_tool_result blocks with stdout, stderr, return_code, and content file entries, plus a reusable container identifier for follow-up requests.
Second, route the repository change to Claude Code's local toolkit, not to the server sandbox. Use Grep to locate the rate-limit call sites, Read the auth file and middleware, Edit with a widened anchor, and Bash for npm test and npx tsc --noEmit verification. Enforce the boundary in settings.json with permissions.deny entries for Bash(rm -rf *) and Read(./.env) so that no analysis artifact path can be leveraged into a destructive local operation.
Third, handle the multicomputer fact explicitly. Server-side outputs and local file state do not persist between the two environments. If the analysis produces a file to be checked in, the file must be created locally with a local Write or with the local text_editor equivalent, not assumed to have been carried over from the server container automatically.
Failure that this avoids. Running untrusted analysis in local Bash would give it full filesystem and network scope. Running the auth fix through the server-side code execution tool would write into an ephemeral container whose filesystem is not the repository. A pipeline that lacks the deny rule would allow the model to execute a destructive command when reasoning under adversarial content.
Observable outcome. Analysis executes with resource and network isolation in the server container, the repository edit lands on the local filesystem under the local permission gate with a targeted diff, and neither environment silently trusts state from the other.
Build exercise material
These exercises are verifiable in a real Claude Code checkout and in a small driver program against the Messages API. Each step states what to run and the observable outcome that proves success. Use a scratch project so no production file is at risk.
Build exercise material: Exercise 1: Grep then Glob then Grep again for caller and test discovery
This exercise drills the core Grep for contents, Glob for sibling tests, Grep for wrappers distinction that the reference page marks as the scenario that turns up constantly.
Setup. In a scratch project create src/orders/OrderProcessor.ts that imports processLegacyOrder from ./legacy, src/billing/RefundHandler.ts that calls applyLegacyOrder which internally calls processLegacyOrder, a barrel src/orders/index.ts that re-exports applyLegacyOrder, and sibling tests src/orders/OrderProcessor.test.tsx and src/billing/RefundHandler.test.tsx that exercise the source modules without mentioning processLegacyOrder directly.
Step 1: Grep for direct callers.
Run a content search for processLegacyOrder.
Expected result: a list of paths whose contents reference the literal, typically the defining file, src/orders/OrderProcessor.ts, and any test that imports the name directly, with line numbers and matching excerpts. Path src/billing/RefundHandler.ts will not appear yet if it only calls the wrapper name.
Step 2: Glob for sibling test files.
For each caller file discovered in step 1, run Glob for the adjacent test pattern /OrderProcessor.test. and /RefundHandler.test., plus the broader */.test.* if you want to confirm coverage breadth.
Expected result: the test files appear by naming convention even though they never mention the deprecated name. This is the observable proof that Glob is not redundant with Grep in this task: Grep found what is inside files, Glob found files by their names.
Step 3: Read the defining file to extract the wrapper name.
Read the file that defines processLegacyOrder and the file that wraps it, collecting exported names.
Expected result: you observe applyLegacyOrder as a wrapper around the deprecated symbol and src/orders/index.ts as a barrel that re-exports it. The import graph now tells you what to search next.
Step 4: Grep for wrapper names.
Run Grep for applyLegacyOrder and for the barrel import path fragment from 'orders' or from '@/orders' as it appears locally.
Expected result: the previously unseen consumer src/billing/RefundHandler.ts now appears, along with any additional indirect callers that imported through the barrel. This completes the indirect coverage that a single literal search misses.
Step 5: Targeted Read for context before any modification.
Read each discovered caller and wrapper with offset and limit scoped to the call sites identified by the Grep hits, rather than loading each whole file.
Expected result: you see imports, call arguments, and return-value handling with minimal token spend, and you have enough context to craft a unique Edit anchor.
Build exercise material: Exercise 2: Anchored editing with the uniqueness guard
This exercise drills Edit as the default, the widen-or-replace_all recovery, and the Read plus Write fallback, with observable messages at each stage.
Setup. In the same scratch project, create src/components/SignupForm.tsx with three identical literal occurrences of return res.status(200).json({ token }) in different functions, so that any Edit that targets only that literal is ambiguous by construction.
Step 1: Attempt a narrow Edit.
Call Edit with old_string set to return res.status(200).json({ token }); and a new_string that adds a rate-limit branch.
Expected result: an error indicating the anchor matched multiple locations, for example Found 3 matches for replacement text or the tool-form equivalent old_string matches 3 locations. The change is refused.
Step 2: Widen the anchor.
Retry Edit with a larger old_string that includes the unique surrounding function name or the unique adjacent line before the return, for example including if (rateLimiter.isBlocked(ip)) context or the function signature above the site.
Expected result: exactly one location now matches and the tool reports success with a diff such as - return res.status(200).json({ token }); and + const ip = req.ip; plus the branch. Only the intended site changed.
Step 3: Use replace_all when the intent is global.
In a separate branch, call Edit with replace_all: true on the same narrow literal when the goal genuinely is to replace every occurrence.
Expected result: all three occurrences are updated atomically with a single tool call, without loading and rewriting the whole file.
Step 4: Fall back to Read plus Write only when the other two cannot express the change.
Construct a scenario where neither widening nor global replacement captures the intent, for example changing two sites in opposite ways where the shared literal has no surrounding uniqueness. Now Read the full file and Write it back with the intended content.
Expected result: the full file is produced with both changes, but at the cost of a file's worth of tokens and with a diff that may touch more than the two target lines. This is the observable cost that justifies Read plus Write as the fallback, not the standard path.
Build exercise material: Exercise 3: Guarded shell usage and permission configuration
This exercise drills the local execution gate that the reference page touches only implicitly and that our lessons treat as load-bearing: settings.json allow, ask, and deny with deny precedence and per-invocation overrides.
Setup. Start from a scratch project's .claude/settings.json or the managed equivalent. Prepare three categories of commands: safe operations such as npm test and npm run lint, readable file operations such as Read(/) and Glob(/), and forbidden operations such as rm -rf *, sudo , Read(./.env), and Read(./secrets/*).
Step 1: Inspect the shipped permission modes.
Identify the six baselines: default, plan, acceptEdits, auto, dontAsk, and bypassPermissions, and note which classes of operations each auto-approves or blocks.
Expected result: you can describe which mode auto-accepts mkdir, touch, mv, cp in the working directory, which one is read-only with no edits or commands, and which one skips prompts entirely and is subject to organization-wide disablement.
Step 2: Configure permissions.allow, permissions.ask, permissions.deny.
Put safe reads and searches in allow, put overwrites in ask, and put destructive and sensitive-path operations in deny. Confirm that tool-name wildcards are only valid after a literal mcp__<server>__ prefix for MCP tools, while built-ins are scoped with parenthesized patterns such as Bash(npm test).
Expected result: a settings.json fragment like the one below validates as JSON and is accepted by the tool layer.
Step 3: Verify deny precedence.
Add a conflicting rule where an allow would permit Bash(rm -rf *) but deny blocks it. Trigger a run that reaches that command.
Expected result: the operation is blocked by the deny rule despite the permissive baseline. This is the proof that permissions.deny is the hard boundary, while prose in CLAUDE.md is only advisory.
Step 4: Exercise a per-invocation override.
Run with -p or --print and combine --permission-mode dontAsk with --allowedTools "Read,Glob,Bash(npm test)" for a read-and-report job, and with --permission-mode acceptEdits for an auto-fix job.
Expected result: in the dontAsk run the session reports without touching files and exits cleanly, while in acceptEdits it auto-accepts file edits but still requires an explicit allow rule for non-filesystem Bash commands. A missing allow in dontAsk aborts the run, which is the correct guarded failure rather than a silent bypass.
Step 5: Observe the -p mode gate.
Run the same write-bearing step without any allow or --permission-mode coverage.
Expected result: the run aborts with a non-zero exit rather than writing the file, because no human is present to answer the approval prompt in non-interactive mode. Adding the file to --allowedTools or to permissions.allow, or selecting a baseline that covers it, is the minimal fix; disabling the permission system globally is not.
Build exercise material: Exercise 4: Server-side text editor, code execution, and web retrieval in a driver program
This exercise drills the API-side boundary and the versioned tool identifiers in a small driver that does not touch Claude Code's local files.
Setup. Write a driver that targets the Messages API. Do not use Claude Code's local Read, Write, or Bash for these steps, the host side of the text editor tool is implemented by your code around tool_use blocks with name str_replace_based_edit_tool and command values of view, str_replace, create, and insert.
Step 1: View with range and max_characters.
Send a request with tools containing type text_editor_20250728, name str_replace_based_edit_tool, and max_characters 10000, and a user turn that asks the assistant to examine primes.py. Handle the assistant's tool_use block where input.command is view and input.path is primes.py, return the file contents with line numbers, and observe truncation when max_characters is smaller than the file.
Expected result: the tool_result containing the file excerpt reaches the next assistant turn, and the model proceeds without you having to guess view shapes. The driver correctly implements the view_range pair for partial reads when requested.
Step 2: Demonstrate the uniqueness error surface.
When the model sends str_replace with an old_str that appears in multiple places, return a tool_result with is_error: true and content Error: Found N matches and observe the model widen the anchor on the next attempt.
Expected result: the model retries with a broader old_str and succeeds with a single-site replacement. This is the server-side parallel to the Claude Code Edit guard.
Step 3: Execute untrusted Python in the server sandbox.
Send a request with tools containing type code_execution_20250825, name code_execution, and a user turn that asks for mean and standard deviation of a numeric list. Observe server_tool_use with name bash_code_execution and the matching bash_code_execution_tool_result with stdout, stderr, return_code, and optional content file entries, plus the top-level container object with id for reuse.
Expected result: the analysis completes without the model installing packages at runtime, confirming the sandboxed runtime with Python 3.11 and pre-installed libraries and no internet access.
Step 4: Chain retrieval with execution for live grounding.
Send a request with tools containing type web_search_20250305 or web_search_20260209, name web_search, max_uses 5. Observe server_tool_use with name web_search, then web_search_tool_result with web_search_result entries containing url, title, page_age, and encrypted_content, plus citations of type web_search_result_location with cited_text alongside the assistant text.
Expected result: subsequent turns that carry encrypted_content and encrypted_index values forward without modification continue successfully, while a missing or altered encrypted value fails with a validation error, proving that continuation requires faithful echo of those fields.
Production code and configuration
The five substantial, language-tagged examples below cover the full surface the prompt requires: content search versus filename matching, targeted reading, anchored editing, guarded shell usage with permission configuration, and server-side tools for code execution, text editing, and retrieval. Each block uses accurate field, flag, and config keys supported by current documentation, states what it proves, its failure boundary, and its observable output. Inline identifiers such as Read, Write, Edit, Bash, Grep, Glob, server_tool_use, tool_result, tool_use_id, view_range, old_str, new_str, str_replace_based_edit_tool, bash_code_execution, text_editor_code_execution, web_search, web_fetch, allowed_domains, blocked_domains, user_location, response_inclusion, max_characters, max_uses, permission_mode, permissions.allow, permissions.deny, container, srvtoolu_, and toolu_ are in backticks as required.
Production code and configuration: Example 1: Content search versus filename matching with Grep and Glob
What this proves: Grep for contents and Glob for paths are not interchangeable, and the exam-tested three-phase sequence is Grep for direct callers, Glob for sibling tests by naming convention, and Grep for wrapper or barrel-mediated indirect callers. It also proves that Bash with find plus xargs grep is technically capable but is the distractor when the prompt asks for the purpose-built tool choice.
Failure boundary: a literal Grep misses indirect callers that import through a wrapper or barrel name, and a Glob that matches */processLegacyOrder* never finds callers whose paths do not contain that string. A broad Grep pattern can also return excessive noise that must be narrowed before reading.
Observable output: two distinct result sets. Grep returns file paths with line numbers and matching excerpts such as src/orders/OrderProcessor.ts:42: await processLegacyOrder(orderId). Glob returns file paths such as src/orders/OrderProcessor.test.tsx and src/billing/RefundHandler.test.tsx by name, even when those tests never mention processLegacyOrder.
import * as fs from "node:fs";
type ToolCall = { tool: string; input: Record<string, unknown> };
function buildTracePlan(deprecated: string, wrapper: string, barrelImport: string): ToolCall[] {
return [
{ tool: "Grep", input: { pattern: deprecated, note: "content search for direct callers and direct-import tests" } },
{ tool: "Read", input: { path: "src/orders/legacy.ts", note: "follow imports to enumerate wrapper and barrel exports" } },
{ tool: "Glob", input: { patterns: ["**/OrderProcessor.test.*", "**/RefundHandler.test.*"], note: "path matching for sibling tests by naming convention" } },
{ tool: "Grep", input: { pattern: wrapper, note: "content search for wrapper name covering indirect callers" } },
{ tool: "Grep", input: { pattern: barrelImport, note: "content search for barrel import path covering consumers via index" } },
];
}
function isWrongTool(choice: string, intent: "contents" | "paths"): boolean {
if (intent === "contents" && choice === "Glob") return true;
if (intent === "paths" && choice === "Grep") return true;
return false;
}
const plan = buildTracePlan("processLegacyOrder", "applyLegacyOrder", "from 'orders'");
for (const step of plan) {
console.log(`${step.tool}: ${JSON.stringify(step.input)}`);
}
console.log("Wrong for caller search?", isWrongTool("Glob", "contents"));
console.log("Wrong for test-by-extension search?", isWrongTool("Grep", "paths"));
fs.writeFileSync("/tmp/trace-plan.json", JSON.stringify(plan, null, 2));Production code and configuration: Example 2: Incremental reading with offset and limit after targeted search
What this proves: the affordable reading discipline is Grep to locate the line, then Read with offset and limit to observe only the relevant window before planning an edit, rather than reading every file upfront and flooding the context window.
Failure boundary: reading the full file on every search hit wastes tokens at roughly one token per four characters. For a 500-line file at 40 characters per line the cost is around 5000 tokens per read, which accumulates quickly across a repository scan. Large results may also be summarized transparently, so the model should narrow the pattern instead of assuming a full dump was retained.
Observable output: a small read of the lines around the hit plus surrounding import and type context, with the rest of the file unloaded, showing the model's context remains focused on the task's dependency graph.
type Hit = { file: string; line: number; excerpt: string };
function targetedReadWindow(hit: Hit, radius: number = 20): { path: string; offset: number; limit: number } {
const start = Math.max(1, hit.line - radius);
return { path: hit.file, offset: start, limit: radius * 2 + 1 };
}
async function grepThenRead(claudeCode: { grep: (p: string) => Promise<Hit[]>; read: (path: string, offset: number, limit: number) => Promise<string> }) {
const hits = await claudeCode.grep("processLegacyOrder");
console.log(`Grep found ${hits.length} direct callers`);
for (const hit of hits) {
const window = targetedReadWindow(hit, 15);
const slice = await claudeCode.read(window.path, window.offset, window.limit);
console.log(`Read ${window.path}:${window.offset}-${window.offset + window.limit}`);
void slice;
}
const wrapperHits = await claudeCode.grep("applyLegacyOrder");
console.log(`Wrapper search found ${wrapperHits.length} indirect callers`);
}
void targetedReadWindow({ file: "src/orders/OrderProcessor.ts", line: 42, excerpt: "await processLegacyOrder(orderId)" });Production code and configuration: Example 3: Anchored editing with uniqueness widening and replace-all versus full rewrite
What this proves: Edit with literal old_string and new_string is the correct first attempt, a non-unique match is recovered by widening the anchor with unique surrounding lines or by setting replace_all: true for a global change, and Read plus Write is the high-cost fallback that overwrites the entire file.
Failure boundary: Edit fails when old_string matches zero or multiple locations and when the file has changed since the last Read so the exact text no longer matches. The tool reports Found N matches or an equivalent multiple-match message and refuses the write. A Read plus Write fallback that touches more than about half the file risks indentation drift and incidental import changes.
Observable output: a blocked first attempt with a multiple-match diagnostic, a succeeding second attempt after widening that reports a single-site diff such as - return res.status(200).json({ token }); and + const ip = req.ip; with the branch, and a replace_all attempt that updates every occurrence in one call.
type EditResult = { ok: boolean; message: string };
type EditInput = { path: string; old_string: string; new_string: string; replace_all?: boolean };
function simulateEdit(file: string, input: EditInput): EditResult {
const count = countOccurrences(file, input.old_string);
if (count === 0) return { ok: false, message: "Error: No match found for old_string" };
if (count > 1 && !input.replace_all) return { ok: false, message: `Error: Found ${count} matches. Provide more surrounding context to make a unique match or set replace_all: true.` };
return { ok: true, message: input.replace_all ? `Applied to ${count} locations` : "Applied to 1 location" };
}
function countOccurrences(text: string, needle: string): number {
let n = 0; let i = 0;
while ((i = text.indexOf(needle, i)) !== -1) { n++; i += needle.length; }
return n;
}
function widenedEditExample(fileContent: string) {
const narrow = 'return res.status(200).json({ token });';
const narrowResult = simulateEdit(fileContent, { path: "src/api/login.ts", old_string: narrow, new_string: narrow });
console.log(narrowResult.message);
const widenedOld = " const token = await issueToken(user);\n return res.status(200).json({ token });";
const widenedNew = " const ip = req.ip;\n if (rateLimiter.isBlocked(ip)) { return res.status(429).json({ error: \"Too many requests\" }); }\n return res.status(200).json({ token });";
const widenedResult = simulateEdit(fileContent, { path: "src/api/login.ts", old_string: widenedOld, new_string: widenedNew });
console.log("widened:", widenedResult.message);
const globalResult = simulateEdit(fileContent, { path: "src/api/login.ts", old_string: narrow, new_string: narrow, replace_all: true });
console.log("replace_all:", globalResult.message);
}
widenedEditExample("a\n const token = await issueToken(user);\n return res.status(200).json({ token });\nb\n const token = await issueToken(user);\n return res.status(200).json({ token });\nc");Production code and configuration: Example 4: Guarded local shell using settings.json allow ask deny
What this proves: Bash is the powerful local tool that requires scoping by command pattern in settings.json, deny rules are the hard boundary that survive permissive modes, and per-invocation flags --permission-mode and --allowedTools are the correct way to scope a -p run without touching the project file.
Failure boundary: placing rules in CLAUDE.md instead of settings.json is advisory and can be misinterpreted, while a permissions.deny entry such as Bash(rm -rf *) or Read(./.env) is enforced by the tool layer before the model's judgment. In non-interactive -p mode an operation not covered by an allow rule or by the chosen permission baseline aborts rather than running silently. Wildcard tool names for built-ins such as Read or Bash are scoped with parenthesized patterns, not with a wildcard tool name, except after a literal mcp__<server>__ prefix.
Observable output: a validated settings.json that passes schema checks, a run where the deny blocks a destructive command despite an otherwise permissive baseline, and a -p invocation that aborts with a non-zero exit when an allow is missing and succeeds when the minimal allow is added.
{
"$schema": "https://json.schemastore.org/claude-code-settings.json",
"permissions": {
"allow": [
"Read(**/*)",
"Grep(**/*)",
"Glob(**/*)",
"Bash(npm test)",
"Bash(npm run lint)",
"Bash(npm run typecheck)",
"Edit(**/*)"
],
"ask": [
"Write(**/*)",
"Bash(npm install *)"
],
"deny": [
"Bash(rm -rf *)",
"Bash(sudo *)",
"Bash(chmod 777 *)",
"Read(./.env)",
"Read(./secrets/**)"
]
}
}#!/usr/bin/env bash
set -euo pipefail
# Guarded read-and-report job: no human present to approve, so dontAsk with a minimal allow
npx claude-code -p "Grep for processLegacyOrder and summarize callers" \
--permission-mode dontAsk \
--allowedTools "Read,Glob,Grep,Bash(npm test)" \
--output-format json > report.json
jq -e '.result' report.json > /dev/null
# Auto-fix job: accept edits but keep network installs behind an explicit rule
npx claude-code -p "Apply rate limiting to src/middleware/auth.ts and verify with tests" \
--permission-mode acceptEdits \
--allowedTools "Read,Edit,Write,Grep,Glob,Bash(npm test),Bash(npm run typecheck)" \
--max-turns 20 \
--output-format json > fix.json
jq -e '.structured_output // .result' fix.json > /dev/null || echo "fix: no structured output, inspecting text result"
# Intentionally missing allow: this should abort with non-zero exit in non-interactive mode
if npx claude-code -p "Install a new dependency" --permission-mode dontAsk --allowedTools "Read,Edit" --output-format json > /tmp/no-allow.json 2>&1; then
echo "unexpected success: allow check did not abort"
else
echo "correctly aborted with exit $?: verify deny precedence and add Bash(npm install) if needed"
fiProduction code and configuration: Example 5: Server-side sandboxed code execution with versioned type
What this proves: the API's code_execution tool is a server-side sandboxed surface with versioned types code_execution_20250825 and later, bash_code_execution and text_editor_code_execution sub-tools, structured result fields stdout, stderr, return_code, content file entries, and a reusable top-level container identifier, distinct from Claude Code's local Bash.
Failure boundary: the container has no internet access, so package installation at runtime is not available and only pre-installed libraries can be used. Execution time beyond the maximum returns execution_time_exceeded, unavailable returns unavailable, and output_file_too_large or invalid_tool_input cover the remaining per-tool errors. A reused container id that has expired cannot be restored and requires a fresh request without it.
Observable output: a response that interleaves server_tool_use with bash_code_execution input and a bash_code_execution_tool_result with captured stdout for the calculation, plus container.id to keep files between requests.
import anthropic
client = anthropic.Anthropic()
response = client.messages.create(
max_tokens=4096,
messages=[
{"role": "user", "content": "Use the code execution tool to calculate the mean and standard deviation of [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]"}
],
tools=[{"type": "code_execution_20250825", "name": "code_execution"}],
)
container_id = None
for block in response.content:
if getattr(block, "type", None) == "bash_code_execution_tool_result":
result = block.content
assert getattr(result, "type", None) == "bash_code_execution_result"
print(result.stdout)
print("return_code:", result.return_code)
if hasattr(response, "container") and response.container:
container_id = response.container.id
print("container:", container_id)Production code and configuration: Example 6: Server-side text editor with view str_replace create insert and the uniqueness guard
What this proves: the str_replace_based_edit_tool with versioned types text_editor_20241022 through text_editor_20250728 drives a host-implemented loop over view, str_replace, create, and insert, with max_characters controlling truncation and str_replace requiring an exact literal match on old_str including whitespace.
Failure boundary: the host must return is_error: true with a message when the file is not found, permission is denied, no match exists for old_str, or old_str matches multiple locations. The model is expected to widen the anchor on the next attempt rather than the host guessing the intent.
Observable output: a view tool_result with numbered file lines, a blocked str_replace with Found N matches, and a succeeding widened replacement with a single-site diff, showing the same guard that Claude Code's local Edit enforces locally.
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic();
const response = await client.messages.create({
max_tokens: 1024,
tools: [
{
type: "text_editor_20250728",
name: "str_replace_based_edit_tool",
max_characters: 10000
}
],
messages: [
{ role: "user", content: "There is a syntax error in my primes.py file. Can you help me fix it?" }
]
});
for (const block of response.content) {
if (block.type === "tool_use" && block.name === "str_replace_based_edit_tool") {
const input = block.input as { command: string; path: string; view_range?: [number, number]; old_str?: string; new_str?: string };
if (input.command === "view") {
console.log(`view ${input.path} view_range=${JSON.stringify(input.view_range ?? [])}`);
} else if (input.command === "str_replace") {
console.log(`str_replace ${input.path} old_str chars=${input.old_str?.length ?? 0}`);
}
}
}
async function handleEditorTool(input: { command: string; path: string; old_str?: string; new_str?: string; file_text?: string; insert_line?: number; insert_text?: string }, fileContent: string) {
if (input.command === "str_replace" && input.old_str !== undefined) {
const needle = input.old_str;
let count = 0; let pos = 0;
while ((pos = fileContent.indexOf(needle, pos)) !== -1) { count++; pos += needle.length; }
if (count === 0) return { is_error: true, content: "Error: No match found for replacement. Please check your text and try again." };
if (count > 1) return { is_error: true, content: `Error: Found ${count} matches for replacement text. Please provide more context to make a unique match.` };
return { is_error: false, content: "Successfully replaced text" };
}
return { is_error: false, content: "ok" };
}
void handleEditorTool;Production code and configuration: Example 7: Server-side web retrieval with versioned search and fetch
What this proves: web_search with types web_search_20250305 and later uses allowed_domains or blocked_domains exclusively, optional user_location with type approximate, max_uses to bound searches, allowed_callers to control direct versus code-execution dynamic filtering, and response_inclusion on web_search_20260318 and later to drop internally consumed nested blocks. Results arrive as web_search_tool_result blocks with web_search_result entries and citations that require faithful echo of encrypted_content and encrypted_index.
Failure boundary: providing both allowed_domains and blocked_domains fails the request, an unsupported country code for user_location fails with a 400, exceeding max_uses returns max_uses_exceeded, and missing or altered encrypted_content on continuation fails validation.
Observable output: an assistant turn that contains server_tool_use for web_search with query, a web_search_tool_result with url, title, page_age, and encrypted_content, and final text interleaved with citations of type web_search_result_location carrying cited_text up to 150 characters.
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic();
const response = await client.messages.create({
max_tokens: 1024,
messages: [{ role: "user", content: "What product changes were announced this month for the payments package? Ground the answer with live sources." }],
tools: [
{
type: "web_search_20250305",
name: "web_search",
max_uses: 5,
blocked_domains: ["untrustedsource.com"],
user_location: {
type: "approximate",
city: "San Francisco",
region: "California",
country: "US",
timezone: "America/Los_Angeles"
}
}
]
});
for (const block of response.content) {
if (block.type === "server_tool_use" && (block as unknown as { name: string }).name === "web_search") {
console.log("web_search query:", (block as unknown as { input: { query: string } }).input.query);
}
if (block.type === "web_search_tool_result") {
const result = block as unknown as { content: Array<{ type: string; url: string; title: string; page_age?: string }> };
if (Array.isArray(result.content)) {
for (const entry of result.content) {
if (entry.type === "web_search_result") {
console.log(`result ${entry.title} ${entry.url} age=${entry.page_age ?? "unknown"}`);
}
}
}
}
}Mechanism and API surface
- Grep for content
- Searches inside files for patterns such as function names or error strings, emitting file path, line number, and line content without path matching, and is the only choice for finding callers by name.
- Glob for path
- Matches file names by pattern such as star star slash star dot test dot tsx or star star slash config dot star, emitting matching paths without reading contents, and is the purpose built choice for test suffix or extension discovery.
- Edit via unique anchor
- Accepts old_string and new_string with exact text matching and optional replace_all. On non unique match the tool fails by design with count rather than guessing, and the correct recovery is to widen the anchor with surrounding signature or comments until unique.
- Read plus Write as fallback
- Read returns a requested window, Write replaces or creates the file with full content. Cost scales with file size on every use, so this path is the documented last resort after widened anchor and replace_all have been tried.
- Incremental discovery ledger
- Grep for entry points, Read only the hit files to follow imports, Grep for wrapper and alias names, Read only what that step justifies. No bulk upfront reads, with each Read justified by a prior hit.
- Wrapper and barrel tracing
- After finding the defining file via Grep and reading it for exported and re exported names, Grep for each name plus the barrel import path, catching consumers that never mention the deprecated function directly.
The decision rules in play
Each rule below is a distinct decision the exam tests, stated with its mechanism, the boundary where the answer flips, and the wrong answers written against it.
Grep searches file contents for patterns, Glob matches file paths by name
Two distinct built-in tools handle discovery. Grep opens file contents and tests each line against a text or regex pattern, returning matching lines with file paths and line numbers. Glob tests file paths and names against a glob pattern such as */.ts and returns only paths, without opening any file. The model selects between them by asking whether the question is about what is inside files or what files are named.
In both examples the return type is different by design. Grep gives evidence lines that justify the next Read. Glob gives a manifest of paths that may be opened or counted without reading contents.
// Flag: exact parameter names inferred from evidence, verify against official docs
// Grep: content search - finds what is inside files
await Grep({ pattern: "processLegacyOrder", path: "src" })
// Returns: src/OrderProcessor.ts:42: await processLegacyOrder(orderId)
// Glob: path matching - finds files by name
await Glob({ pattern: "**/*.test.ts" })
// Returns: src/OrderProcessor.test.ts, src/RefundHandler.test.tsFile path and file content are disjoint namespaces. A filename containing auth does not imply the file imports a module named auth, and a file importing auth rarely has auth in its filename. Using the wrong namespace produces systematic false positives and false negatives at once. the tested material repeatedly shows agents that search filenames when they should search contents missing most real callers, and agents that search contents when they should match paths returning noise.
Boundary. The deciding question is "does the answer depend on characters inside the file?" If yes, the answer is Grep. If the answer depends on extension, directory, or naming convention regardless of contents, the answer is Glob. The boundary flips when a task mixes both. Finding callers of a deprecated function is Grep because it depends on call sites inside files. Finding the sibling test file for a caller is Glob because it depends on the naming convention CallerName.test.* regardless of what the test contains. Tasks that ask for both in sequence require Grep then Glob, not one tool for both.
Recurring specifics. - Grep patterns seen: processLegacyOrder, verifyToken\(, import.*@company/auth, SYNC_CONFLICT, entity version mismatch, ERR_731, deprecation_warning, fact_revenue_daily, InvalidStateTransition, eval\(, timeout as literal - Glob patterns seen: /.test.ts, /.spec.ts, /.test.tsx, /.config., /webpack.config., config//.json, /migrations/.sql, frontend//.test.tsx, src//.js, content/domains//*.mdx - Grep returns file path plus line number plus matching line. Glob returns file paths only. - Common trigger phrases: "find files that import" or "find usages" or "find all callers" map to Grep. "find all test files" or "find config files" or "find all TypeScript files" map to Glob.
Proposal. use Glob with a pattern like */processLegacyOrder* to find callers of a function named processLegacyOrder.
Why it attracts. glob patterns feel like search and the function name appears in the pattern.
Why it fails. Glob tests only the path string. A file named OrderProcessor.ts that calls the function is missed, and a file named processLegacyOrderHelper.ts that never calls it is returned.
When it would be right. when the task is to find files whose names contain that token, not files whose contents call it.
Proposal. use Glob with */auth to find files importing @company/auth.
Why it attracts. the package name contains auth.
Why it fails. most files that import the package have unrelated names such as userController.ts. Both false positives and massive false negatives result.
When it would be right. when the task asks for files whose names contain auth, not files that import the package.
Proposal. use Grep with pattern *.test.ts to find test files.
Why it attracts. Grep is a search tool and the pattern looks like what is sought.
Why it fails. Grep searches inside file contents. It may find a file that contains the string *.test.ts in a comment while missing every actual test file.
When it would be right. never for path enumeration. The narrow case where Grep for test inside files overlaps with test files is coincidence, not correctness.
- - Mutation: function name appears as
calculateTaxin the library and as aliasescomputeOrderTaxandcomputeInvoiceTaxin wrappers. Single-nameGrepbecomes insufficient and the alias-enumeration rule takes over, but the content-search choice itself does not change. - Mutation: file type filter added such as "only TypeScript files that importformatDate." The correct narrowing isGrepfor the import pattern, optionally scoped by path, notGlobfor the path alone.
Glob is the correct tool for extension and naming-pattern enumeration
Glob is invoked with a glob pattern and returns every file path that matches, regardless of file contents. The agent does not open files to decide. Typical patterns include an extension anchor such as /.json, a naming convention such as /.test., or a directory scope such as config//.json. The returned list is then used as input to Read, to a migration plan, or to a report manifest.
For large repositories the pattern should be scoped. src//.js is preferred over /.js because the latter scans unrelated directories such as node_modules and may hit file-count limits, increasing latency and noise.
# Equivalent shell intuition - Glob replaces manual find invocations
# Flag: shell equivalents illustrative only, built-in Glob is preferred
# Find all TypeScript test files - Glob
Glob("**/*.test.ts")
# Find all JSON configs under config - Glob
Glob("config/**/*.json")
# Find migration scripts by naming convention - Glob
Glob("**/*_*.sql") # matches YYYYMMDD_description.sqlPath enumeration is a metadata operation, not a content operation. Reading file contents to test a filename is wasteful by orders of magnitude, and content-searching for a filename token is semantically wrong. A dedicated path matcher does the job in one call, with output that is already the answer for tasks like "list every test file" or "present a manifest of config files."
Boundary. Use Glob when the selection criterion is computable from the path alone. If any part of the criterion requires inspecting file contents, Glob alone is insufficient. Finding migration files by naming convention YYYYMMDD_description.sql is Glob because the date prefix is in the filename. Finding files that reference a Snowflake table fact_revenue_daily is Grep because the table name appears inside file contents. A task that asks for "all .ts files that define React components" reveals a mixed case: Glob for /.tsx covers JSX, but *.ts files defining components via React.createElement are missed, so Grep for React.FC or React.createElement must complement Glob.
Recurring specifics. - Patterns: /.test.ts, /.spec.ts, /.test.tsx, /OrderProcessor.test., /.config., /config., /migrations/.sql, /_.sql, /.json, src//.js, frontend//.test.tsx, content/domains//.mdx - Scoping advice: prefix with directory such as src/ or config/ to avoid scanning node_modules or build output. - Output is paths only, often hundreds of files, and is frequently the direct input to a subsequent Grep or to selective Read calls.
Proposal. use Grep with a keyword like test to find test files.
Why it attracts. test files contain the word test.
Why it fails. many non-test files contain test, and some test files may not contain that keyword in a searchable way. It is a content approximation of a path criterion.
When it would be right. never for pure path enumeration.
Proposal. use Bash with find. -type f -name "*.test.ts" piped to analysis.
Why it attracts. shell find is familiar and flexible.
Why it fails. it bypasses the built-in path tool that is purpose-built, stable, and integrated with context accounting. the tested material penalizes routing through Bash when built-ins exist.
When it would be right. when built-ins are unavailable or when a shell pipeline genuinely needs custom post-processing beyond simple listing, but still not as the default.
Proposal. use Read on every directory listing manually.
Why it attracts. reading feels thorough.
Why it fails. it requires sequential Read calls per directory and scales poorly across dozens of directories.
When it would be right. never for enumeration.
- - Mutation: ask for
/.test.tsversus/.spec.tsversus/.test.. All areGlob, only pattern specificity changes, including the combined/.{test,spec}.. - Mutation: ask for files under a specific directory such assrc//.jsversus whole repo/.js. The scoped form is preferred for token cost and noise reduction.
Grep is the correct tool for literal-string, error-message, and import-statement content search
Grep is invoked with a text or regex pattern and scans file contents across the repository. It returns every file path and line where the pattern occurs, often with surrounding context depending on implementation. This is the tool for questions shaped as "find every file that contains X" where X is a string inside files.
The pattern verifyToken\( with the opening parenthesis is a recurring refinement: it anchors to calls specifically, filtering out comments, imports, or helper names that merely mention the token without calling.
// Flag: parameter shape illustrative, verify exact Grep schema in official docs
// Find every import of a deprecated module
await Grep({ pattern: "from '@company/auth'" })
// Find the source of a production error
await Grep({ pattern: "SYNC_CONFLICT" })
// Find every call site of a function before changing its signature
await Grep({ pattern: "verifyToken\\(" })
// Find every reference to a table name
await Grep({ pattern: "fact_revenue_daily" })Content questions are answered by content evidence. Filename heuristics cannot tell whether a file calls a function, logs an error, or imports a module. Only the bytes inside files contain imports, call sites, error strings, and table references. Grep is the single operation that answers "which files contain this pattern" directly, and its output is the minimal justification for every subsequent Read.
Boundary. Use Grep when the pattern must appear verbatim or as a regex inside file text. The opposite is Glob when the pattern is a path convention. A frequent exam trap is to present a content question with a path-sounding distractor, for example "find files that import config_loader" with a Glob option that matches /config_loader. The path match finds files named for the module, not files that import it. The true signal is the import statement inside the file, which only Grep can find. Similarly, a question about test files that actually asks "find tests that exercise processLegacyOrder transitively through a wrapper" still requires Grep for the wrapper name after Glob has found the sibling test paths.
Recurring specifics. - Error localization strings: SYNC_CONFLICT, entity version mismatch, InvalidStateTransition, ERR_731, NullPointerException-related messages - Import patterns: import.@company/auth, import.from 'utils/auth', require('@company/auth'), billing_utils, config_loader - Call patterns: processLegacyOrder, processPayment(, sendNotification, formatCurrency, verifyToken(, eval( - Table and endpoint strings: fact_revenue_daily, /api/v1/users/create, POST /api/v1/users/create - Return includes file path, line number, and matching line, enabling targeted Read with offset at the relevant region.
Proposal. use Glob with */eval to find files using eval().
Why it attracts. the dangerous function name appears in the pattern.
Why it fails. filename has no correlation with whether file contents call eval(). Most callers are named for their purpose, not the function they contain.
When it would be right. when searching for files whose names start with eval.
Proposal. use Read on the main entry file then follow imports recursively.
Why it attracts. it traces reachable code.
Why it fails. it misses dynamically loaded modules, test files, utility scripts, and any file not reachable from the entry point, and it exhausts context before covering the graph.
When it would be right. after Grep has identified entry points, Read then follows imports incrementally from those specific files.
Proposal. use Bash with ls -R | grep eval or find ... | xargs grep as the most flexible approach.
Why it attracts. shell pipelines are powerful and familiar.
Why it fails. built-in Grep is purpose-built, structured, and more reliable without shell error handling or output parsing, and the exam explicitly prefers built-ins over Bash for standard search.
When it would be right. for transformations that truly need shell composition, not for standard content search.
- - Mutation: error string is highly distinctive such as
SYNC_CONFLICT: entity version mismatch. The most efficient search is the literal distinctive substring, not a search for error-handling infrastructure or directory names. - Mutation: function appears under multiple names due to wrappers, so a singleGrepbecomes incomplete and must be repeated per alias after alias discovery.
Search before Read - narrow with Grep or Glob before loading file contents
The correct sequence is a cheap, broad search to identify the relevant file subset, then expensive, narrow Read calls on only those files. The search acts as a filter, Read acts as inspection. No file is opened until the search has justified it.
A variant is Glob then Read when the criterion is a path pattern, but Glob then Read for a content question is still an anti-pattern because the path prefilter is irrelevant.
// Anti-pattern: Read everything, then decide
for (const file of allFiles) await Read({ path: file }) // exhausts context
// Correct: Grep then Read only hits
const hits = await Grep({ pattern: "processPayment" })
for (const file of hits.files) await Read({ path: file })Read loads full file contents into the model context, paying token cost for every byte regardless of relevance. Grep and Glob return only paths and matching lines, which is orders of magnitude cheaper. In a large repository the cost difference is the difference between answering in one turn and exhausting the window before the answer is found. the tested material ranks "read all files upfront" as the single biggest context anti-pattern.
Boundary. Search-then-read holds whenever the set of relevant files is unknown and must be discovered. The opposite becomes correct when the file is already known precisely, for example when the user says "read src/auth/middleware.ts and explain it." There, direct Read is appropriate because no discovery is needed. The boundary is discovery versus known-target. If the task says "find every file that" anything, search is mandatory before reading.
Recurring specifics. - Typical sequences: Grep for an error code then Read the matching file. Grep for import pattern then Read each importer. Glob for */.test.ts then Read a specific test manifest. - Context costs: a repository of hundreds of files read up front can consume tens of thousands of tokens before any reasoning, while Grep then Read pays tokens proportional to relevance. - Artifacts: scratchpad files and Glob manifests help preserve findings when many files are eventually read.
Proposal. use Glob to list all */.js then Read each one sequentially to look for a function name.
Why it attracts. it enumerates candidates.
Why it fails. it is the same as reading all files when the filter is not content-relevant. Grep for the function name filters directly.
When it would be right. when the question is about file-path enumeration, not content.
Proposal. start by reading README.md or CLAUDE.md plus directory listings, then read candidate files.
Why it fails. it builds high-level context. Why it fails for content search: it still reads without a content anchor. For content questions, Grep for the distinctive string is faster and more direct than building a structural map first.
Proposal. rely on Write to create an index file listing class names then search the index.
Why it attracts. it avoids rescanning.
Why it fails. it adds a write-and-maintain step for information Grep already provides in one call, and the index goes stale.
- - Mutation: codebase is 5,000 files and the error code is
ERR_731. The argument for search-first strengthens because cost scales with repository size. - Mutation: file set is known to be fifteen files spanning decorators and middleware around 8,000 lines total. Search-first still applies, but architectural entry-point analysis may complement it by reading the base class first.
Incremental discovery is mandatory - never read all files upfront
Incremental discovery means each tool call is justified by the previous result. The agent starts with a narrow anchor such as a function name, class name, or error string, runs Grep to find entry points, reads only those hits, follows imports or call sites discovered in those reads with further targeted Grep or offset Read calls, and repeats. At no point does it load the whole repository or a whole module directory before it has a concrete lead.
The offset and limit pattern seen in the tested material is intentional: after Grep gives a line number, Read with a window around that line avoids loading an entire 600-line file when only the handler around the error matters.
// Incremental pattern - each step justified by prior evidence
const entryFiles = await Grep({ pattern: "InvalidStateTransition" })
const entryContent = await Read({ path: entryFiles.files[0] })
// Follow imports discovered in that file
const importedModule = parseImport(entryContent) // e.g. "./state-machine"
const relatedFiles = await Grep({ pattern: "state-machine" })
const relatedContent = await Read({ path: relatedFiles.files[0], offset: 120, limit: 80 })Context budget is finite and attention degrades as token count grows. Loading every file upfront spends budget on irrelevant content at the highest cost, and later reasoning must compete with that noise. Incremental discovery keeps cost proportional to relevance and preserves signal: every byte in context is there because an earlier step proved it matters.
Boundary. Incremental discovery holds whenever relevance is unknown at the start, which is the normal case for unfamiliar codebases, legacy systems, or production bugs. The opposite becomes correct when the task scope is already bounded, such as "add a helper between two functions in this known 150-line module." There, the file is the scope, so direct Read plus localized Read with offset is appropriate without a broad Grep. Even then, the principle remains: do not expand to unrelated directories.
Recurring specifics. - Anti-patterns flagged: Read every file in the repository, Glob all */.js then Read each, recursive import traversal from an entry point loading every reachable file, Glob 200 modules before reasoning. - Correct pattern named in the reference material: Grep for entry points, Read to follow imports, Grep again to trace usage, Read only what is revealed as relevant. - Scenarios of scale: 200 files, 350 files across 40 directories, 500,000 lines across 200 modules, 1,400 files across forty modules, 5,000-file codebase. Scale increases the penalty for bulk loading but does not change the pattern. - Mitigations for unavoidable multi-file work: delegate to subagents and use scratchpad files rather than accumulating raw reads in the main conversation.
Proposal. read the top-level entry file then recursively read every imported file.
Why it attracts. it follows the dependency graph.
Why it fails. the graph from an entry point in a large codebase contains many files with no reference to the target symbol, exhausting context before the search completes.
When it would be right. as a second step after Grep has already identified that a specific import chain is relevant, not as the initial sweep.
Proposal. use Glob to enumerate all source files under the checkout and pricing modules then Read each fully before reasoning.
Why it attracts. it claims completeness.
Why it fails. it still front-loads large file sets before any concrete lead, saturating the window and causing later edits in the wrong module.
When it would be right. never as an initial sweep. After Grep identifies that those modules contain discount logic, incremental offset Read inside them is correct.
Proposal. increase context window or split the repository into microservices so everything fits.
Why it attracts. it removes the constraint instead of working within it.
Why it fails. context size is a model limit, not a configuration flag, and refactoring a codebase to accommodate the agent reverses the responsibility.
When it would be right. never as the primary fix. Targeted search plus selective read is the intended solution.
- - Mutation: error string is known and distinctive. Incremental discovery collapses to one
Grepplus oneReadof the hit, the shortest instance. - Mutation: function is defined in a core library and re-exported through wrappers. Entry-pointGrepfinds the definition,Readreveals aliases, then per-aliasGrepplus offsetReadtraces consumers.
Edit is the default for targeted file modification using a unique anchor
Edit performs a surgical string replacement. The agent supplies old_string and new_string. The tool finds the single occurrence of old_string in the file and replaces it with new_string without rewriting unrelated lines. Only the changed lines travel in the request, not the whole file.
If old_string matches exactly one location the operation succeeds atomically. If it matches zero or multiple locations the tool fails with a non-unique-match error, which is a safety mechanism, not a defect.
// Flag: field names illustrative, verify exact Edit schema in official docs
await Edit({
path: "src/OrderProcessor.ts",
old_string: "function processOrder(id: string)",
new_string: "function processOrder(id: string, validate: boolean = true)"
})Targeted replacement minimizes context cost and risk. Read plus Write loads the entire file, holds it in context, and writes it back, paying token cost for every unchanged line and creating a larger surface for transcription errors. Edit touches only what must change, so token cost stays proportional to the edit and the unchanged majority never enters the context budget for writing.
Boundary. Edit holds when the surrounding context can be made unique with a short anchor, typically a few lines including a distinctive neighboring statement, comment, or signature line. The opposite becomes correct when uniqueness is impossible at reasonable length, for example when a file contains six identical boilerplate sections differing only in trailing logic, or when the literal to change is a short token such as timeout that appears as a substring inside other keys. There the unique anchor cannot be constructed without copying dozens of lines accurately, so fallback to Read plus Write becomes appropriate.
Recurring specifics. - Success requires exact character match including whitespace and line breaks. Even a single space difference causes failure. - Cost contrast: two small changes in a 500-line file cost a few lines via two Edit calls versus the full 500 lines via Read plus Write. - Edit with replace_all: true is a separate path for intentional global substitution and is covered as its own rule. - Evidence penalizes defaulting to Read plus Write for every modification and penalizes jumping straight to it after a single non-unique failure.
Proposal. use Read then Write for every change because it is simpler and more reliable.
Why it attracts. it always succeeds regardless of uniqueness.
Why it fails. it burns tokens on the whole file for what is usually a one-line change, which the tested material explicitly flags as wasteful and penalized.
When it would be right. when Edit genuinely cannot be made unique after widening, or when a structural insertion in the middle of repetitive code makes anchoring fragile.
Proposal. use Bash with sed for every replacement.
Why it attracts. sed is powerful and familiar.
Why it fails. it bypasses the built-in edit safety checks, is harder to reason about in reviews, and in evidence it introduces collateral rewrites such as rewriting socket_timeout when only timeout was intended.
When it would be right. for collapsing a thousand-file bulk update into a scripted pipeline, not for a single targeted surgical change.
Proposal. use Grep to verify existence then Write the whole file.
Why it attracts. it confirms the target exists.
Why it fails. Grep finds the pattern but Write still overwrites the file unnecessarily. Edit with a unique anchor is more precise and cheaper.
When it would be right. Grep before Edit can be useful reconnaissance, but Write after it is not the correct modification primitive.
- - Mutation: file is 800 lines and only one signature changes.
Editremains the answer with a single unique anchor. - Mutation: file is 200 lines and the same URL appears eight times and all must change. A singleEditwithreplace_allis the correct mutation, not eight separateEditcalls.
Non-unique anchor failure triggers widening - expand old_string until it is unique
When Edit reports that old_string matches multiple locations, the documented first recovery is to include more surrounding context in old_string so the match becomes unique. The agent uses Grep or prior Read to see which occurrence is the intended one and what lines surround it, then constructs a longer anchor that pins only that site.
A second built-in recovery variant is replace_all: true when every occurrence should be changed. Both stay on Edit and cost almost no extra context.
// First attempt - too short, matches three locations
await Edit({ path: "config/app.ts", old_string: "timeout: 30", new_string: "timeout: 60" })
// Error: old_string matches 3 locations
// Recovery - widen with surrounding context unique to the target site
await Edit({
path: "config/app.ts",
old_string: " // Primary API timeout\n timeout: 30,\n retries: 3",
new_string: " // Primary API timeout\n timeout: 60,\n retries: 3"
})Non-unique failure is a safety gate. Without it the tool might change an unintended site. Widening preserves that safety while keeping the operation surgical. The extra lines are the minimal disambiguation needed, so cost remains near one Edit rather than a whole file rewrite. Escalating immediately to Read plus Write abandons this safety and efficiency for a larger, riskier rewrite.
Boundary. Widening holds when a unique anchor can be constructed with a modest number of additional lines. The opposite becomes correct when the file contains near-identical blocks repeated with only subtle trailing differences, such as several registration blocks differing only in a wrapper decorator, or boilerplate sections identical across six locations where the only distinguishing context is far away. In those cases constructing a long yet accurate old_string of thirty or more lines is error-prone, and Read plus Write becomes the reliable fallback. the tested material treats reading all 200 files upfront as penalized, but reading the one file that must be rewritten is not.
Recurring specifics. - Error phrasing: old_string matches 3 locations or non-unique match or target text appears in multiple locations. - Recovery instruction phrasing: widen old_string with more surrounding lines until it pins down one location, or set replace_all: true if every occurrence should be updated. - Cost comparison: widening reuses the same Edit call with a longer string, a few extra lines of context. Read plus Write loads and writes the entire file. - Evidence explicitly calls jumping straight to Read plus Write an escalation shortcut and a penalized anti-pattern.
Proposal. immediately fall back to Read plus Write after any non-unique failure.
Why it attracts. it always works and feels pragmatic.
Why it fails. it burns context tokens for what is usually a one-line change with a few extra anchor lines, and the tested material penalizes this as wasteful when widening would have succeeded.
When it would be right. when widening cannot achieve uniqueness at reasonable length due to repeated blocks.
Proposal. retry the same short old_string repeatedly or request a higher retry limit.
Why it attracts. it hopes for transient success.
Why it fails. deterministic matching will fail identically each time, wasting turns.
When it would be right. never.
Proposal. use Grep to get line numbers then Edit by line number.
Why it attracts. line numbers feel precise.
Why it fails. Edit does not accept line numbers, only text anchors. The correct use of Grep is to find which occurrence is intended and what surrounds it, then widen the text anchor.
When it would be right. for offset Read to inspect the intended occurrence, not for Edit invocation itself.
- - Mutation: variable
userDatatocustomerDataappears twelve times and all should change. Widening is the wrong recovery,replace_all: trueis correct. - Mutation: file has repetitive docstrings and the insertion is between two functions with similar structure. Widening by a short anchor still fails due to repetition, soReadplusWritebecomes correct.
replace_all performs intentional global substitution in a single Edit operation
Edit accepts a replace_all flag. When set, the tool replaces every occurrence of old_string with new_string in one atomic operation, without requiring multiple Edit calls. This is the correct path when every instance of a literal should be changed throughout the file.
Without replace_all, Edit expects a unique match and fails if the literal appears more than once. With it, multiplicity is the intent.
// Rename a variable everywhere in one call
await Edit({
path: "src/session.ts",
old_string: "userData",
new_string: "customerData",
replace_all: true // Flag: verify exact field name in official Edit schema
})
// Replace every occurrence of an old endpoint URL
await Edit({
path: "src/api/routes.ts",
old_string: "POST /api/v1/users/create",
new_string: "POST /api/v2/users",
replace_all: true
})Global substitution is a distinct intent from single-site editing. Repeating Edit per occurrence would cost one call per site and risk partial completion if one call fails. replace_all expresses the intent once, applies it consistently, and succeeds or fails as a unit.
Boundary. replace_all holds when every occurrence of the exact literal should be updated and no substring distinction exists. The opposite becomes correct when only a subset must change, especially when a short literal appears as a substring inside longer identifiers. In a 60-file rename from timeout to request_timeout_ms, a global replace_all of timeout would also rewrite socket_timeout and comments. There, per-occurrence disambiguation with unique-anchor Edit plus Read plus Write fallback for non-unique sites is required. the tested material presents a platform team that used sed 's/timeout/request_timeout_ms/g' and broke three services for exactly this reason.
Recurring specifics. - Literals that require global replacement: userData, timeout when isolated, POST /api/v1/users/create, copyright year in headers when the task is intentionally global across many files. - Efficiency claim: one replace_all call versus N separate Edit calls or Read plus Write of the whole file. - Evidence pairs replace_all: true with "if all occurrences should be updated" as the exact documented phrasing.
Proposal. call Edit twelve times, once per occurrence with unique surrounding context.
Why it attracts. it reuses the unique-anchor pattern.
Why it fails. it is inefficient by a factor of N and introduces N failure points.
When it would be right. when only one or a few of the twelve occurrences should change and each requires distinct disambiguation.
Proposal. use Read plus Write to load the file and replace in memory.
Why it attracts. it handles any multiplicity.
Why it fails. it loads the entire file when replace_all achieves the same result without that overhead.
When it would be right. when some occurrences must be left unchanged and per-occurrence disambiguation is needed.
Proposal. use Bash with sed for global replace.
Why it attracts. shell sed is concise.
Why it fails. it bypasses the structured built-in and risks collateral rewrites of substrings.
When it would be right. for a thousand-file batch where each file's operation is mechanically identical and a single script replaces a thousand Write calls, but even there a scoped Grep then per-file replace_all may be safer when substring risk exists.
- - Mutation: literal
timeoutappears insidesocket_timeoutand comments that must stay unchanged.replace_allbecomes the wrong choice, per-occurrence anchoredEditis correct. - Mutation: eight occurrences in one file, all should change but the file is 200 lines.replace_allin one call is more efficient than eight separateEditcalls or aReadplusWrite.
Read plus Write is the last-resort fallback when Edit cannot be made unique
When widening cannot achieve a unique anchor at reasonable length, the agent loads the full file with Read, modifies the content in memory at the correct location by line index or structural cue, and writes the complete updated file back with Write. This bypasses the uniqueness requirement entirely because no anchor matching is performed.
A variant is a small Read slice with offset then Write of the corrected whole, but the essential pattern is load, patch in memory, persist.
// Fallback path when Edit cannot anchor
const content = await Read({ path: "utils/helper_module.py" })
const lines = content.split("\n")
const insertionPoint = findInsertionPoint(lines) // e.g. between def function_a and def function_b
lines.splice(insertionPoint, 0, newHelperFunction)
await Write({ path: "utils/helper_module.py", content: lines.join("\n") })Edit requires exact text identity between old_string and a single file region. In files with repetitive structure such as near-identical boilerplate sections, identical registration blocks differing only in wrapper logic, or repeated docstrings and variable names, the set of surrounding lines that distinguish one occurrence may be long and error-prone to reproduce. Read plus Write removes that transcription burden by operating on line-level control rather than string-anchored control, guaranteeing the change lands at the intended offset.
Boundary. Fallback holds when reasonable widening still collides or would require copying thirty or more lines with whitespace sensitivity. The opposite remains Edit with widening when a modest context addition suffices. the tested material is explicit that the ordered preference is widen then replace_all then Read plus Write, and that jumping straight to the fallback without attempting widening wastes context and is penalized. The cost contrast is deliberately framed: fallback costs a file's worth of tokens versus a few lines for widened Edit.
Recurring specifics. - Triggers: file has repetitive timeout lines, six identical boilerplate sections, several near-identical response-construction lines for header insertion, 150-line utility with repeated docstrings, configuration file with non-unique anchors. - Recovery phrasing: "use Read to load the full file contents and then Write the modified file" as the reliable modification regardless of repetition. - Failure to attempt widening first is characterized as an escalation shortcut and a context-efficiency defect. - Bulk-operation caveat: writing a copyright year across a thousand files via individual Read plus Write would be the most inefficient pattern, a thousand inference turns. Bulk Bashsed collapses it to one turn, but that is a different rule about bulk.
Proposal. split the file into smaller files so each piece contains a unique anchor.
Why it attracts. it manufactures uniqueness.
Why it fails. it restructures the codebase to accommodate a tool limitation instead of using the documented fallback.
When it would be right. never as the corrective step.
Proposal. use Glob to find a different file containing a unique version of the text.
Why it attracts. it searches for uniqueness elsewhere.
Why it fails. the task is to modify this file, not a different one.
When it would be right. never for this problem.
Proposal. use Bash with sed targeting a line number.
Why it attracts. line numbers bypass uniqueness.
Why it fails. line-number addressing is fragile to file changes and bypasses built-in safety, and is explicitly discouraged as error-prone.
When it would be right. for bulk scripted sweeps, not for surgical single-file fallback.
- - Mutation: target text appears in three places and only one should change, but surrounding context is moderately distinctive. Widening remains correct, fallback is not yet needed. - Mutation: target line
mode=RetryMode.BACKOFF,appears two or three times within the same adapter file differing only in trailing logic.Editkeeps colliding, so fallback becomes correct. Usingsedin this mutation corrupts a paragraph because substring matching is too broad.
Bash versus built-in Grep - prefer the built-in for content search
Both Grep and Bash with grep can search file contents, but the exam treats them as non-equivalent. Grep is the purpose-built built-in for content search, with structured parameters, stable output of file path plus line number plus matching line, and no shell error handling or quoting concerns. Bash with a shell find or grep pipeline is the general escape hatch, carrying command-construction risk, unstructured output, and higher latency.
The built-in also avoids the specific ls -R | grep eval trap where a shell pipeline searches directory listings rather than file contents, producing false positives and false negatives.
# Built-in Grep - preferred for standard content search
# Flag: Bash equivalents illustrative to show the contrast, not the recommended path
# Built-in
Grep({ pattern: "ERR_731" })
# Versus Bash equivalent that the tested material penalizes for this task
# Bash({ command: "grep -r 'ERR_731' --include='*.ts' ." })Built-ins are optimized for the operations they cover and integrate cleanly with context accounting. Bash can do the same search but adds failure modes: malformed quoting, missed include filters, reliance on file-system listing rather than content, and output that must be parsed. For standard code search the specialization outweighs flexibility. The reference hierarchy names Grep as the built-in optimized for this operation and labels Bash as the fallback for what built-ins cannot do, such as running tests, builds, and installs.
Boundary. Prefer Grep whenever the operation is standard content search: finding function callers, error strings, import statements, or literal occurrences. The opposite becomes correct when the operation genuinely requires shell execution. Running the test suite after a change, executing a build, installing packages, writing and running a sed script for a thousand-file bulk header update, or invoking gh CLI for pull requests are Bash tasks. The deciding phrase is "is this a standard code search or a shell execution need?"
Recurring specifics. - Phrasing in explanations: "Grep built-in specifically designed optimized for content search", "prefer built-ins over Bash for operations they are designed for", " Bash is the right tool when built-in tools cannot accomplish the task." - Multi-step histories where an agent shells out via Bash with ad-hoc find plus grep that are slow and occasionally malformed, then is corrected to Grep. - The ls -R | grep distractor that searches file and directory names, not contents, producing an evaluation-utils.js false positive and missing data-processor.js that actually calls eval(). - Another Bash distractor: find. -type d -name "auth" that searches directories named with auth rather than files containing imports.
Proposal. use Bash because it is more powerful and should always be preferred.
Why it attracts. shell tools are famously flexible.
Why it fails. power does not imply suitability for a purpose-built operation. the tested material states Bash is appropriate for shell execution, not as a general file operation replacement.
When it would be right. when shell execution is genuinely needed such as running tests or builds.
Proposal. both are equivalent and choice does not materially affect performance.
Why it attracts. both can find the string.
Why it fails. equivalence ignores token efficiency, structured returns, and error handling. the tested material repeatedly marks this as incorrect.
When it would be right. never for standard search.
Proposal. use Bash with grep only for "complex" searches.
Why it attracts. complexity seems to justify shell use.
Why it fails. even complex pattern matching is within Grep scope. Flexibility is needed for custom pipelines, not for pattern complexity alone.
When it would be right. when the pipeline truly composes multiple shell stages beyond simple search.
- - Mutation: task is to run the test suite after making changes.
Bashbecomes the correct choice. - Mutation: task is to execute asedbulk update across a thousand files.Bashbecomes correct as the efficient bulk path.
Bash is correct for shell execution - tests, builds, and read-only git inspection
Bash executes arbitrary shell commands. When a task genuinely needs a shell, there is no built-in substitute. Correct Bash tasks seen in evidence include running the test suite, executing builds, installing packages and npm commands, invoking CLI tools such as gh, running git log for commit history, and writing then executing a sed script for bulk updates.
Scope matters: Bash is for execution needs, not for reimplementing what Grep, Glob, Read, or Edit already do.
# Legitimate Bash tasks - no built-in equivalent
Bash({ command: "npm run test" })
Bash({ command: "npm install" })
Bash({ command: "git log --oneline -20" })
Bash({ command: "eslint --fix" })
# Bulk bulk update - Bash collapses a thousand turns into one
Bash({ command: "find . -type f -name '*.ts' -exec sed -i 's/2024/2025/g' {} \\;" })Built-ins cover file discovery, content search, reading, and targeted modification. They do not cover process execution, package management, or VCS history. The agent loop therefore needs an explicit shell channel for those operations. Without Bash the agent cannot verify changes, install dependencies, or answer "why was this changed" questions whose answer lives in git history or wikis rather than current file text.
Boundary. Use Bash when the operation is process execution or requires shell composition. The opposite is to avoid Bash when a built-in covers the task. Finding callers, listing files by extension, reading a file, or editing a line are built-in tasks. Running tests after editing, exploring history via git log, or collapsing a thousand identical edits into a sed script are Bash tasks. The reference material names the separation: built-ins for file operations, Bash for shell execution like build and test.
Recurring specifics. - Permission mapping: "read_file" plus "bash" is the minimal allow list for a skill that reads files and runs git log. "bash" alone suffices for curl, npm install, npm run lint, eslint --fix, and make clean when those are the shell needs. - Read-only Bash for exploration is treated as low risk: git log, ls -R, and similar inspection without modification are contrasted with destructive Bash that needs confirmation. - Token efficiency note: writing a sed script and running it via Bash for a thousand-file copyright-year update collapses a thousand Write turns into a single reasoning plus execution turn.
Proposal. built-in tools always suffice, Bash should never be used.
Why it attracts. it avoids shell risk entirely.
Why it fails. some operations genuinely require shell execution, and the tested material marks this absolutist position as incorrect.
When it would be right. for a restricted read-only audit scope where write and execution are intentionally denied, but not as a general principle.
Proposal. Bash is only for CI/CD pipelines, not interactive sessions.
Why it attracts. it compartmentalizes risk.
Why it fails. Bash is appropriate whenever shell execution is needed, regardless of environment.
When it would be right. never as a restriction. The question is task type, not pipeline versus interactive.
Proposal. Read or Glob can replace git log for history questions.
Why it attracts. they are read-oriented.
Why it fails. commit history lives in git, not in file contents or paths.
When it would be right. when the history is mirrored in documentation that Grep plus Read can find, but git log via Bash remains the direct source for EVOLUTION questions.
- - Mutation: skill needs to read files and run
git log. Minimal allow list isread_fileplusbash, notbashalone for safety orread_fileplusgitas nogitpermission exists in isolation. - Mutation: skill needs only to search for a pattern viagrep. Required permission isbash, not a separategreppermission, because runninggrepis a shell command.
Bash as general file manipulation is an anti-pattern when built-ins exist
Routing file discovery, reading, or surgical editing through Bash when a built-in covers the task is penalized. Examples include using Bash with find piped to grep to locate callers that Grep finds directly, using Bashls -R plus visual review to enumerate configs that Glob enumerates, using Bashsed for a single targeted line change that Edit handles surgically, or using Bashcat to read a file that Read reads.
The shell works functionally in many of these cases, which makes the distractor plausible, but the tested material flags it as the wrong tool when a built-in exists.
# Anti-pattern - Bash reimplementation of a built-in operation
# Flag: these illustrate what not to do for file operations
Bash({ command: "find . -type f -name '*.js' | xargs grep -l 'processLegacyOrder'" }) # use Grep
Bash({ command: "ls -R | grep auth" }) # use Glob or Grep depending on path vs content
Bash({ command: "sed -i 's/userData/customerData/g' src/foo.ts" }) # use Edit replace_all when isolatedBuilt-ins carry structured parameters, predictable output shapes, and integration with context and permission accounting. Shell pipelines require quoting, error handling for partial failures, output parsing, and concern for shell environment differences such as indirection that can bypass naive string-matching validators. They also trade the surgical safety of anchored Edit for broad textual substitution.
Boundary. The anti-pattern holds for standard file operations: discovery, search, read, write, edit. The opposite becomes correct for bulk or composite shell work that has no built-in equivalent: running tests, building, installing packages, invoking external CLIs, or scripting a thousand-file bulk fix precisely because it collapses turns. The deciding test is "does a built-in already cover this operation cleanly?"
Recurring specifics. - Phrasing: "prefer built-ins over Bash for operations they are designed for", " Bash is a general-purpose escape hatch, combining file discovery and content search in one shell command creates brittle dependencies." - Combining find plus grep in Bash is flagged as functional but not preferred when Grep and Glob exist. - Sequential Read of hundreds of files versus one Grep identifying the same subset is explicitly called wasteful. - The thousand-file sed case is carved out as the exception where Bash is the token-efficient answer.
Proposal. use Bash for everything because it is more powerful.
Why it attracts. one tool for all jobs.
Why it fails. it discards specialization and introduces shell-risk and context-cost regressions.
When it would be right. never as a default. Only for shell-specific needs.
Proposal. both built-in and Bash are equivalent, choice does not matter.
Why it attracts. functional equivalence.
Why it fails. non-functional differences matter for reliability, safety, and cost, and the exam chooses the built-in when both functionally succeed.
When it would be right. never as a content-search answer.
Proposal. use Bashls -R | grep eval to search for dangerous function usage.
Why it attracts. ls plus grep sounds like search.
Why it fails. it searches directory listings, not file contents.
When it would be right. never for content search. Even correcting to grep -r inside Bash, built-in Grep is still preferred.
- - Mutation: task requires search plus
Bashinstall of a dependency. The correct answer mixes:Grepfor search,Bashfor install, notBashfor both. - Mutation: bulk header update across a thousand files.Bashbecomes correct because no built-in collapses a thousand independent writes withoutBash.
Write automatically creates parent directories for nested paths
Write creates a new file at the given path. If the path contains directories that do not yet exist, the tool creates them automatically. No separate mkdir or Bash directory-creation step is required.
This behavior is distinct from Edit and Read, which target existing files.
// Nested path - parent directories created automatically
await Write({ path: "config/env/production.yaml", content: yamlContent })
// No prior Bash({ command: "mkdir -p config/env" }) neededFile creation is a create-or-replace operation on a path, not on a directory entry. Making parent creation automatic removes a common failure mode where a write fails because an intermediate directory was not created first, and it avoids an unnecessary extra tool call and permission.
Boundary. Auto-creation holds when using Write to create or fully replace file contents at a nested path. The opposite needs Bashmkdir -p only if the workflow explicitly wants to create directory structure independent of file writing, or when some external tooling must see the directory before any file exists. For the normal case of writing configuration files into config/env, Write alone is sufficient.
Recurring specifics. - Path example: config/env/production.yaml with missing config/env tree. - Distractors propose Bash with mkdir -p then Write, or Read to verify existence then Write, or full Bash creation of structure plus files in one shell operation. - Correct choice is consistently Write without a preceding directory step.
Proposal. use Bashmkdir -p before Write.
Why it attracts. shell habit.
Why it fails. it adds an unnecessary step when Write already handles parent creation.
When it would be right. when the task is to create directories without writing files.
Proposal. Write fails if parents do not exist and the developer must create them.
Why it attracts. many filesystem APIs fail without parents.
Why it fails. this tool's contract is explicitly auto-create.
When it would be right. never for Write.
Proposal. use Read to verify directory existence then Write.
Why it attracts. verification feels safe.
Why it fails. Read on a directory path does not list contents reliably and adds a wasted call.
When it would be right. never for this creation step.
- - Mutation: agent needs to create both directories and files.
Writealone still handles the combined need. - Mutation: agent needs to create only directories with no file. ThenBashmkdir -pis the tool becauseWriterequires a file target.
Least privilege via allowedTools and per-agent tools field restricts capability structurally
Capability is restricted structurally, not by prose. allowedTools or the tools field in an AgentDefinition or skill frontmatter lists exactly which tools an agent or skill may call. Anything omitted does not exist for that execution. For example, a read-only analysis skill lists Read, Grep, and Glob and omits Edit, Write, and Bash, so no invocation can modify files or run shell commands regardless of what the model is asked to do.
Tool definitions also occupy context, so a narrower tool set reduces noise and helps the model select the right operation.
# Flag: exact frontmatter key name inferred as allowed-tools or allowedTools, verify in official docs
---
allowed-tools: [Read, Grep, Glob]
---// Flag: field name illustrative, verify AgentDefinition schema in official SDK docs
const analyzerDefinition = {
description: "code-analyzer",
tools: ["Read", "Grep", "Glob"], // Edit, Write, Bash omitted - structurally unavailable
prompt: "Inspect architecture but never modify files."
}Prompts influence behavior, but tool configuration is what the runtime enforces. A model can misinterpret guidance, be confused by injected instructions in fetched content, or be persuaded by a user request to act outside its intended role. Removing the capability removes the path entirely, so enforcement does not depend on model compliance. the tested material calls this allowedTools as a hard boundary and contrasts it with advisory instructions that can be violated.
Boundary. Structural restriction holds whenever a role has a narrow, well-defined job: security audit, code review, document analysis, synthesis. The opposite becomes correct when the parent or coordinator still needs broad access while a delegate is narrow. Per-agent scoping achieves this: the parent retains full tools, each subagent receives only its role's tools, and least privilege is satisfied without removing capabilities from the workflow as a whole.
Recurring specifics. - Hard boundary phrasing: allowedTools restriction is a hard boundary, cannot be expanded at runtime, cannot be lifted mid-invocation, request for expansion is denied, and runtime permission expansion is not supported. - Read-only patterns: allowedTools: [Read, Grep, Glob] for security audit, bug investigation that cannot push commits, docs-writer that can read code but not run shell, review PR without file-editing tools. - Cases where the tools field is omitted: subagent inherits every tool from the parent, including Bash, Edit, Write, which is convenient but grants excess capability for a purely analytical role. - Prefixing shell directly with ! is a human shortcut, not a model tool-scope mechanism.
Proposal. add a system prompt line saying never modify files or never use shell.
Why it attracts. it is easy and seems clear.
Why it fails. it is behavioral guidance, not enforcement, and the tested material repeatedly shows the model can violate advice under confusion or injection.
When it would be right. as a supplement, never as the sole control.
Proposal. add a PostToolUse hook that logs violations for audit.
Why it attracts. it detects misuse.
Why it fails. logging is after the damage and does not prevent modification.
When it would be right. as defense in depth alongside hard tool restriction, not instead of it.
Proposal. run every subagent in background mode or set its model to a smaller variant to limit risk.
Why it attracts. it sounds like containment.
Why it fails. model choice and background mode do not change tool authorization scope.
When it would be right. never for capability restriction.
Deterministic blocking uses PreToolUse hooks exiting with code 2 and permissions.deny - not prompt prose
Two client-enforced controls block disallowed operations before they execute, regardless of what the model decides. A PreToolUse hook inspects the pending tool name and input before execution and blocks by exiting with a specific code. A permissions.deny rule in settings pattern-matches the tool or command and blocks declaratively. Both run at the client layer, between tool request and execution.
deny overrides allow and even bypassPermissions in the documented hierarchy, and multiple hook handlers run in parallel and merge by most restrictive wins.
# Flag: exact hook contract inferred, verify exit codes in official hook docs
# PreToolUse hook that blocks a file write
if [[ "$CLAUDE_TOOL_NAME" == "Write" && "$CLAUDE_TOOL_ARG_FILE_PATH" == *".env.production"* ]]; then
echo "Modification of .env.production is blocked by policy" >&2
exit 2 # Flag: evidence says exit 2 is hard block, exit 1 is non-blocking error
fi// Flag: exact settings key names illustrative, verify in official settings docs
{
"permissions": {
"deny": ["Write(.env.production)", "Bash(git push --force*)", "Read(.env)", "Read(.env.*)"]
}
}Advisory guidance can be overridden by user messages or injected content. Client controls cannot. When a policy must never be violated, such as never force-pushing, never deleting production configs, never running rm -rf, enforcement must live in a layer the model cannot talk past. the tested material repeatedly contrasts detective versus preventive controls: PreToolUse prevents, PostToolUse logging only detects after completion, and CLAUDE.md prose only influences.
Boundary. Preventive controls hold when the requirement is absolute or the agent operates unattended such as CI. Guidance holds when the requirement is preferential. For a production database agent that should only run read-only SELECT, a PreToolUse hook validating each Bash command is preventive and required, because a model that occasionally attempts UPDATE reveals that prompt wording alone is insufficient. For a workstation where the team merely prefers a certain style, a CLAUDE.md rule may suffice.
Recurring specifics. - Hook contract: exit 2 hard-blocks and surfaces stderr to the model, exit 0 means no decision and permission flow continues, other exits are errors but still allow. - permissions.deny pattern examples: Write(.env.production), Bash(git push --force), Read(.env), Read(.env.). - Destructive Bash operations warrant extra caution even when Bash is granted for general purpose such as tests and exploration. Confirmation flows or hooks requiring approval for destructive patterns are the mitigation. - Deny rules match command patterns and are only as strong as their coverage: long flags versus short flags and reordered arguments may not match a narrow pattern, so hooks add richer parsing logic. - Practical caution shown: exit 1 does not block, it is a non-blocking error that still allows the tool, so a policy hook must use exit 2.
Proposal. add a stronger system prompt directive forbidding writes or destructive commands.
Why it attracts. it is quick and feels emphatic.
Why it fails. model compliance is probabilistic and the tested material shows testing with such prompts still producing violations.
When it would be right. as a supplement, not as the enforcement mechanism.
Proposal. add a PostToolUse hook that logs every Bash command to an audit log.
Why it attracts. it creates accountability.
Why it fails. it observes after execution and cannot prevent data modification.
When it would be right. for detective audit, not preventive enforcement.
Proposal. rely on OS-level read-only directory permissions alone.
Why it attracts. it is a strong filesystem guarantee.
Why it fails. it is coarse. It may break legitimate git operations and other CI steps, and it does not cover command-specific policies such as allowing git log while blocking git push --force. Fine-grained tool-level scoping is more precise.
Tool description quality drives selection between built-in and MCP or custom search tools
When a custom or MCP tool competes with a built-in such as Grep, Glob, or Read, the model selects using the description text it has. A sparse description like "Searches code" or "Searches the codebase" loses to a rich built-in description the model knows well from training. A rich description that states capability, output format, example queries, and when to prefer itself over the built-in shifts selection reliably. Prompt lines saying "prefer X" help only slightly.
A second pattern is name overlap. Two tools named analyze_content and analyze_document both starting with "Analyzes content..." are indistinguishable in their opening phrase, so the coordinator misroutes even though one is for web results and one is for uploaded documents. Front-loading the distinguishing phrase such as "extract_web_results" fixes routing.
# Weak - loses to built-in Grep
description: "Searches code."
# Strong - wins for reference-tracing tasks
description: |
Resolves a symbol to its definition and returns all callers including
those reached through re-exported wrappers, with file and line ranges.
Use this instead of Grep for caller, definition, or reference queries.
Grep returns only literal text matches. Returns ranked hits with
call-graph context from a semantic index.Description is the model's primary selection mechanism. Order in config, priority ranking, and alphabetical naming have no bearing. Without distinguishing text the model defaults to the familiar built-in because its behavior is better modeled. Enriched description provides the missing evidence for the right choice.
Boundary. Enriching description holds when a custom tool is more capable for a specific task such as semantic search, AST-aware resolution, or structured column-type returns. The opposite becomes correct when the task genuinely favors the built-in, such as trivial literal-string lookup where text matching is the intent. There Grep should be preferred and the description should say so, explicitly stating when literal matching is appropriate.
Recurring specifics. - Sparse descriptions flagged: "Searches code.", "Searches the codebase.", "Searches the knowledge base.", "Extracts tables.", "Analyzes content and returns insights.", "Analyzes content and extracts key information." - Fixes stated: "Enhance the MCP tool's description to explain its distinct capabilities and output in detail, differentiating it from Grep" and specifying when to choose it over plain text search. - Evidence notes that expanding prompt with routing keywords plus few-shot examples reduced Grep usage from around sixty four percent to around forty nine percent but did not stabilize, while rewriting the description is the most direct and durable fix. - For overlapping names: rename analyze_content to extract_web_results with "processes and returns information retrieved from web search and URLs" so the web versus document distinction is front-loaded.
Proposal. remove Grep from allowedTools so the model is forced to use the custom tool.
Why it attracts. it guarantees routing.
Why it fails. it removes a legitimately useful built-in for trivial lookups and is blunt compared to fixing the description.
When it would be right. only as a narrow guardrail for a specific subagent scope where Grep truly has no role.
Proposal. add a system prompt line always prefer MCP tools.
Why it attracts. it is a short instruction.
Why it fails. it is brittle, does not scale, and the tested material shows it only partly reduces fallback.
When it would be right. as a hedge, but not as the primary fix.
Proposal. add a routing classifier before the coordinator or intercept with a PreToolUse hook that blocks Grep for caller queries.
Why it attracts. it centralizes policy.
Why it fails. it papers over the root cause at the wrong layer and creates coupling or hard blocks where nuanced description would route correctly.
When it would be right. never as the first fix.
Too many simultaneously available tools degrades selection reliability
Every available tool adds decision complexity. As total distinct tools grows, selection reliability declines. Research pipelines with around eighteen tools combining built-in plus MCP show frequent confusion such as picking Grep when Glob is correct and vice versa. Specialist subagents with narrow tool sets restore reliability, and the coordinator retains broader access.
the tested material recommends a soft target of around four to five tools per specialist in one analysis, with cross-role tools added sparingly for high-frequency needs.
// Anti-pattern - synthesis agent with full system toolkit
const synthesisAgentTools = ["web_search", "doc_fetch", "pdf_parse", "citation_lookup", "translate", "sentiment_analysis", "summarize", "keyword_extract", /* ...16 total */]
// Correct - scoped to synthesis role with one narrow cross-role need
const scopedSynthesisTools = ["compile_report", "format_citation", "verify_fact"] // Flag: illustrative tool names, verify against your MCP catalogChoice among many similar tools increases the chance of picking the wrong near-neighbor. A synthesis agent given the full search toolkit plus analysis toolkit is tempted to perform searches mid-synthesis instead of using provided findings, producing duplicate, unvetted results. Scoping reduces the decision space so the agent focuses on its actual job, and per-agent context isolation further sharpens that focus.
Boundary. Limiting holds when tools span multiple specializations. The opposite becomes correct when additional tools genuinely serve the same specialization, but even there the practical ceiling remains low. the tested material shows that adding a narrow verify_fact tool for simple lookups while still routing complex verification through the coordinator is the correct way to add capability without over-granting the full search toolkit.
Recurring specifics. - Counts: around eighteen tools causing degraded reliability, nine tools causing misuse of summarise and sentiment_analysis by a web search agent, sixteen tools sobering synthesis, four to five as the recommended max per agent in one study. - Single Agent with many types is described as losing specialist-context focus, combining tools avoids orchestration overhead but sacrifices quality. - Coordinator with two specialist subagents, one for web search and one for internal documents, with the coordinator synthesizing results, is the preferred pattern for multi-source tasks.
Proposal. keep all tools available in every agent for flexibility, just in case.
Why it attracts. it avoids up-front scoping decisions.
Why it fails. flexibility is purchased with reliability loss and specialization breakdown.
When it would be right. never as the default. Scoped plus narrow cross-role is the pattern.
Proposal. log which tools each agent calls and review later.
Why it attracts. it creates visibility.
Why it fails. it is detective after the fact and does not prevent duplicate web searches mid-synthesis.
When it would be right. as observability, not as the fix for over-exposure.
Proposal. merge the web search and synthesis agents into one to eliminate handoffs.
Why it attracts. fewer handoffs.
Why it fails. it combines two roles with different tool needs, preserving the over-exposure or creating a new one.
When it would be right. never for this problem.
Server-side tools execute on provider infrastructure versus client tools executed by the application
In the API tool-use loop, two categories exist. Client tools are defined by the application with a JSON schema such as input_schema. The model emits a tool_use block, the application executes the tool, and the next request must include a tool_result block keyed by tool_use_id to continue. Server-side tools such as web search, web fetch, and code execution are executed by the provider during the turn and normally return results inside the API response without requiring the application to supply a tool_result for that server tool call.
The loop checks stop_reason. When it is tool_use, the application executes requested client tools and appends results before sending the next request. Server tools do not follow that appended-result path in the same way.
// Flag: field names illustrative, verify against official API docs
// Client tool definition
{
"name": "get_customer",
"description": "Fetch a customer record by id",
"input_schema": { "type": "object", "properties": { "id": { "type": "string" } } }
}
// Turn progression depends on category
// Client tool: application must return tool_result with tool_use_id before the next model turn
// Server tool: provider executes during the turn, results appear in the API responseExecution location determines ownership. The application owns security, runtime, permissions, error shape, and protocol continuation for client tools. The provider owns those for server tools. Mixing the two without distinction leads to missing continuations, misinterpreted protocol success versus business success, and mishandled authorization boundaries.
Boundary. Client versus server holds whenever a workflow mixes both in one session. Modern financial and operational tasks in the tested material deliberately combine web search, web fetch, and code execution with custom client tools to test whether the implementer tracks who executes what. The opposite becomes correct only in a pure client-only or pure server-only workflow where no mixing occurs, but even then the handler, permissions, and continuation pattern for the chosen category must still be implemented explicitly rather than assumed.
Recurring specifics. - Phrasing in every server versus client item: the correct option "chooses the execution type deliberately and implements the handler, permissions, and continuation only where the tool category requires them." - Distractors: assume every tool runs inside the model, return a tool_result for every server-tool call even when no client tool is mixed in, or assume a client-schema tool is executed automatically by the provider. - Correct handling: distinguish schema validity from business authorization, preserve provenance and correlation, and distinguish retryable protocol failure from invalid or unauthorized business request.
Proposal. assume every tool runs inside the model.
Why it attracts. it treats the API as a black box.
Why it fails. the model only emits requests. Execution happens in application code or provider infrastructure, and the boundary determines error and permission behavior.
When it would be right. never.
Proposal. return a client tool_result for every server-tool call.
Why it attracts. it looks uniform.
Why it fails. server tools normally complete within the API request and have different handling mechanics.
When it would be right. only when a client tool is mixed into the same turn should its result follow the client continuation pattern.
Proposal. assume the provider automatically executes client-schema tools.
Why it attracts. the schema is provider-defined in some cases.
Why it fails. schema definition and execution responsibility are distinct. The application remains responsible for executing client tools.
When it would be right. never.
Distinctions that decide answers
| This | Not this | How to tell them apart |
|---|---|---|
| Grep | Glob | Grep searches what is inside files. Glob matches file names by path pattern. The question wording tells you which you need, inside versus named star star slash star dot test dot tsx. |
| Edit with unique anchor | Read plus Write for a one line fix | Edit touches exactly the identified string with minimal context. Read plus Write replaces the whole file and is a last resort because it costs far more tokens. |
| Widen anchor with surrounding context or replace_all true | Jump straight to Read plus Write on non unique match | The widened anchor or replace_all keeps the fix on Edit at nearly zero extra cost. Jumping to Read plus Write escalates immediately to the heavyweight fallback. |
| Incremental Grep then Read then Grep | Read all files upfront | Incremental traversal spends tokens only on files that prior Grep hits justified. Bulk reading upfront is the most expensive discovery failure in the built ins. |
| Bash find plus xargs grep as combined step | Grep for callers then Glob for sibling tests | Bash can surface a similar result set, but the exam's expected trace separates concerns by tool purpose and preserves the Grep then Glob then Grep shape for wrapper tracing. |
Traps
The tempting answer. Run Glob with a pattern such as star star slash star processLegacyOrder star because Glob accepts patterns that look like names.
Why it fails. Glob can only match star star slash star processLegacyOrder star against path names, not inside OrderProcessor dot ts where the call lives. Content search is always Grep and Glob returns nothing for this task.
What is correct. Use Grep with the literal function name and an appropriate path scope to retrieve call sites with file and line information.
The tempting answer. Search file contents for the literal test substring to find star star slash star dot test dot tsx files because Grep will technically surface the substring.
Why it fails. Grep surfaces test as a content hit not a path match, which is slower and less precise, while Glob is purpose built for extension and directory matching and matches the suffix directly without opening contents.
What is correct. Use Glob with star star slash star dot test dot tsx or star star slash config dot star for path based discovery by suffix.
The tempting answer. Load every file into context before any search because reading feels like diligence and avoids missing a hidden consumer.
Why it fails. A two hundred file upfront read is primarily a context budget burn that crowds out synthesis and validation, where incremental Grep for entry points then targeted Read for imports is the tested pattern that spends proportionally to relevance.
What is correct. Grep first for entry points, Read only the hit files, then Grep for wrapper names and Read only what that step justifies.
The tempting answer. Use Read plus Write for every file change because that path never raises a non unique anchor error.
Why it fails. It spends a file's worth of tokens on every line change. The exam marks the Edit first, widen then Write last ordering as the expected workflow, with Write as a last resort.
What is correct. Attempt Edit with the shortest plausible unique anchor, widen until unique or set replace_all where a global change was intended, and only then escalate to Read plus Write.
The tempting answer. Treat the non unique match error as proof that Edit cannot handle the file and escalate immediately to the heavyweight fallback.
Why it fails. The widened anchor path succeeded in the worked trace with only a few extra lines of surrounding signature and comment, while the fallback costs the entire file on every use, so escalation skips the cheaper correct recovery.
What is correct. Supply surrounding context that makes the anchor unique or select replace_all true for the global case, keeping the fix on Edit before considering Read plus Write.