Selecting an agent harness: tool loading behavior
Compare config-time tool composition with runtime tool loading so a worker agent sees the right tools for the task.
Tool catalogs grow after the first useful worker ships.
A tax analyst starts with document retrieval, then adds filing rules, customer account data, policy checks, and escalation paths. A marketing worker starts with brand guidance, then adds research sources, publishing tools, and approval hooks. A code-review worker may use repo context, shell, test runners, and review policies every few minutes.
If the harness sends every tool schema on every turn, request size grows with the catalog rather than the task. If the harness hides too much, the agent may spend turns discovering or requesting a tool that should already be available.
Part 3 asks when the model sees each tool. The answer matters because production workers rarely use every available tool on every turn.
In this post
Tool count is only half the measurement. The other half is loading behavior.
If a worker uses a small, stable tool set, configure the harness once and keep the request predictable.
If a worker has a large catalog with a long tail of rarely used tools, runtime loading can keep idle tools out of the model request.
Tool-loading mechanisms
There are two practical mechanisms.
| Mechanism | How it works | Best fit | Main tradeoff |
|---|---|---|---|
| Config-time composition | Decide the worker's tool set before the session starts. The harness sends only those tools. | Workers with a small and known tool set. | Simple and predictable, but occasional tools either stay loaded or stay unavailable. |
| Runtime tool loading | Keep a larger catalog available, but send only likely tools for the current turn. Load more when the task calls for them. | Workers with many integrations where most tools are idle on most turns. | Reduces context for large catalogs, but discovery can add latency or an extra step. |
Config-time composition is the first lever to reach for. It is easier to reason about, easier to test, and enough for many workers. If a worker only needs a knowledge-search tool, a ticket tool, and a narrow write action, do not make runtime loading the first design.
Runtime loading is for a different shape of worker: the catalog is large, the working set changes by task, and most tools are idle most of the time.
Harness behavior
The five harnesses split into three groups.
| Harness | Tool-loading posture | Practical reading |
|---|---|---|
| Goose | Runtime routing is available. | Useful when the tool catalog is large and only a small subset is relevant to a turn. |
| Claude through API or SDK | Deferred tool loading is available. | Useful for large catalogs, especially when tools are described but not always needed. |
| Claude Code | Coding-oriented default set. | Good when the worker is expected to use the coding toolkit frequently. |
| OpenCode | Agent profiles can reduce built-ins. | Strong for config-time composition when the target tool set is known. |
| Codex CLI and DeepAgents | Fixed coding-oriented profiles in this comparison. | Good for repo-heavy workflows; less flexible for domain workers that need only a few tools. |
The important point is not that runtime loading is always better. It is better only when it matches the shape of tool use.
Workload test
Before you choose, measure tool use over real tasks. Even a small sample is useful.
| Question | If the answer is yes | What it suggests |
|---|---|---|
| Does the worker use the same tools on most turns? | The working set is stable. | Prefer config-time composition. |
| Are there many tools that are needed only in rare branches? | The catalog has a long tail. | Prefer runtime loading. |
| Is latency more important than context size? | Tool discovery may hurt the experience. | Keep the tool set small and explicit. |
| Does each customer or workspace mount different integrations? | The catalog changes by environment. | Runtime loading or per-agent profiles become more valuable. |
The measurement does not have to be complicated. Log the tools offered to the model, the tools actually called, and the turns where the agent had to ask for a capability that was not loaded. That gives you the shape of the catalog.
Implications for selection
For most production workers, start with config-time composition. Remove unused defaults, add only the tools the job needs, and keep the request stable.
Reach for runtime loading when the agent has a broad integration surface:
- customer-specific tool catalogs
- internal operations agents with many systems
- research agents with many sources
- product agents that need different tools in different workflow branches
Part 1 measured how expensive it can be to carry tools by default. Part 3 adds the operating rule: do not optimize for "more tools available." Optimize for "the right tools visible on this turn."