Don't Make Your Agent Guess

Why Tool Design Is Crucial for Agent Harness Design

Say you are building an agent-based system and want an agent to help write a follow-up email. You tell it: “Draft the email, but do not send it.” Then you give it access to Gmail, including a send_email function.

What could possibly go wrong?

Most teams start by improving the prompt. Add “never send without approval.” Add “ask first.” Add “be careful with external communication.” That is natural, because prompts are easy to change and they are how we usually iterate on agents.

But if the agent has the tool, you need to assume it will eventually use it.

The problem is not that the instruction was too weak. The problem is that the architecture made a dangerous action available and then hoped language would prevent it. That is the hidden design challenge in agent safety: reliability is not just about what you tell the agent. It is about what the agent is actually able to do.

Prompts Are Not Permission Systems

After building a couple of agents and discussing harness design with teams, we keep seeing the same pattern: teams over-invest in prompts and under-invest in tool design.

We get it. Prompts are the fastest feedback loop. You try a workflow, see where the agent fails, then improve the prompt, add a skill, or tweak the instructions. For many problems, that is exactly the right move. But proper agent harness design needs to consider more than instructions. The tools themselves have to encode what the agent is allowed to do.

And this will matter more as we keep pushing agents to become more autonomous, more long-running, and more deeply connected to the systems our users rely on every day: email, repositories, CRMs, databases, ticketing systems, cloud accounts, and internal APIs. As that happens, the cost of a bad tool call goes up. A wrong answer in a chat is annoying. A wrong action in a connected system can send the email, modify the ticket, expose the secret, or delete the data.

The model does not need to be malicious for this to happen. It only needs to be confused, overconfident, or operating with the wrong context. If confusion can turn directly into a production action, the architecture has already failed.

So the real question for developers is not only “How do I write better instructions?” It is “What authority am I giving this agent, and where is that authority enforced?”

The right tool is not an implementation detail. It is the boundary of the agent’s authority.

Zoom

Broad Tools Turn Small Mistakes Into Big Incidents

A lot of agent setups are built around broad tools and restrictive prompts. You give the agent Gmail access and say, “Do not send without approval.” You give it CRM access and say, “Only look at relevant customers.” You give it filesystem access and say, “Do not touch production files.” You give it cloud credentials and say, “Only use staging.” You give it database access and say, “Do not delete anything.” This can work in demos. It can work in happy-path tests. It can even work most of the time in production. But “most of the time” is not the safety bar for tools that can create real side effects.

Instructions are soft guardrails. They depend on the model remembering them, interpreting them correctly, prioritizing them over other context, and applying them in edge cases. That is a lot to ask from a system that is constantly being fed changing context: tool results, user messages, logs, tickets, code, and other information.

The failure mode is not made up. In one widely discussed PaaS incident, an AI coding agent had access to infrastructure credentials powerful enough to modify production systems. While trying to solve what appeared to be a staging issue, it found an API token, used it, and executed a destructive call that deleted a production database along with its backups. The important lesson is not “the model was evil.” It was trying to help. The lesson is that the environment allowed a confused agent to convert an ordinary mistake into a serious incident. No amount of “be careful with production” is as strong as credentials that cannot touch production.

Hard Restrictions Beat Soft Guardrails

The useful distinction is soft guardrails versus hard restrictions.

Anything that gets converted into a user message or model context should be treated as a soft guardrail: instructions from AGENTS.md, SKILL.md, system prompts, guidelines, or any other prompt text. These describe what the model should do.

Hard restrictions are different. Tool availability, scoped credentials, permission checks, validation, and approval flows define how the model can interact with the outside world.

So instead of relying on this soft guardrail:

Tool: gmail.send(to, subject, body)
Instruction: “Only send after approval.”

Enforce it through the harness:

Tool: gmail.create_draft(to, subject, body)
if (user.hasApproved())
    Tool: gmail.send_approved(draft_id)

In the first design, the agent can send the email and you are relying on the prompt to stop it. In the second design, the agent can draft the email, but the harness owns the approval step. Without approval, the send action is not available.

The same pattern applies to other systems. Instead of giving an agent crm.query(sql), expose crm.get_open_opportunities(customer_id) or crm.get_recent_customer_activity(customer_id) for customers the current user or thread has access to.

Instead of slack.post(channel, text), expose slack.create_post_draft() and only call slack.approve_post() after clear approval, ideally with tracing information about who approved it and why. Instead of full database access, expose a task-specific read model.

The goal is not to make every tool tiny for the sake of it. The goal is to make the tool match the job. A broad tool exposes what the underlying system can do. A good agent tool exposes what the agent is allowed to do for this workflow.

The Right Tools for the Job

The counterintuitive part is that narrower tools often make agents more capable, not less.

Bad tool design forces the agent to guess. Which system should I use? Which data is relevant? Which action is allowed? Does this require approval? Is this production or staging? Is this side effect acceptable? Every one of those guesses is a chance for failure.

Purpose-built tools reduce the number of decisions the model has to infer from context. They make the right action obvious and the dangerous action unavailable. They also give the model better semantic guidance.

A tool called get_open_opportunities(customer_id) communicates much more intent than query(sql). A tool called create_email_draft tells the agent the safe next step. A tool called request_human_approval tells it exactly where the workflow should pause.

The interface itself becomes part of the instruction.

This is especially important when agents work over internal company data. “Connect the agent to Gmail” is usually too broad. “Use the approved internal-email dataset” is safer and clearer. “Connect the agent to the CRM” is too vague. “Expose recent customer communication, open opportunities, and allowed next actions” gives the agent the context it needs without handing it the entire system.

The sandbox, the dataset, or the narrow tool becomes the unit of access.

That is the shift: do not ask the agent to infer policy from a prompt if you can encode that policy into the tool contract.

A Practical Checklist for Agent Tool Design

Before you connect an agent to a system, ask these questions:

• What exact job is this tool for?
• Does the agent need read access, write access, or action access?
• Can the workflow be split into draft, preview, approval, and execute steps?
• What is the smallest useful input schema?
• What should this tool never expose?

And then ask production:

• Can this affect production systems?
• Can it delete data or backups?
• Are credentials scoped to the task and environment?
• If the model is confused, what is the safe default?

That last question is the one that usually reveals the design problem.

If the model is confused and the worst plausible outcome is “it asks for clarification,” you are in a good place. If the worst plausible outcome is “it deletes the database,” “sends the customer email,” or “modifies production,” the tool is too powerful for the workflow.

The Tool Is the Boundary

Agents are not made reliable by telling them to be careful. They are made reliable by giving them environments where the correct action is easy and the dangerous action is impossible. Prompts still matter. Skills still matter. Human approval still matters. But none of them replace tool design.

The future of agents is not just better models. It is better harnesses: clearer tools, narrower permissions, safer defaults, and fewer guesses.

So the next time you connect an agent to a system, ask one question: does this tool expose only what the agent needs to do its job, or everything the system can do? The answer will tell you whether you are building a reliable agent or a production incident waiting to happen.