AI Agent Guardrails vs Hard Constraints: What a Filter Cannot Do

AI agent guardrails are filters. They look at what goes into the model and what comes out, and they try to catch bad content.

Filters can be wrong. A hard constraint is a limit the system enforces, whatever the model says or does.

Most vendor pages sell you the first and call it security. For anything that can move money, change records, or send messages, you need the second.

Two Kinds of Control

Here is the split I use.

A guardrail checks content. It reads text and decides if the text looks safe. A classifier, a rule list, or a second model does the check. The answer is a guess, even when it is a good guess.

A constraint limits capability. The agent cannot do the thing, because the system does not allow it. The database login has no delete right. The tool has no “send to external address” option. A human must click approve before the refund goes out.

The difference shows up when something goes wrong. A guardrail fails open or closed depending on how well it recognizes the attack. A constraint holds even if the model has been fully fooled.

The field has a name for the failure of the second kind. OWASP calls it Excessive Agency and lists three causes: too much functionality, too many permissions, and too much autonomy. None of those are fixed by a better filter. They are fixed by taking things away.

A short glossary entry on AI guardrails covers the basic definition. This post is about where the definition stops being enough.

Input and Output Guardrails

Guardrails are still useful. Just know what they do.

Input guardrails run on what the user sends. They block off-topic requests, strip or mask personal data, and flag known attack patterns. The OpenAI Agents SDK describes the basic design: a check runs on the input, and if it trips a “tripwire,” the run stops.

Output guardrails run on what the agent sends back. They catch personal data leaks, toxic text, banned topics, and answers that are not backed by the source documents. Amazon Bedrock Guardrails lists content filters, denied topics, word filters, sensitive information filters, and grounding checks. The docs say the sensitive information filter works by “probabilistic detection.” That word matters. Probabilistic means it can miss.

Tool-call checks sit between the two. NVIDIA NeMo Guardrails calls these execution rails: checks on the input and output of tools the model calls. This is the point where a guardrail starts to look like a constraint. If the check is plain code (is this amount under $500, is this account on the allow list), it is a constraint. If the check is a model asking “does this look okay,” it is still a guardrail.

Good uses for guardrails:

• Keep a support bot on topic.
• Mask card numbers and national IDs before they reach a log.
• Catch obvious abuse and obvious policy breaks.
• Flag low-confidence answers for review.
• Give you a signal. A rising count of tripped checks tells you something is off.

Bad use for guardrails: being the only thing between an attacker and an irreversible action.

Why a Filter Cannot Be the Last Line

Three reasons, all from published work.

First, nobody has a proof that filters stop prompt injection. OWASP’s prompt injection entry says it is “unclear if there are fool-proof methods of prevention.”

Second, filters are sold on weak numbers. Security researcher Simon Willison wrote in June 2025 that guardrail products “almost always carry confident claims that they capture ‘95% of attacks’ or similar,” and that in web security “95% is very much a failing grade.” I agree. If an attacker gets unlimited tries, 5% is a door.

Third, attackers adapt. A paper from researchers at Anthropic, Google DeepMind, OpenAI, and universities, The Attacker Moves Second, tested 12 published defenses against attackers who tuned their attacks to each defense. They report attack success above 90% for most, and note that most of those defenses had first reported near-zero success rates. Static test sets flatter the filter.

None of this says to drop filters. It says to plan as if a filter will sometimes fail, and ask what happens next.

Hard Constraints in the Tool Layer

Constraints live outside the model. The model can ask. The system decides.

The research group behind Design Patterns for Securing LLM Agents against Prompt Injections states the rule well: once an agent has ingested untrusted input, it must be constrained so that input cannot trigger any consequential action. Note the word “constrained,” not “filtered.”

Here are the constraints I look for in an agent design:

1. Least-privilege credentials. The agent connects with its own identity, with only the rights the task needs. A read-only job gets a read-only login. See AI agent permissions and least privilege.
2. Narrow tools. One tool, one job. “Look up order status” is a tool. “Run any SQL” is a hazard.
3. Parameter limits in code. Refund cap, daily send cap, allowed recipient domains, allowed record types. Plain code that rejects the call.
4. Approval gates. A person approves actions that cannot be undone. See human-in-the-loop AI agents for how to design the gate so people do not just click through.
5. Network and data boundaries. The agent can only reach the hosts it needs.
6. Rate limits and spend limits. A stuck loop hits a ceiling, not a bill.
7. Audit logging. Every tool call is recorded outside the model’s control. See what an AI audit trail is.

Meta’s Agents Rule of Two (October 2025) gives a clean way to pick which to apply. An agent can have three properties: it processes untrusted input, it can reach sensitive systems or private data, and it can change state or communicate outside. Meta’s advice is that an agent should have no more than two in one session. If it needs all three, a human must be in the loop.

That is a constraint, not a filter. You remove a capability so the bad outcome cannot happen.

Controls by Risk Level

Not every agent needs every control. Match the control to what the agent can do.

Risk level	What the agent can do	Guardrails to add	Constraints to require
Low	Answer questions from public or internal docs. No tools.	Topic filter, output check for personal data	Read-only data access, logging
Medium	Read private data. Draft messages or records for a person.	Input filter, output PII mask, grounding check	Per-user permissions, narrow tools, human sends or saves
High	Write to systems of record. Send external messages.	Tool-call checks, anomaly flags	Write scopes limited by code, caps, allow lists, approval above a threshold, full audit log
Critical	Move money, change access rights, touch patient or legal records.	Same as high, plus review sampling	Two-person approval, no autonomous execution, kill switch, tested rollback

Two notes on the table.

At the top of the table, guardrails act like a smoke detector. They warn you. The constraints carry the weight.

And the first row is not “no controls.” It is “controls that fit.” A docs bot with a topic filter and read-only access is fine.

The Ignore Test

This is the test I use in reviews. It takes five minutes per control.

For each control on your list, ask: if the model ignores this completely, does the control still hold?

• “The system prompt says never issue refunds over $500.” The model can ignore a prompt. Fails the test.
• “The refund tool rejects any amount over $500 in code.” Holds. Passes.
• “A classifier scans outputs for customer emails.” The classifier can miss. Fails.
• “The agent’s login cannot read the customer email column.” Holds. Passes.
• “The agent is told to ask before deleting.” Fails.
• “The delete tool needs a signed approval token from a person.” Passes.

A control that fails the test is still allowed. It just cannot be the one you count on for a high-risk action. Mark it as a guardrail and make sure a constraint sits behind it.

Run the test in two passes. First on paper, using your design doc. Then live: try to make the agent break each rule using a test account. Plant instructions in a document it reads. Ask it directly. Ask it politely. Do this against staging, never against production data. If the constraint holds, log it as verified. If a guardrail trips, log how it tripped. Keep the log. When the model or the prompts change, run it again.

For a fuller treatment of how the injection side works, see prompt injection in AI agents.

Cost and Latency of Guardrails

Do guardrails slow the agent? Some do. It depends on how you run them.

A guardrail that is a regex or a rule list adds almost nothing. A guardrail that calls a second model adds a model call. I have no universal number for you, because it depends on the model, the size of the text, and your network. Measure it in your own stack.

The tradeoff is the same one the OpenAI Agents SDK docs describe. You can run an input guardrail in parallel with the agent. That gives the best latency. But if the guardrail trips, the agent may have already used tokens and run tools. Or you can run it in blocking mode, so the agent never starts if the check fails. That costs time but prevents wasted work and early side effects.

My recommendation:

• Use blocking mode when the agent has tools that change something.
• Use parallel mode for chat-only agents where the worst case is wasted tokens.
• Run output checks only on the final answer to the user, and tool-call checks on every tool call that matters.
• Use a separate, small model for screening. Anthropic’s guidance on building agents says that having one model instance handle the request while another screens it tends to work better than asking one call to do both.

Constraints in code cost almost nothing at runtime. That is another reason to put the heavy lifting there.

Approval gates do add delay, since a person is involved. That is the point. Put them only where the action is costly or hard to undo.

Review Checklist

Use this before launch. Answer each with yes or no.

Capability

• Each tool does one narrow job.
• The agent has its own credentials, with the least rights needed.
• Write and send actions are limited by code (caps, allow lists, scopes).
• Irreversible actions need a person’s approval.

Filters

• Input checks exist for the main abuse cases.
• Output checks mask personal data and catch ungrounded answers.
• Tripped checks are logged and counted.

Proof

• Every control passed the ignore test, or is marked as a guardrail with a constraint behind it.
• Someone tried to break each rule in staging.
• Every tool call is logged outside the model.
• There is a kill switch and an owner who can use it.
• The tests re-run when the model, prompt, or tools change.

For the wider list of security items, use the enterprise AI security checklist. For who owns these decisions, see AI governance in practice.

Bottom Line

Guardrails advise. Constraints enforce.

Use filters to catch the common and the obvious. Use constraints for anything you cannot afford to get wrong.

Next step: list every action your agent can take. For each one, run the ignore test. Wherever it fails, add a limit in code or an approval gate before launch.

FAQ

What are AI guardrails? AI guardrails are checks that run on the input to a model, the output from it, or the tools it calls. They block or flag content that looks unsafe or off-topic. They are usually rules, classifiers, or a second model, so they work by judgment and can be wrong.

What guardrails does an agent need? At minimum: a topic filter, masking for personal data, a check on answers that must be grounded in source documents, and logging of every tripped check. Add tool-call checks if the agent can write or send. Then add hard constraints (least-privilege access, caps, approvals) for any action that is costly or hard to undo.

Guardrail or permission: which one? Permission, whenever the risk is an action. A permission is enforced by the system and holds if the model is fooled. A guardrail is a filter that may miss. Use guardrails for content problems like tone, topic, and data leaks. Use permissions for what the agent is allowed to do.

Do guardrails slow the agent? Rule-based checks add very little. Checks that call another model add a model call. You can run input checks in parallel to save time, but then the agent may act before the check finishes. Measure in your own setup and choose blocking mode when the agent can change things.

How do I test them? Use the ignore test: if the model ignores the control, does it still hold? Then try to break each rule in staging with direct requests and instructions hidden in documents. Log the results. Re-run the tests whenever the model, prompt, or tools change. Attackers adapt, so a one-time test is not enough.

Three Ways to Work With TMA

Need an agent built? We deploy production AI agents in your infrastructure. Working pilot. Real data. Measurable ROI. → Schedule Demo

Want to co-build a product? We’re not a dev agency. We’re co-builders. Shared cost. Shared upside. → Partner with Us

Want to join the Guild? Ship pilots, earn bounties, share profit. Community + equity + path to exit. → Become an AI Architect