What Is AI Guardrails? | TMA Glossary

AI guardrails are controls around an AI model or agent. They check what goes in, what comes out, and what the agent tries to do. If a check fails, the system blocks, rewrites, or escalates.

Think of them as the rules of the road for an agent. The model decides where to go. The guardrails stop it from driving off a cliff.

Two kinds of guardrails

I split guardrails into two groups because they fail in different ways.

Probabilistic guardrails use a model to judge another model. A classifier flags toxic text, personal data, or an attack. They catch a lot. They also miss things and sometimes block good requests.

Deterministic guardrails are plain code and configuration. A tool allow list. A dollar limit. A rule that email can only go to approved domains. They do not get tricked by clever wording.

The UK National Cyber Security Centre makes the same point about prompt injection. Protections should focus on deterministic, non-LLM safeguards that constrain what the system can do. It also warns that deny-listing known bad phrases is unreliable. See the NCSC’s December 2025 post.

Where guardrails sit

Input. Screen user messages and retrieved content for attacks, personal data, or off-topic requests.
Output. Check answers for policy violations, leaked data, or the wrong format.
Action. Check each tool call before it runs. This is the most important layer for agents.
Flow. Keep the conversation on an approved path.

Real tools and research

NVIDIA’s NeMo Guardrails paper (October 2023) describes an open-source toolkit for adding programmable rails to LLM applications. The rails are user-defined, independent of the underlying model, and interpretable. Examples include avoiding harmful topics and following a predefined dialogue path.

Meta’s Llama Guard (December 2023) is a model that classifies both prompts and responses against a safety risk taxonomy. You can adjust the taxonomy categories to fit your use case.

At the framework level, the NIST AI Risk Management Framework gives the governance structure around all of this. Version 1.0 came out in January 2023 with four functions: Govern, Map, Measure, and Manage. A Generative AI Profile followed in July 2024.

Guardrails are not the whole defense

Guardrails reduce risk. They do not remove it. The OWASP guidance on prompt injection lists input and output filtering as one mitigation among several. The others include least privilege, validated output formats, human approval for high-risk actions, and adversarial testing.

So I treat guardrails as one layer in a stack:

Narrow permissions, so the agent can only do what it needs.
Deterministic checks on every action.
Model-based filters on inputs and outputs.
Human approval on irreversible steps.
Logs and alerts, so you see what the guardrails caught.

A short example

Say a customer-service agent can issue refunds. A guardrail stack could look like this. An input filter screens for injection attempts. A policy check confirms the order belongs to the logged-in customer. A deterministic rule caps automatic refunds at a fixed amount. Anything above the cap waits for a person. An output check removes account numbers from the reply. Each layer is simple. Together they cover failures that no single layer would.

Common mistakes

One big filter. A single classifier at the front door, nothing at the action layer.
Rules in the prompt only. A system prompt saying “never do X” is a request, not a control.
No measurement. You do not know the false-positive rate, so users get blocked without anyone noticing.
Set and forget. Attacks change. Guardrails need testing on a schedule.
Blocking silently. Logs should record every block so you can tune the rules.

How to test guardrails

A guardrail you have never attacked is a guess. Build a small set of test inputs for each layer: known attacks, borderline requests, normal requests, and requests that should be refused. Track two numbers. How many bad inputs get through? How many good inputs get blocked?

Both matter. A filter that blocks too much trains users to work around the system, and that pushes work into unapproved tools. Review blocked requests on a schedule, tune the rules, and rerun the set after each change. Log every block with the rule that fired so you can explain it later.

Prompt injection is the attack guardrails are most often asked to stop. Human in the loop is the guardrail that uses a person instead of code. For permission design, read AI agent permissions and least privilege and the enterprise AI security checklist.