AI Agent SLA: What to Put in the Contract

An AI agent SLA that only promises uptime protects the vendor, not you. An agent can be 100 percent available and wrong all day.

Write the agreement around five things: an accuracy floor measured on an agreed test set, escalation times, drift review, notice of model changes, and a defined maintenance scope. Then attach remedies that trigger on bad output, not only on downtime.

This is general information, not legal advice. Have your own counsel review anything you sign.

Why Uptime Alone Is Not Enough

Standard cloud SLAs measure availability. Take the AWS Bedrock SLA. It counts the share of requests that avoid server errors in each five-minute interval. Service credits are 10 percent of monthly charges below 99.9 percent uptime, 25 percent below 99.0, and 100 percent below 95.0. It says nothing about whether the model’s answer was correct. That is fine for a model API. It is not enough for an agent that books, approves, or replies on your behalf.

Law firms are saying the same thing. Morgan Lewis (October 2025) notes that uptime and response-time SLAs may miss AI-specific failures, and that behavior can drift while the system stays technically available. Mayer Brown (February 2026) argues for replacing uptime-only metrics with outcome measures such as timeliness and complaint rates, with service credits that trigger on performance failures.

There is also a gap in the contracts people sign today. A TermScout analysis, reported by Stanford Law (March 2025), found that 17 percent of AI vendor contracts warrant that the product complies with its documentation, against 42 percent of SaaS contracts. The article does not give the sample size, so treat the numbers as a signal, not a benchmark. The signal is still clear: do not assume the default paper covers quality.

Researchers see the same hole. The AgentSLA paper (November 2025) says AI agents need formal SLAs and that no agreed quality model exists for them yet. Plan to write your own terms.

Accuracy Floor on an Agreed Eval Set

This is the most important clause. It answers “how do you measure accuracy in an SLA?”

An accuracy floor is a minimum score the agent must hit on a fixed set of test cases. You and the vendor agree on the set before signing. The SLA names the score, the set, and how often it is run.

What to write down:

The workflow. One named workflow per SLA schedule, not “the agent” in general.
The eval set. A versioned list of real cases with correct answers, drawn from your data. Both sides hold a copy. Say who may add cases and how changes are approved.
The metric. Define it in plain words. “Share of invoices matched to the right purchase order” beats “accuracy.” Mayer Brown uses that kind of example.
The floor. A number, plus a measurement window (say, weekly).
Who runs it. Ideally both parties can, and the results are logged.
Handling of ties and disputes. Name who decides if a case is labeled wrong.

Size the eval set so the floor means something. This is my own arithmetic, not a sourced figure. If the agent truly scores 90 percent, a 95 percent margin of error is about 5.9 points on 100 cases, 4.2 on 200, 2.6 on 500, and 1.9 on 1,000. A floor of 90 percent measured on 100 cases will fail some weeks by chance alone. Use a few hundred cases at least, and write the floor with that noise in mind.

The AgentSLA paper makes a related point. Because model output is nondeterministic, it attaches an uncertainty value to each metric. You do not need that formality, but your contract should say how run-to-run variance is handled. For a deeper guide to building the test set, see my AI agent evaluation guide.

An example, clearly hypothetical: “On the agreed 400-case invoice set, the agent matches at least 93 percent of invoices to the correct purchase order, measured weekly. Two consecutive misses trigger the remedies in Schedule C.”

Escalation and Response Time

Two clocks matter here, and agents add a third.

Vendor response to incidents. Define severity tiers with response and fix targets. Morgan Lewis mentions severity tiers, customer suspension rights, and shadow-mode testing before live use as AI-specific tools.
Your right to switch the agent off. The SLA should let you suspend the agent without penalty when quality drops. Put the kill switch in writing.
Agent escalation to a person. The agent should hand off to a human within a stated time and under stated conditions: low confidence, a flagged topic, a customer request. Write the rule and measure it. My piece on escalation design covers how.

A simple severity table to start from:

Severity	Example	Vendor acknowledges	Vendor mitigates
1	Agent takes wrong actions in production	30 minutes	4 hours, or you may suspend
2	Accuracy below floor, no harm yet	4 business hours	2 business days
3	Minor defects, workaround exists	1 business day	Next scheduled release

The times above are examples to negotiate from. Pick numbers your own team can actually respond to.

Drift Review

Drift means the agent’s quality changes without anyone editing it. Inputs change. Documents change. A model update shifts answers. The AgentSLA paper proposes a drift metric that compares aggregated metrics across consecutive time windows. Morgan Lewis says contracts may require the vendor to step in, pause, or retrain even when the system is up.

Write a drift clause with four parts:

Cadence. A scheduled review, for example monthly, plus an automatic review when the accuracy floor is missed.
Evidence. The vendor shares eval results and a sample of failures, not a summary.
Refresh. Both sides add new real cases to the eval set at each review, so the test does not go stale.
Action. What the vendor must do, and by when, if scores fall.

The question “who pays to fix drift?” is the one that causes fights. My answer: put it in the contract before launch. A fair split is that the vendor pays when drift comes from its changes (a model swap, a prompt edit, a tool update). You pay, or the work is billed under the retainer, when drift comes from a change on your side, such as a new document format or a new product line. If cause is unclear, agree on a joint diagnosis within a fixed number of days.

Model Change Notice

Agents sit on top of models that other companies retire and replace. Your vendor may not control that schedule.

Look at what providers actually promise. Anthropic states that it gives at least 60 days’ notice before retiring publicly released models, and warns that deprecated models are likely to be less reliable. For example, it notified developers on April 14, 2026 that Claude Sonnet 4 and Claude Opus 4 would be retired on June 15, 2026. OpenAI’s deprecations page describes its own notice windows, which differ by model type. Read each provider’s current terms. They change.

So a provider’s notice is the minimum you can expect. Your vendor needs to pass it on and do the testing. Ask for:

Notice to you of any change to the underlying model, prompts, tools, or data sources that could change behavior. Pick a window (say, 30 days for planned changes). Emergency changes get notice as soon as possible after the fact.
A test of the new setup on your eval set before it goes live, with results shared.
Shadow mode or a staged rollout for material changes.
A right to refuse or delay a change that drops scores below the floor.
A rollback plan.

Also ask which model versions are pinned. Unpinned “latest” aliases can change behavior without notice.

Maintenance Scope

“What belongs in a maintenance retainer?” Most disputes start here, because “maintenance” means different things to each side. Spell it out. For a longer look, read AI agent maintenance after deploy.

Usually in scope

Monitoring dashboards and alert response
Scheduled eval runs and the drift review
Prompt and configuration fixes for defects
Model migration testing when a provider retires a model
Incident reports and root-cause write-ups
Security patches to the agent code and its dependencies
Updates to the eval set at each review

Usually priced separately

New workflows, new tools, or new data sources
Large changes in input format or volume
Retraining or fine-tuning on new data
Model and API usage charges
Training for your staff
Work caused by changes on your side

Add the one-line rule: anything not listed is out of scope, and changes follow a written change-order process.

Remedies

Remedies are what make the clauses real. Without them the floor is a hope.

Service credits tied to missed accuracy or escalation targets, not only to downtime. Mayer Brown describes credits that trigger on performance failures.
Cure period. The vendor gets a set number of days to fix a miss.
Suspension right. You can pause the agent at no penalty.
Termination for repeated misses. For example, three misses in a quarter.
Audit and logs. Decision logs you can read. See what to log.
Liability. Stanford’s summary of the TermScout data says 88 percent of AI vendors cap their liability. Expect a cap and negotiate it, with carve-outs for things like data breaches.
Exit. Your right to take the prompts, eval sets, and documentation with you. See ownership and exit rights.

If you are issuing an RFP, put these terms in it from the start. My AI agent RFP template shows where.

For a framework to anchor the governance language, the NIST AI Risk Management Framework organizes the work into Govern, Map, Measure, and Manage. Its Generative AI Profile followed in July 2024. It is voluntary, but it gives you shared vocabulary with a vendor’s compliance team.

Bottom Line

An uptime guarantee does not tell you whether the agent is right.

Write the SLA around an accuracy floor on an agreed eval set, escalation times, drift review, model-change notice, and a maintenance scope. Attach remedies to quality, not only to downtime.

Next step: before you sign, build the eval set with the vendor and agree on the floor in writing.

FAQ

What should an AI agent SLA cover? It should cover uptime, but also an accuracy floor on an agreed test set, escalation and incident response times, a drift review process, notice of model changes, a maintenance scope, and remedies. Each should name a number, a method, and an owner.

How do you measure accuracy in an SLA? Agree on a fixed, versioned set of real cases with correct answers. Define the metric in plain words, set a floor and a measurement window, and say who runs the test. Use enough cases that normal variation does not trigger false misses.

What happens when the model changes? It depends on your contract. Providers retire models on their own schedules, and Anthropic, for one, states at least 60 days’ notice for public models. Your vendor should pass that on, test the new model on your eval set before go-live, and give you a right to delay a change that lowers scores.

Who pays to fix drift? Agree on it before launch. A common split is that the vendor pays when drift follows from its own changes, and you pay or use retainer hours when it follows from changes on your side. For unclear cases, set a joint diagnosis deadline.

What belongs in a maintenance retainer? Monitoring, scheduled eval runs, drift reviews, defect fixes, model migration testing, incident reports, and security patches. New workflows, new data sources, and retraining are usually priced separately. Write the list down.

Is an AI agent SLA enforceable if the vendor uses a third-party model? It can be, if the vendor, not you, takes responsibility for the service you buy. Expect the vendor to resist promises about things it does not control, such as a provider outage. Negotiate what it does control: testing, rollback, notice, and response times. Ask your counsel how liability is allocated.

Three Ways to Work With TMA

Need an agent built? We deploy production AI agents in your infrastructure. Working pilot. Real data. Measurable ROI. → Schedule Demo

Want to co-build a product? We’re not a dev agency. We’re co-builders. Shared cost. Shared upside. → Partner with Us

Want to join the Guild? Ship pilots, earn bounties, share profit. Community + equity + path to exit. → Become an AI Architect

AI Agent SLA: What to Put in the Service Agreement