AI Agent KPIs: Three Numbers a COO Should Review Weekly

A COO who runs a back-office agent needs three AI agent KPIs in a weekly review. Not thirty.

1. Tasks completed without rework.
2. Exception and escalation rate.
3. Cost per completed task.

Most KPI guides are written for customer support, where the metrics are containment and CSAT. Back-office work is different. A mis-posted invoice or a wrong claim code does not show up in a survey. It shows up in a month-end close. Here is how I’d set up the sheet.

Why usage is not value in AI agent KPIs

The easiest numbers to pull are usage numbers: logins, prompts, sessions, tasks started. They go up fast. They tell you people are trying the tool. They do not tell you the work got done.

Google Cloud’s guide to KPIs for production AI agents (February 2026) splits agent metrics into three groups: reliability and efficiency, adoption and usage, and business value. Adoption is one group of three. It is not the goal.

Perception is also a poor measure. In a 2025 study by METR, 16 experienced developers worked on 246 real issues. They took 19 percent longer with AI tools than without. They still believed AI had sped them up by 20 percent. That was a coding study with a small sample, so I would not stretch it to your back office. But it shows why people’s impressions are not a metric.

And demo accuracy is not production accuracy. Salesforce AI Research’s CRMArena-Pro benchmark found leading agents reached around 58 percent success on single-turn CRM tasks and about 35 percent on multi-turn ones. The CMU-led TheAgentCompany benchmark found the best agent completed 30 percent of tasks on its own. Those papers date from 2024 and 2025, and models have improved since. The point stands. Measure your agent on your work.

Number 1: Completed tasks without rework

This is the one number I’d keep if I could only keep one.

Definition: the share of tasks the agent started that finished and stayed finished. No human edit, no reversal, no reopen within a set window.

Formula:

Rework-free completion rate = tasks completed with no human correction / tasks started

Define “completed” and “rework” in writing before launch. Examples for back-office work:

• Invoice coded and posted, and not recoded within 14 days.
• Claim intake created, and not reopened for missing fields.
• Ticket triaged to the right queue, and not rerouted.
• Vendor record updated, and not reverted.

Rework counts any correction made after the agent reported done. It does not include a case the agent sent to a person on purpose. That is an exception, and it goes in Number 2.

Google’s guide offers matching measures for agents that run in the background: acceptance rate, implicit rejection rate (how often humans undo or revert the agent’s work), and time-to-verify. Its warning on time-to-verify is the one I’d repeat: if review takes longer than doing the task by hand, the friction outweighs the value.

Track time to correct next to this number. It is how long a correction takes once someone spots the error. A 2 percent rework rate that takes three days to catch is worse than a 6 percent rate caught the same day.

Number 2: Exception and escalation rate

Definition: the share of tasks the agent could not finish alone. It has two parts. Keep them apart.

• Escalation rate: the agent hands off by design. It hit a rule, a low confidence score, or a missing input.
• Failure rate: the agent got stuck, timed out, errored, or gave up without a clean handoff.

Escalation is healthy at the start. A good agent knows when to stop. Failure is the number to drive down. For the design of the handoffs, see AI agent escalation design.

What is a good exception rate?

I do not have a clean benchmark to give you, and I’d distrust anyone who does. Exception rates depend on the workflow, the data quality, and how the term is defined. The published figures I found for back-office work came from vendor blogs or from secondary summaries I could not trace to the original study. I did not use them.

So build your own baseline:

1. Measure the human exception rate on this workflow today. How often does a person get stuck, ask a question, or send it back?
2. Set a launch target at or above that rate. A new agent will escalate more than your experienced staff.
3. Set a 90-day target that is lower, with a floor you accept. Some cases should always go to a person.
4. Watch the trend, not just the level. A rising rate means something changed: data, volume, or a model.

Anthropic’s own measurements show why the trend matters. In its autonomy research (February 2026), the average number of human interventions per session in internal Claude Code use fell from 5.4 to 3.3 over about four months, while the success rate on the hardest tasks doubled. Interventions fell while success rose, and the team tracked both over time. The study also notes that there is no agreed definition of an agent session. Define your own unit and keep it fixed.

Number 3: Cost per completed task

Definition: the full weekly cost of running the agent, divided by tasks completed without rework.

Cost per completed task = (platform + model usage + review labor + maintenance share) / rework-free completed tasks

Divide by rework-free tasks, not by tasks started. That way failures and rework raise the cost, as they should. Google’s guide makes the same case for pairing cost with success rate, since a cheap agent that fails often is expensive per outcome.

Include:

• Platform and model fees. See what an agent costs per month to run.
• Human review time, at a loaded hourly rate. This is the cost teams forget.
• A share of maintenance and engineering time.
• Rework time, which is review labor spent on fixes.

Compare it to the manual cost per task for the same work. If you have no baseline, measure it for a week before launch. The comparison also feeds your ROI math and your payback period.

A worked example

This is a hypothetical week for an invoice-coding agent. The numbers are made up to show the math.

Item	Value
Tasks started	1,200
Completed by the agent	1,020
Exceptions (escalated or failed)	180
Completed, then reworked within 14 days	61
Completed without rework	959
Platform fees	$900
Model usage	$310
Review labor: 22 hours at $45	$990
Total weekly cost	$2,200

The three numbers:

• Rework-free completion: 959 / 1,200 = 79.9 percent.
• Exception rate: 180 / 1,200 = 15.0 percent. Split it: say 150 escalations and 30 failures.
• Cost per completed task: $2,200 / 959 = $2.29.

Say the manual cost is $6.50 per invoice. The agent is cheaper. But 20 percent of tasks still need a person, and the rework rate on completed tasks is 6.0 percent (61 / 1,020). Those two figures tell you where to work next.

Leading indicators

The three numbers are the scoreboard. These warn you before they move.

• Coverage: the share of eligible volume that actually goes to the agent. If it is low, the agent is not being used, or the routing rules are too strict. This is also the best way to measure adoption in a back-office setting.
• Override rate: how often reviewers change the agent’s proposed answer.
• Time to verify: how long a reviewer takes per case. Rising time means trust is falling or the outputs got harder to check.
• Input quality: missing fields, bad scans, or new document formats.
• Queue age: the age of the oldest item waiting on a person.
• Spend per day: a sudden jump means a loop, a retry storm, or a change in volume.

How do I measure adoption?

Use coverage plus override rate, not logins. Coverage says whether work is flowing through the agent. Override rate says whether people accept what it produces. Workday’s list of AI KPIs groups metrics into accuracy, efficiency, user experience, and cost. Those four groups are a fair check that your sheet leaves nothing out.

For deeper tracing and logs behind these numbers, see AI agent observability tools.

Weekly review agenda

Thirty minutes. Same time each week. Same sheet.

1. The three numbers vs target and vs last week (5 minutes). Red, amber, or green.
2. Top three causes of exceptions (7 minutes). Read ten real cases, not a summary.
3. Rework cases (5 minutes). What was wrong, how long to find it, who caught it.
4. Leading indicators (3 minutes). Coverage, override rate, queue age, spend.
5. Changes since last week (3 minutes). Prompts, models, tools, data. See change control for AI agents.
6. Decisions (7 minutes). One owner and one due date per action.

Attendance: the process owner, the agent’s builder, and one reviewer who does the work. The COO reads the sheet and asks questions. At the end of each quarter, use your scale-or-stop criteria to decide the next step.

Dashboard layout

Keep it to one screen.

Top row:     [Rework-free completion %]  [Exception rate %]  [Cost per completed task $]
             each with target, last week, and 8-week trend line

Middle row:  Exceptions by cause (bar)     Rework by cause (bar)

Bottom row:  Coverage %   Override rate %   Time to verify   Queue age   Daily spend

Rules for the sheet:

• Three headline numbers, large. Everything else is small.
• Every number has a target and a trend. A number alone says little.
• Every red number links to ten example cases.
• Definitions sit on the page, so nobody argues about what “done” means.

Bottom Line

Three numbers, every week: rework-free completion, exception rate, and cost per completed task.

Define “done” and “rework” before launch. Baseline the manual process first. Split exceptions into planned escalations and failures.

My recommendation: build the sheet in the pilot, not after it. If you cannot measure the agent in week one, you will argue about it in month six.

FAQ

What KPIs should AI agents have? For back-office agents, start with three: tasks completed without rework, exception and escalation rate, and cost per completed task. Add coverage, override rate, and time to verify as leading indicators. Skip vanity metrics like sessions or prompts.

How do I measure AI agent adoption? Measure coverage, the share of eligible work that goes through the agent, and override rate, how often reviewers change its output. Logins and prompt counts show curiosity, not adoption.

What is a good exception rate? There is no reliable universal benchmark. Baseline your human exception rate on the same workflow, set a launch target at or above it, then a lower 90-day target. Separate planned escalations from failures, and watch the trend.

What is cost per completed task? It is the total weekly cost of the agent divided by tasks completed without rework. Include platform fees, model usage, human review time, and a share of maintenance. Dividing by rework-free tasks means failures raise the number.

How often should I review AI agent metrics? Weekly for the first quarter, in a 30-minute meeting using the same sheet each time. Once the numbers are stable, you can move to every two weeks, with alerts for any red threshold in between.

Three Ways to Work With TMA

Need an agent built? We deploy production AI agents in your infrastructure. Working pilot. Real data. Measurable ROI. → Schedule Demo

Want to co-build a product? We’re not a dev agency. We’re co-builders. Shared cost. Shared upside. → Partner with Us

Want to join the Guild? Ship pilots, earn bounties, share profit. Community + equity + path to exit. → Become an AI Architect