AI Agent Vendor Security Assessment: A One-Page Review

A standard vendor questionnaire asks where your data sits. An AI agent vendor security assessment has to ask what the software can do with it.

That is the difference. A chatbot reads. An agent acts: it sends, updates, approves, and deletes.

Below is a one-page review built for a buyer’s security team. Five areas, a few hard questions in each, and a scorecard at the end.

Why Agents Change Vendor Risk

Most vendor risk reviews assume software does what its code says. Agents add a model in the middle that decides what to do from text it reads.

Two things follow.

First, the vendor’s agent gets tool access. Tool access means credentials, and credentials mean your systems are now part of the vendor’s attack surface.

Second, the agent reads content nobody on your team wrote. Emails, tickets, web pages, uploaded files. Any of it can carry instructions. Security writer Simon Willison calls the dangerous mix the lethal trifecta: access to private data, exposure to untrusted content, and a way to communicate externally. If one agent has all three, he writes, an attacker can trick it into sending your data out. He also points out that once you combine those tools yourself, there is little a vendor can do to protect you.

So the first screening question for any agent vendor is simple: does one agent hold all three? If yes, ask what breaks the chain. See the glossary entry on prompt injection for how the attack works.

OWASP names the cause in its 2025 LLM list as Excessive Agency, with three roots: excessive functionality, excessive permissions, and excessive autonomy. Those three give you the structure for the rest of this review.

If you want the broader program around this, start with my enterprise AI security checklist. This post is the vendor-facing slice.

Area 1: Scope and Permissions

This area covers OWASP’s first two roots. What can the agent touch, and with what rights?

OWASP gives a clean example of too much permission: an extension meant only to read from a database that connects with an identity that can also update, insert, and delete.

Ask the vendor:

• What tools does the agent have, by name? You want a list, not a category.
• What is the narrowest permission each tool needs? Read-only where possible.
• Does each tool call run as the end user or as a shared service account? Shared accounts hide who did what and widen the blast radius.
• Can we restrict tools per customer, per workflow, or per user group?
• How do you stop the agent from calling tools outside its job? A prompt that says “do not do X” is not a control. A permission that blocks X is.

How to confirm the agent stays in scope: ask for a test. Have the vendor run a request that tries to push the agent past its limits in a demo tenant, and watch the tool-call log. If they cannot show you the blocked call, you only have their word.

For the full treatment of rights, read least privilege for AI agents.

Area 2: Secrets and Credentials

Where do the keys live, and who can read them?

This is where a vendor’s incident becomes your incident. In August 2025, Google’s threat intelligence team reported that an actor stole data from Salesforce customer instances using compromised OAuth tokens tied to the Salesloft Drift third-party app. The activity ran from at least August 8 through August 18, 2025. Salesloft and Salesforce then revoked all active access and refresh tokens for the app.

The lesson is not about one vendor. A token the vendor holds on your behalf is a key to your system. If their store is breached, your data is exposed.

Ask:

• Where are our credentials stored? A named secrets manager, with encryption at rest. Not environment files, not the prompt, not a database column.
• Can the model see a secret? It should not. Tool code should attach credentials after the model picks the action.
• Are tokens scoped and short-lived? Long-lived broad tokens are the Drift pattern.
• Can we bring our own credentials and revoke them in one step? Test revoke during the pilot.
• Do you validate token audience? The MCP specification calls accepting tokens not issued for the server an anti-pattern, and says servers must not accept them. If the vendor’s agent uses MCP, ask how they follow this. My post on MCP security for the enterprise goes deeper.

A good answer is specific and boring. A bad answer is “we take security very seriously.”

Area 3: Logging and Review Rights

If you cannot read what the agent did, you cannot investigate it.

NIST’s Guide to Computer Security Log Management, SP 800-92, has been a standard reference since 2006. It is not about AI, but the principle holds: logs only help if they are generated, kept, and reviewed.

For agents, “the logs” needs a definition. Ask for all of these:

• Every tool call, with arguments, result, time, and the identity it ran as.
• The user request and the model’s decision to call the tool.
• Approvals and denials, with who approved.
• Errors, retries, and blocked actions.

Then ask the access questions:

• Can we export these logs to our own system? Read-only access in the vendor’s dashboard is not enough for an incident.
• How long do you keep them, and can we set the period?
• Are logs tamper-evident?
• Do your staff read our prompts? Some providers review flagged traffic by default. Microsoft’s Azure OpenAI documentation, for example, describes an abuse-monitoring process where human reviewers can access flagged prompts and completions, and says approved customers can apply to modify it. Whatever the vendor’s model provider does, you want to know.

For a fuller list of what to capture, see what to log in an AI agent audit trail.

Area 4: Data Flow and Subprocessors

Name every party that sees your data. A normal agent has more than you expect: the model provider, a vector database, a logging tool, an email gateway, a search API.

If you operate under GDPR, Article 28 gives you a contract hook. A processor shall not engage another processor without prior written authorization from the controller. Where the authorization is general, the processor must tell you about additions or replacements so you can object. The same article requires the processor to pass your data protection obligations down by contract, and says the first processor remains fully liable if the other one fails.

Even if GDPR does not apply to you, ask for the same terms. Specific asks:

• A current subprocessor list, with what each one receives and where it runs.
• Advance notice of changes, with a right to object. Ask for the notice period in days.
• A statement on model training. Your data is not used to train any model, theirs or their provider’s.
• Retention at each hop. The vendor may keep nothing, but their model provider may keep prompts for abuse review. Ask for each hop in writing.
• Region. Where is data stored and processed?
• Audit rights. Article 28 also requires the processor to allow audits. A vendor who refuses is telling you something.

If you need the contract side, my list of questions to ask an AI agent vendor covers it.

Area 5: Incident Handling and Approval Points

Two things here: how humans stay in the loop, and what happens when something goes wrong.

Approval points. OWASP’s guidance for excessive autonomy is to require a human to approve high-impact actions before they run. Ask the vendor to list which actions need approval, who approves, and whether you can change the list. Sending money, deleting records, and emailing outside your company should be on it by default.

Incidents. Under GDPR Article 33, a controller must notify the regulator within 72 hours where feasible, and a processor must notify the controller “without undue delay.” “Without undue delay” is not a number. Put a number in the contract. I’d ask for 24 hours or less from the vendor’s awareness. That is my preference, not a legal rule.

Ask:

• How do we reach you at 2 a.m.? A named contact, not a support form.
• Can you shut off one agent or one customer’s access fast? Ask for the steps and time.
• What do you give us after an incident? Timeline, affected data, root cause.
• Do you test your response? Ask when they last ran an exercise.

The One-Page AI Agent Vendor Security Assessment Scorecard

Score each line 0 (no or unknown), 1 (partial), or 2 (yes, shown with evidence). Evidence means a document, a log export, or a live test. A sales claim is a 0 until proven.

#	Check	Score
1	Tools listed by name, each with narrowest permission
2	Agent does not hold private data, untrusted input, and external send in one path (or the chain is broken)
3	Calls run as the end user, not a shared super-account
4	Blocked out-of-scope call shown in a live test
5	Credentials in a secrets manager, never visible to the model
6	Tokens scoped and short-lived, revocation tested
7	Full tool-call logs exportable to our system
8	Log retention set by us, tamper-evident
9	Human approval on high-impact actions, list editable by us
10	Current subprocessor list with data, region, retention at each hop
11	Advance notice of subprocessor changes with right to object
12	No training on our data, stated in the contract
13	Audit rights in the contract
14	Incident notice number in hours, named contact, tested kill switch

How to read it. Maximum is 28.

• 24 or more: proceed to pilot.
• 18 to 23: proceed with the gaps written into the contract and a fix date.
• Below 18: not ready. Ask them to return when the gaps are closed.

Treat any 0 on lines 5, 7, or 9 as a stop until fixed. Those three are where the worst failures hide. The cut points are my judgment, not an industry standard. Adjust them to your risk.

Run it in a day. One person reads the vendor documents, one person joins a live call, and both score independently, then compare.

Bottom Line

Agents change vendor risk because they act, not because they are smart. Review what they can reach, where the keys live, what you can see, who else touches your data, and what happens when it fails.

Use the scorecard. Require evidence, not claims. Put the numbers in the contract.

My recommendation: do not sign until you have seen a blocked out-of-scope call and an exported log from a test tenant. Those two demos tell you more than any questionnaire.

FAQ

What do I ask a vendor whose agents have tool access? Ask for a named list of tools, the narrowest permission each one needs, and whether calls run as the end user or a shared account. Then ask which actions need human approval and whether you can change that list.

How do I confirm the agent is limited to its scope? Test it. Have the vendor run a request that tries to push the agent past its limits in a demo tenant, and show you the blocked call in the tool-call log. A prompt instruction alone is not a control. A permission that blocks the action is.

Where should secrets live? In a managed secrets store, encrypted, with the model never able to read them. Tokens should be scoped and short-lived, and you should be able to revoke them in one step. The 2025 Salesloft Drift incident is a reminder that vendor-held tokens are a way into your systems.

What logs can I read? You should be able to export every tool call with its arguments, result, time, and the identity it ran as, plus approvals and errors. Ask how long logs are kept and whether you can set that period. Read-only dashboard access is not enough for an investigation.

Which subprocessors see my data? Ask for the current list, what each receives, where it runs, and how long it keeps it. Under GDPR Article 28, a processor needs your written authorization to add another one. Ask for the same terms even if GDPR does not apply to you.

Three Ways to Work With TMA

Need an agent built? We deploy production AI agents in your infrastructure. Working pilot. Real data. Measurable ROI. → Schedule Demo

Want to co-build a product? We’re not a dev agency. We’re co-builders. Shared cost. Shared upside. → Partner with Us

Want to join the Guild? Ship pilots, earn bounties, share profit. Community + equity + path to exit. → Become an AI Architect