Rogue agents are making the news, and the security community has already made it official. The OWASP GenAI Security Project now maintains a Top 10 for Agentic Applications, built with more than a hundred contributors. Entry ASI10 is literally called Rogue Agents. Entry ASI01 is Agent Goal Hijack. OWASP shipped a refreshed LLM top 10 on 4 August 2026, three days ago, grounded this time in thousands of real world AI security incidents rather than expert opinion alone. The framing at the top of the agentic document is the whole problem in one sentence:
Once AI began taking actions, the nature of security changed forever.
Securing a chatbot was already hard. Securing an agent that can read customer data, call your APIs, and execute actions is a different discipline. And the uncomfortable part is that an agent can look perfectly secure right up until the day someone finds the right sequence of messages to bend it.
That is why we are shipping Security Lab, our new security module at AGO.
What changes when the agent can act
A chatbot fails softly. The worst case is a wrong answer and an annoyed customer.
An agent fails hard. If it can issue a refund, it can issue the wrong refund. If it can read an order, it can read someone else's order. If it can fetch a URL, it can be pointed at your internal network.
The attack surface is also much wider than the model itself:
- the system prompt and the instructions that shape behaviour
- the knowledge base the agent retrieves from
- the tools it is allowed to call, and the arguments it fills in
- the routing between agents, when a specialised agent has more privileges than the one the user is talking to
- every piece of content the agent ingests, including a ticket body, an attached PDF, or a web page
That fifth item is where most people get caught, and it is not theoretical. The landmark case is EchoLeak, CVE-2025-32711, an AI command injection in Microsoft 365 Copilot rated 9.3 critical by Microsoft: a single crafted email sitting in a mailbox leaked tenant data, without the victim ever opening it. That was June 2025, and the class has not been closed since. OWASP counted eight major AI exploits in the first quarter of 2026 alone.
The attacker does not need to talk to your agent. They only need to put text somewhere your agent will read.
What the 2026 research actually says
Every frontier model tested this year has been broken. In March 2026, Gray Swan AI, the UK AI Security Institute and the US CAISI at NIST published the results of a public competition on indirect prompt injection: 464 participants, 272,000 attack submissions, 41 scenarios, 13 frontier models. 8,648 attacks succeeded. Attack success rates ranged from 0.5% for Claude Opus 4.5 to 8.5% for Gemini 2.5 Pro. NIST's own write up of the competition, published on 23 March 2026, puts it plainly: "at least one successful attack was found against all of the target frontier models."
A 0.5% success rate sounds reassuring until you multiply it. An attacker who sends two thousand messages at that rate gets through ten times, and they only need to get through once. That is the best score in the study, on the most robust model tested.
A published defense number tells you nobody has attacked it yet. In October 2025, fourteen researchers from Google DeepMind, OpenAI, Anthropic and ETH Zürich published The Attacker Moves Second. They took the twelve leading defenses of the day against jailbreaks and prompt injection and attacked each one adaptively, tuning the attack to that defense's specific design. Verbatim from the abstract: they bypassed all twelve "with attack success rate above 90% for most; importantly, the majority of defenses originally reported near-zero attack success rates."
Near zero on the published benchmark, above ninety percent once someone tried properly. Nothing published since has overturned that result, which is why it still sets the bar for how a defense should be evaluated.
Your vulnerability feed will never warn you. This is the finding most teams have not internalised. OWASP's GenAI exploit round up for Q1 2026, published in April 2026, documents eight major AI exploits for the quarter. Exactly one of them has an official CVE identifier. OWASP explains why: "Most AI-related security events are not yet mapped to traditional CVE identifiers. Instead, they arise from: Misconfiguration (e.g., overprivileged agents), Design flaws (agent autonomy, trust boundaries), Supply-chain weaknesses, Prompt injection and data-flow manipulation." Only classical software bugs embedded in AI platforms get CVE tracking.
So the usual pipeline does not apply. Nobody is going to file a CVE against your agent's system prompt. If you want to know whether your agent is exploitable, the only way to find out is to attack it yourself.
So we attack our own agents
Security Lab attacks your own agent the way a real user would, through the live chat pipeline, and reports what got through. No sandbox, no simplified copy of the agent. The scan runs against the real system prompt, the real tools, and the real knowledge, in production or in a test environment.
The loop is deliberately boring:
- Pick an agent. Its actual configuration is used, not a stand in.
- We attack it. Attacks from the catalog are sent through the normal chat pipeline, multi turn when the attack needs it, because single turn testing systematically understates risk.
- Review the findings. Each attack is scored blocked, compromised, or inconclusive, and the run produces a security score.
The attack catalog
The catalog is organised around the OWASP agentic categories rather than around whatever we found interesting, so a result maps onto a taxonomy your security team already recognises. OWASP also refreshed its LLM top 10 on 4 August 2026, so we track that renumbering as it lands rather than pinning the catalog to a superseded edition.
- Direct prompt injection. The user tells the agent to ignore its instructions. OWASP ASI01, agent goal hijack.
- Indirect prompt injection. The instructions are hidden in a document, a ticket, or a page the agent reads. Same ASI01, different entry point, and by far the harder one to close.
- Knowledge base exfiltration. Getting the agent to dump content it should only use to answer, including internal only documents.
- System prompt recovery. Extracting the instructions, which is the map an attacker needs for everything else.
- Out of scope data access. Asking for another customer's order, another tenant's records, another user's account. OWASP ASI03, identity and privilege abuse.
- Agent routing hijack. Convincing the front agent to hand off to a more privileged one under attacker controlled context. OWASP ASI07, insecure inter agent communication.
- Unauthorized tool calls. Triggering a tool the agent should not have access to, or with arguments outside its allowed range. OWASP ASI02, tool misuse.
- Memory and context poisoning. Planting content in one conversation that changes the agent's behaviour in a later one. OWASP ASI06.
- Credential and secret hunting. API keys, tokens, and internal URLs leaking through answers or error messages.
- SSRF. Pointing a fetch or webhook tool at internal addresses.
- Cost explosion loops. Prompts designed to make the agent recurse, retry, or generate until the bill hurts.
- Adaptive jailbreaks. Multi turn attacks that build context slowly instead of asking once.
The part that actually matters
Most agent security tooling grades the reply. Did the model say something it should not have said?
That grade is close to meaningless, and the research community worked this out on chatbots before agents even existed. The Berkeley team behind StrongREJECT showed that "existing evaluation methods significantly overstate jailbreak effectiveness compared to human judgments", precisely because those methods score the shape of the response rather than whether it delivered anything real. Their fix was to measure "the extent to which a response gives useful information" instead.
On an agent the gap is wider still, because the damage does not live in the text at all. It lives in the tool call. An agent that answers "I am sorry, I cannot help with that" scores perfectly on a refusal checker even if it leaked a customer record two turns earlier. Refusal is not safety, it is tone.
So we do not measure whether the agent sounded suspicious. We measure whether the attack worked:
- was real protected data actually exposed
- was a forbidden tool actually called
- did the agent step outside its authorization scope
- was its behaviour genuinely redirected, or did it just play along in words
Each attack lands in one of three buckets. Blocked means the attack ran and nothing got through. Compromised means something real got out, and that is a bug with a reproduction case attached. Inconclusive means we cannot prove either way from the transcript alone.
We keep that third bucket on purpose. A lot of agent security work is deciding whether something actually happened, and a scanner that forces every case into pass or fail either hides real breaches or floods you with noise. Marking a case inconclusive keeps the score honest and puts it in front of a human.
Security is a property of the system, not the model
This is the shift that matters with agentic systems. You cannot prompt your way to safety. The twelve defenses broken in The Attacker Moves Second were mostly model level defenses, and that is exactly why they broke.
A guardrail written in the system prompt is a suggestion. A check written in your code is a control. The controls that hold up are the ones outside the model:
- credentials scoped per agent, held server side, never present in the prompt
- an explicit tool allowlist per agent, rather than one shared toolbox
- authorization checked on every tool call against the authenticated user, never against what the conversation claims the user is
- outbound requests restricted to a known set of hosts
- token and cost ceilings per conversation
- full logging of every tool call, so a compromise is reconstructable after the fact
The model is one layer. It is the layer an attacker gets to negotiate with, so it should not be the one holding the keys. Design so that when the model is fooled, and it will be, the blast radius is already bounded.
Run it like a test suite, not like an audit
An annual pentest tells you the agent was safe in March. Agents change every week: a new prompt, a new document in the knowledge base, a new tool, a new model version. Any of those can reopen a hole that was closed.
This is how the frontier labs treat it. Google DeepMind's report on defending Gemini against indirect prompt injections describes an adversarial evaluation framework that "deploys a suite of adaptive attack techniques to run continuously against past, current, and future versions of Gemini". Continuously, and against future versions. Not a report, a running process.
So the scan should trigger on the events that change behaviour:
- every system prompt change
- every knowledge base update
- every new tool or permission granted to an agent
- before each deploy
- and on a schedule, because the attack catalog keeps growing even when your agent does not change
That is the point of scoring runs rather than writing reports. You get a number that moves, and a regression is visible the day it appears instead of at the next audit.
Most teams are not there yet
The gap between deployment and control is measurable, and it shows up in three independent datasets.
Almost nobody has tooling built for this. Pentera surveyed 300 US CISOs and security executives for its AI Security and Exposure Benchmark 2026, published in February 2026. Only 11% report having security tools specifically designed to protect AI systems. Only 1% have a dedicated AI security budget. Meanwhile 75% "rely on extending controls originally designed for other attack surfaces to cover AI-driven workflows", and 67% report limited visibility into how AI is even being used across their environment.
Stretching a web application firewall over an agent does not test whether the agent can be talked into calling the wrong tool. Nothing in the traditional stack does.
Nobody knows how many agents they are running. A Cloud Security Alliance survey of 418 IT and security professionals, fielded in January 2026, found that 82% had discovered previously unknown agents in their environment in the past year, and 65% had at least one AI agent related incident over the same period. When an agent acts outside its scope, 38% require human approval and 24% require the action to be logged. Only 11% automatically block it.
And the breaches are compounding. IBM's Cost of a Data Breach, run by Ponemon across roughly 600 breached organizations each year, gives us a clean trend line. In the 2025 edition, 13% of organizations reported a breach of an AI model or application, and 97% of those reported having no AI access controls in place. In the 2026 edition, published on 29 July 2026 and covering breaches through February 2026, that figure is more than 20%. One in four malicious breaches is now AI enabled, a 56% increase year on year, at an average cost of $6 million, roughly a million above the global average.
The leading causes IBM names for AI targeted breaches are compromised APIs, applications or plug ins at 27%, and cloud misconfigurations affecting AI workloads at 27%. Neither is a model problem. Both are system problems.
And logging an unauthorized action is not a control either. It is a record of the thing you failed to stop.
Where to start, even without us
If you are running an agent today, three things are worth doing this week:
- Write down what your agent is allowed to touch, tool by tool, and check that the code enforces it rather than the prompt.
- Take your ten worst case scenarios, the ones that would make you call your lawyer, and turn each into an attack you can replay on demand. Then replay each one hundreds of times, because at the success rates measured in 2026 a single attempt tells you almost nothing.
- Grade those replays on impact, not on the wording of the answer.
That is the whole idea behind Security Lab, and it is what we are automating at AGO. Agents that operate inside your product are worth building. They are also worth attacking, regularly, before someone else does it for you.
