Four of nine AI email agents tested by Carnegie Mellon researchers detected fewer than 10% of adaptive phishing emails and complied with malicious requests more than 45% of the time S¹. The same agents handled pre-existing phishing emails safely, which means the failure is not in recognising a scam but in resisting one written specifically for them S¹.

My read: This is the first benchmark I've seen that treats phishing as a persuasion problem rather than a classification problem. The gap between catching stock phishing and folding under adapted attacks is the real finding. I'm skeptical of the 45% compliance figure until the nine agents are named and the test is replicated independently, but the methodology, borrowing Cialdini's persuasion principles to reverse-engineer what makes agents fold, is the kind of thinking security teams should borrow.

The gap nobody was measuring

Computer-use agents increasingly read and act on email, handling private information across desktop and web applications S¹. Existing phishing datasets were built for conventional detection: train a classifier to flag a malicious message and move on S¹. That framework assumes the only question is whether a message gets flagged. It ignores what happens next, how an agent responds, what it volunteers, what it clicks.

This matters because an agent that detects a phishing email but still complies with its request is arguably worse than one that never detected it at all. The detection creates a false sense of security. As we found when a separate benchmark showed AI agents can't learn user habits even after 15 hours of observation, the gap between what agents appear to do and what they actually do reliably is where the real risk lives.

How the attack works

Justin Tan, Miao Li and Ding Zhao at Carnegie Mellon built AgentPhishing, a benchmark that tests whether persuasive phishing emails can evade detection and induce LLM email agents to comply with malicious requests S¹P². The system, called PhishOpt, optimises reusable email-generation skills against individual agents under fixed benchmark constraints and a realism constraint, meaning the emails have to look like something a real attacker might send S¹.

The authors then analysed the generated emails using Cialdini's persuasion principles, the six-lever framework from social psychology that covers reciprocity, scarcity, authority, commitment, liking and consensus, to identify which strategies correlated with agent failures S¹. Those findings fed into a universal generation skill designed to work across agents rather than against just one S¹.

The approach echoes a pattern we've seen in backdoor research that hides attacks inside logical reasoning: the most effective attacks on AI systems don't break the model, they work within its normal behaviour.

What the numbers say

In an evaluation of nine agents, four exhibited phishing-detection rates below 10% and malicious-request compliance rates above 45% S¹. The paper does not name the nine agents tested, and the statistics are self-reported from a controlled benchmark that has not been independently replicated S¹. The preprint, listed on arXiv under cs.CR and q-fin.GN, has not been peer-reviewed S¹.

The critical contrast is between the two conditions. Agents that handled pre-existing phishing emails safely, the kind already in public datasets, still fell for attacks adapted to their behaviour S¹. That split is the story: conventional phishing detection works, and adaptive phishing evasion works too.

Who should pay attention

Any organisation deploying LLM-based agents to triage, summarise or respond to email. The benchmark methodology, and the PhishNChips dataset the authors have made available on Hugging Face P⁴, give security teams a way to stress-test their own agents before deployment. A practical first step for a security lead: run your agent against adaptive phishing emails rather than the stock corpus it was likely trained on, and measure compliance rate alongside detection rate.

Microsoft's own LLMail-Inject challenge explored a related angle, running a public competition on adaptive prompt injection against email agents P³. The Carnegie Mellon work extends that thread from injection to persuasion, asking not whether an attacker can slip instructions past an agent but whether they can talk it into cooperating. This is the same trust boundary that OpenAI crossed when it trained agents on Ironclad contracts: every time an agent acts on a user's behalf, persuasion becomes an attack surface.

The next checkpoint is peer review and independent replication. Until then, the four-in-nine figure is a signal, not a verdict.

Sources: S1 — AgentPhishing: A Benchmark of LLM Email Agents Under Adaptive Attack · P2 — AgentPhishing: A Benchmark of LLM Email Agents Under Adaptive Attack · P3 — README.md · P4 — AreLit/PhishNChips · Datasets at Hugging Face · P5 — Open-Eval-Commons/OpenEval · Datasets at Hugging Face

Written from 5 sourced items, 4 of them primary.

Agent performance under adaptive phishing 4 of 9 worst performers