A new red-teaming framework called AMT-X achieved attack success rates of up to 100% against six frontier AI models, according to an arXiv preprint published on 14 July 2026 [S1]. The framework's strictest test, which required responses to contain complete and operational harmful detail, still broke through between 66.7% and 78.6% of the time [S1]. The gap between those two numbers, up to 33 percentage points, is where the real story hides [S1].

Most LLM safety testing today works like a bouncer checking IDs at the door. You fire a single harmful prompt at the model, a single judge scores the response, and you move on. AMT-X's authors argue this approach fundamentally underestimates risk because it ignores the most natural attack vector of all: a conversation [S1].

Real manipulation rarely lands in one message. It builds over turns, reading the target's reactions and adjusting. AMT-X, short for Adaptive Multi-Turn Exploitation, formalises that process into a phase-structured state machine [S1]. Each phase has a goal. The attacker reads semantic signals from the victim model's replies to decide when to advance, stall, or retreat. Nothing is free-form or ad hoc [S1].

The scoring system is equally deliberate. Instead of one AI judge giving a single score, AMT-X uses a multi-role jury with phase-conditioned checklists [S1]. Success is gated on whether the harm produced is actionable. Mere presence does not count. That distinction matters enormously, as the results show.

Two numbers, one uncomfortable truth

Under a lenient threshold, where any harmful content counts as a success, AMT-X scored between 97.6% and 100% across the six models [S1]. Under a stricter gate requiring the response to be complete, real, and operational, the rate dropped to between 66.7% and 78.6% [S1].

That is still a very high number. Even the strictest standard, the one demanding a working recipe rather than a vague gesture, failed more than two-thirds of the time.

The up to 33 percentage point gap between the two thresholds reveals something the authors highlight directly: current evaluation methods that treat any harmful output as a breach are measuring the wrong thing [S1]. A model that produces a half-answer it thinks is compliant is not safe. It is just better at appearing safe.

The evaluation covered seven Moderation sub-categories and ran against six frontier victim models queried under their default safety alignment, with no added moderation layers [S1]. The models are not named in the available excerpt, and the results have not been independently verified [S1].

What it means

The core finding is not that AI models can be tricked. Everyone in the field knows that. The finding is that the way we test for vulnerability is itself vulnerable. Single-turn attack datasets and single-judge scoring, the dominant approach today, miss the kind of slow, adaptive pressure that a determined adversary actually applies [S1].

AMT-X makes that pressure reproducible. By casting the attack as a state machine with defined phases and checklist-gated evaluation, the framework turns what was once an art into something measurable and repeatable [S1]. Other research groups have been moving in the same direction. A 2025 preprint on multi-lingual multi-turn automated red teaming explored similar territory across languages [P2]. The MTSA project on GitHub offers an open implementation of multi-turn safety alignment through multi-round red-teaming [P4]. RedCoder, another GitHub project, applies automated multi-turn red teaming specifically to code-generating LLMs [P5].

The broader current is clear. Safety testing is shifting from single-shot probes to sustained, multi-turn adversarial sessions. AMT-X adds a structured scoring layer that distinguishes between a model that leaks a fragment and one that delivers a full, working harmful output.

For a regular person using AI tools, the takeaway is simple. The safety filters you see on a chatbot are tested against quick, obvious attacks. They are far less tested against someone willing to spend ten messages carefully steering the conversation. The 66.7% strict-gate success rate means that even under demanding standards, a patient attacker succeeds more often than not [S1].

What it means for business

A two-person startup building a customer-facing AI agent has a concrete problem on its desk. If the agent handles user conversations across multiple turns, and the only safety testing the team has done is firing single harmful prompts during development, they are testing against the easiest attacks to survive.

AMT-X's results suggest that default safety alignment, the out-of-the-box filtering shipped by model providers, breaks under sustained multi-turn pressure between 66.7% and 78.6% of the time under strict conditions, and up to 100% under lenient scoring [S1]. A small firm relying solely on the provider's built-in alignment is operating with a false sense of security.

The practical response is not to panic but to test differently. Teams deploying AI agents should run multi-turn red-teaming sessions against their own applications as well as the base model. Open-source frameworks like MTSA [P4] and RedCoder [P5] offer starting points. The AMT-X paper's checklist-gated approach, which separates partial harm from operational harm, gives teams a way to score their own vulnerability with more honesty than a single pass-or-fail metric [S1].

For a suburban agency using AI to draft client communications, the risk profile is lower but not zero. Multi-turn manipulation of a drafting assistant is harder to exploit than an open-ended chatbot, but the principle holds: the more turns a user controls, the more surface area they have to steer output somewhere harmful.

What we don't know yet

The six frontier victim models are not identified in the available excerpt [S1]. Without knowing which models were tested, it is impossible to compare performance across providers or assess whether specific safety architectures fared better.

All attack success rates are self-reported by the authors and have not been independently replicated [S1]. The paper is an arXiv preprint and has not undergone peer review [S1]. The q-fin.GN classification on the preprint is unusual for an LLM safety paper, though it does not affect the content claims.

The up to 33 percentage point figure represents the maximum observed gap, not a uniform gap across all models and categories [S1]. Some models or attack categories may show a much smaller spread between lenient and strict scoring.

The next concrete event to watch is whether the authors release the model names, the full category breakdowns, or the code. Independent replication by a separate lab would be the real test of whether these numbers hold. Until then, the framework's methodology, structured multi-turn attacks with checklist-gated scoring, is the part most likely to influence how safety teams work, regardless of whether the headline numbers survive scrutiny.

If this kind of reporting is useful to you, subscribe to keep reading the stories that explain what AI developments actually mean for the people building with them.

Sources

Generated from an audited evidence pack with primary-source research. Social-media items are discussion signals, not verified facts. Nothing here is financial, legal or medical advice.