An arXiv preprint published on 16 July 2026 by researchers from Microsoft and Walmart Global Tech describes a framework that submits structured adversarial prompts to AI models at various levels of sophistication, ranging from straightforward harmful requests to encoded attacks intended to evade safety filters [S1][P2]. According to the researchers, the most technically complex category, encoded prompts, had the highest rate of success in evading these safety mechanisms [S1]. If the most difficult-to-detect attacks are the most effective, how many organisations using AI are genuinely evaluating them?

How the framework works

The framework, known as the Adversarial Prompting Framework (APF), creates adversarial prompts with increasing levels of complexity [S1]. At the most basic level, it sends direct harmful requests to a model, similar to what a user might openly type. At the advanced level, it encodes those identical requests so that the model's safety filters have difficulty identifying them as hazardous [S1]. The authors, Yash Bhatnagar at Microsoft, along with Kunal Banerjee and Anirban Chatterjee at Walmart Global Tech, state that this tiered method allows organisations to assess where and how their AI systems fail, rather than merely determining if they fail [P2][S1].

This paper contributes to an expanding collection of research on automated red-teaming for large language models. In May 2024, Facebook Research published AdvPrompter, an earlier tool for adversarial prompting [P3]. Another automated red-teaming framework for large language model security was published on arXiv by a separate research team [P4]. The APF paper introduces a claim of enterprise-level application: the researchers assert their implementation delivers automated testing with quantitative security metrics tailored for corporate settings [S1].

What it means

Adversarial prompt attacks function by designing inputs that deceive an AI model into generating content it is meant to deny, such as harmful instructions, leaked training data, or biased outputs. The APF framework approaches this as a testing challenge instead of an offensive instrument. By creating prompts at rising levels of sophistication and logging which ones succeed, it generates a map indicating where a model's defences remain intact and where they fail [S1].

The primary discovery, that encoded prompts evade safety mechanisms most efficiently, is significant because encoding is the attack method least likely to be detected by standard content filters [S1]. A filter trained to identify harmful keywords in plain text might overlook the identical request when concealed through encoding. If a model accepts encoded input and processes it without identifying the underlying intent, the safety layer is bypassed without the attacker needing to exploit a software vulnerability. The attack operates through the model's inherent text-processing ability.

The researchers note significant variations in model vulnerabilities across different attack vectors [S1]. Put simply, a model that denies a direct request for harmful content might still comply if the same request is encoded. The difference between those two results represents the security vulnerability the framework aims to identify and quantify.

This preprint tackles the issue from the opposite direction: instead of constructing stronger defences, it develops more effective attacks to reveal where current defences break down.

What it means for business

For a two-person law firm utilising an AI assistant to draft client correspondence, the risk is tangible. If a model can be persuaded via encoded prompts to generate content it should deny, the firm faces a compliance gap undetectable by standard testing. The APF framework's assertion of automated testing with quantitative metrics [S1] suggests a tool capable of executing a series of adversarial prompts against a deployed model and providing a scorecard.

For a suburban real estate agency inputting customer queries into a chatbot, the same reasoning holds. The chatbot might deny a direct request for inappropriate content, yet an encoded version of that same request could evade detection. Without systematic testing across different levels of sophistication, the agency has no method to determine this.

The researchers are affiliated with Microsoft and Walmart Global Tech [P2], lending some credibility to the enterprise framing. However, the assertions of practical enterprise application are self-evaluated and lack independent validation [S1]. The paper has not undergone peer review [S1].

What we don't know yet

The preprint does not identify which AI models underwent testing, and the abstract does not reveal the quantitative success rates supporting the claim that encoded prompts were most effective [S1]. Lacking those figures, evaluating how much poorer models perform against encoded attacks compared to direct ones is impossible.

The paper is categorised under both cs.CR (cryptography and security) and q-fin.GN (general finance), yet the abstract presents the framework broadly instead of focusing on financial institutions [S1]. The finance classification might stem from author affiliations or intended applications, but the evidence does not substantiate a finance-specific interpretation.

The framework's assertions regarding enterprise deployment stay unverified. No named organisation is confirmed to be utilising APF in production [S1]. Peer review, should it occur, will examine whether the quantitative metrics withstand scrutiny and whether the encoded-prompt result is reproducible in independent tests.

The next development to monitor is whether the researchers publish code or a comprehensive technical report containing model names and success-rate percentages. Absent those, the framework remains a methodology description rather than an executable tool. If you want to follow what happens when the data is released, subscribe and we will provide it.

Sources

More from Not A Tech Guy

Generated from an audited evidence pack with primary-source research. Social-media items are discussion signals, not verified facts. Nothing here is financial, legal or medical advice.