LeakGauge, a new detection method detailed in an arXiv preprint published 18 August 2026, flags context-leakage attacks on large language models with an AUROC of 0.944 to 0.996 across 11 models including GLM-5.2 and Kimi-K3 S¹. It costs fewer than 0.5K extra parameters and 10.34 milliseconds of latency, cheap enough to sit in front of every user query. The question every AI team now faces: if leakage leaves a detectable signal this reliable, what does that mean for the system prompts, retrieved documents, and API keys sitting inside production agents today?

My read: This is the first leakage-detection method I have seen that works from the model's own output probabilities rather than its internal hidden states, which means it could run in production without custom model access. The AUROC range is strong, but I want to see how it holds up against adaptive attackers who know the gauge suffix and can craft prompts to avoid triggering it. The 10.34 ms latency claim is the number that matters for deployment: if real, it is invisible to users. I am skeptical of the 0.996 upper bound until independent testing confirms it, because author-reported benchmarks on unseen attacks still test attacks the authors designed or selected.

The attack hiding in plain text

Context-leakage attacks exploit a simple architectural fact: modern LLMs process confidential context such as system prompts and retrieved documents in the same window as user queries S¹. When a user sends a carefully crafted adversarial prompt, the model can be induced to disclose those confidential contexts S¹.

LeakGauge targets a different layer from input-side defences: not the attack input itself, but the model's response, catching the moment the model begins to reveal what it should not.

Prior research suggested that leakage signals live in the model's hidden states, the internal representations computed during generation S¹. The problem is that extracting hidden states requires access to the model's internals, which is impractical for most deployments, especially API-only models where you never see the weights S¹.

How LeakGauge reads the model's tell

LeakGauge works by appending a suffix to the model's response, a short probe that gauges whether the model is in a leaking state S¹. It then reads the prefill token probabilities, the model's own confidence about what comes next, and maps those probabilities to an attack-risk score S¹.

The key design choice: the gauge is content-agnostic. Instead of checking whether the model is about to output the first tokens of the actual confidential content, LeakGauge verbalises leakage behaviour in general terms S¹. The authors found this approach yields steadier signals than trying to match specific secrets S¹.

That steadiness matters because attackers do not always copy text word-for-word. The signal holds when the content changes language or when the attack shifts from quoting the system prompt verbatim to paraphrasing its meaning S¹. A detector that only catches verbatim leaks would miss the more sophisticated attacks.

LeakGauge exploits the insight that the model's output probabilities shift in a detectable way when it is being induced to leak, even if the leak itself is subtle.

The authors tested LeakGauge across 11 LLMs, including GLM-5.2 at 753 billion parameters and Kimi-K3 at 2.8 trillion parameters S¹. On unseen attacks, the method scored an AUROC between 0.944 and 0.996 S¹. AUROC measures how well a detector separates genuine attacks from benign queries. A score of 1.0 is perfect; 0.5 is a coin flip.

Through activation-steering interventions, the authors showed their risk score is sensitive to an internal leakage-related direction in the model S¹. In plain terms: the observable signal from the output probabilities lines up with something real happening inside the model, a genuine internal pattern.

What to do about it

LeakGauge is a detection tool, not a prevention mechanism. It tells you a leak is happening or about to happen. It does not block the response.

For a team running a customer-support agent that retrieves internal policy documents and feeds them into the model's context, the workflow looks like this: every user query goes in, the model generates a response, and LeakGauge scores that response in 10.34 milliseconds using fewer than 0.5K extra parameters S¹. If the score crosses a threshold, the system can hold the response or flag it for review. The detector runs on the output side, so it works even with API-only models where you cannot touch the internals.

The code is public on GitHub S¹, which means a security team can clone it and test it against their own prompts to see whether the AUROC numbers hold for their specific model and threat model before trusting it in production.

One practical step this week: if your team runs an LLM agent with sensitive context, pull the LeakGauge repository from GitHub and run it against a small set of your own adversarial test prompts. The 10.34 ms latency claim and the AUROC numbers are author-reported from a preprint, so validate before you deploy.

What we don't know yet

The preprint has not been peer-reviewed, and every performance claim is author-reported S¹. The 11 tested LLMs are named only by two examples, GLM-5.2 and Kimi-K3, so the full list, and whether it includes API-only models like GPT or Claude, is not disclosed in the abstract S¹.

The AUROC scores were measured on unseen attacks, but unseen means attacks the authors held out from their own set, not attacks designed by independent adversaries who know LeakGauge exists. An adaptive attacker who can see the gauge suffix might craft prompts that avoid triggering the signal. The paper does not address this scenario in its abstract.

The parameter counts for GLM-5.2 (753B) and Kimi-K3 (2.8T) are presented as given by the authors, not independently verified S¹. Whether LeakGauge performs identically on smaller open-weight models or on closed API models remains an open question.

The next signal: peer review and independent reproduction. If a security lab publishes a replication study confirming the AUROC range on different attack sets, that is when this moves from interesting preprint to deployable defence. We will check the claim against it.

If you want this kind of analysis in your inbox every week, subscribe to keep reading.

Sources: S1 — The Model's Tell: Measuring Context-Leakage Attack Signals with Behavi · P2 — The Model’s Tell: Measuring Context-Leakage Attack Signals with Behavi · P3 — The Model’s Tell: Measuring Context-Leakage Attack Signals with Behavi · P4 — PRISM: Generation-Time Detection and Mitigation of Secret Leakage in M · P5 — safety-research/bloom

More from Not A Tech Guy

Generated from an audited evidence pack with primary-source research. Social-media items are discussion signals, not verified facts. Nothing here is financial, legal or medical advice.