An arXiv paper posted on 10 September 2026 asks a question that should unsettle anyone deploying AI agents: do the models know when they have actually succeeded? S¹ The authors propose two methods that read a model's internal activations to predict task outcomes, and claim this can be done with zero extra compute cost S¹. If that works, it changes how we monitor agents in production. If it doesn't, the gap between agent confidence and agent competence stays wide open.

My read: This is the first paper I've seen that tries to predict agent success from internal model states rather than from the agent's own output or external verification. The zero-overhead claim is the one to watch. Most reliability techniques cost either extra prompts or multiple model runs, and a method that needs neither would be genuinely useful. But the abstract has grammatical errors ("getting adopted", "an an interaction") and the paper has not been peer-reviewed S¹. I would treat the outperformance claim as a hypothesis until the full methodology and data are examined.

The problem: agents confidently fail

AI agents do something traditional machine learning systems don't. They plan, call tools, interact with dynamic environments, and then declare whether they succeeded S¹. That last step is where things break. A related arXiv paper, "From Confident Closing to Silent Failure," characterises this exact pattern: agents that assert they have succeeded even when their actions have actually failed P³. Salesforce AI Research has separately studied "Agentic Confidence Calibration" for the same reason P². The core issue is that an agent's verbal confidence and its actual competence are poorly aligned.

Techniques like StepGuard try to block harmful agent actions before they run. But blocking bad actions and knowing whether good actions worked are different problems. This paper targets the second one.

Reading the model's mind

The paper introduces two complementary methods S¹.

Latent Trajectory Dynamics (LTD) tracks how a model's residual-stream representations, the internal activation patterns that build up as a conversation progresses, change across an entire interaction S¹. Think of it as watching the model's internal state evolve turn by turn, looking for patterns that correlate with eventual success or failure.

The Action Representation Probe (ARP) takes a narrower approach. It examines the representations the model forms at the exact moment it decides on an action, and predicts from those whether the action will lead to success S¹. Where LTD looks at the whole trajectory, ARP zooms in on decision points.

Both methods share a key property: they read signals the model already computes. No extra prompts, no multiple model runs. The authors describe this as a zero-overhead reliability monitor S¹.

What the evidence actually shows

The methods were tested across three interactive benchmarks: Bash, SQL, and Python S¹. Three model families served as subjects: Qwen14B, Qwen7B, and DeepSeek6.7B S¹. The authors report that their approaches reliably beat baselines that rely on surface-level generation and sequence-based calibration S¹.

That claim needs context. "Surface-level generation" means judging success from the agent's own text output, which we already know is unreliable P³. "Sequence-based calibration" refers to methods that look at the sequence of tokens or actions for confidence signals. Beating both is meaningful if the margins are real, but the abstract does not disclose specific accuracy numbers, and the paper has not undergone peer review S¹.

A separate GitHub project, CalVerT by Ashwin Vinod, takes a related approach: enhancing agents with calibrated verifier telemetry to boost action and learning in knowledge-heavy tasks P⁵. That project, created in June 2026 with an MIT licence and written in Python and Shell, suggests the idea of reading internal signals for agent reliability is gaining traction beyond a single paper.

What to do about it

Consider a DevOps team that uses an AI agent to run database migrations. The agent executes a SQL migration script, reports "migration complete," and moves on. Two hours later, a downstream service breaks because a column rename didn't propagate. The agent was confident. It was wrong.

Today, catching that requires either a human review step or an external verifier that re-checks the database state. Both add cost and latency. The approach in this paper, if it holds up, would let the team read a confidence score from the model's internal state at the moment it decided the migration was done. A low score would trigger a human review. A high score would let the pipeline proceed automatically.

That is the practical promise. But until the paper discloses its full methodology and numbers, the most concrete thing a team can do this week is audit their own agent workflows for the false-success pattern. Run ten agent tasks where you already know the ground truth. Check whether the agent's stated confidence matches the actual outcome. If there is a gap, you have the problem this paper is trying to solve.

What we don't know yet

The paper does not disclose specific accuracy scores or quantitative metrics in its abstract S¹. The outperformance claim is self-reported by the authors and has not been independently verified. The methods have only been tested on three benchmarks (Bash, SQL, Python) and three model families (Qwen14B, Qwen7B, DeepSeek6.7B) S¹. Whether they generalise to other tasks, larger models, or closed systems like GPT or Claude, where internal representations are not accessible, remains an open question.

The abstract contains grammatical errors, suggesting an early or unedited preprint S¹. The paper has not been peer-reviewed or accepted to a conference S¹.

The next signal: whether a revised version (v2) appears on arXiv with full methodology and quantitative results. We'll check the outperformance claim against the disclosed numbers when it lands.

Subscribe and we'll flag the revision the moment it hits arXiv.

Sources: S1 — Do Agents Know When They Succeed? Calibrating Agent Confidence from In · P2 — Agentic Confidence Calibration · P3 — [2606.09863] From Confident Closing to Silent Failure: Characterizing · P4 — Agentic Confidence Calibration · P5 — ashwinn-v/CalVerT

More from Not A Tech Guy

Generated from an audited evidence pack with primary-source research. Social-media items are discussion signals, not verified facts. Nothing here is financial, legal or medical advice.