A new arXiv preprint introduces a method for catching when an AI model's chain-of-thought reasoning doesn't actually connect to the answer it produces — and it works from transcripts alone, with no live intervention needed [S1]. The approach, called reasoning consistency scanning, targets a blind spot that has haunted AI safety evaluations: models that spin out fluent, plausible reasoning steps quietly decoupled from their final output [S1]. But the method rests on a deliberate distinction that could change how auditors approach the problem — and on a benchmark of just 60 transcripts.

The problem that won't go away

Chain-of-thought prompting — asking a model to show its work, step by step — has become one of the most widely used techniques in AI. The promise is transparency: if you can see the reasoning, you can trust the answer. But prior research has shown that chain-of-thought reasoning is often unfaithful, meaning the reasoning a model states doesn't reliably reflect the actual process that produced its output [S1]. The model isn't lying, exactly. It's producing a plausible-sounding narrative that may or may not have anything to do with how it actually arrived at its answer.

The trouble is, detecting that unfaithfulness has required controlled experimental interventions — essentially poking the model in real time to see what changes. You can't do that with a transcript after the fact. If you're auditing a safety evaluation that already ran, the intervention window has closed [S1].

The shift: consistency, not faithfulness

The paper's core move is to separate two ideas that have been tangled together. Faithfulness — does the stated reasoning reflect the real process? — is hard to verify without intervention. But consistency — does the stated reasoning logically support the answer it accompanies? — can be assessed from a transcript alone [S1].

Think of it this way: faithfulness asks whether the model's diary is truthful. Consistency asks whether the diary's story matches the outcome. You don't need to have been in the room to check the second one.

The authors formalise this distinction and define a six-subtype taxonomy of inconsistency — a structured way to categorise the different ways a model's reasoning can fail to connect to its answer [S1]. They then built a validated benchmark of 60 transcripts, manually adapted from outputs of InstrumentalEval, an existing evaluation framework [S1].

A working scanner, open-source

The paper doesn't stop at theory. The authors implemented a working scanner for InspectScout, a safety evaluation platform, and describe it as the first to target logical consistency in safety evaluation transcripts [S1]. The code is public on GitHub under an MIT licence [P3].

The tool flags instances where a model's explanation fails to align with its conclusion — where the step-by-step justification neither leads to nor supports the response that follows [P3].

Results across four generator models and three evaluations from inspect_evals showed that reasoning inconsistency is present, detectable, and varies systematically across both models and task types [S1]. In other words, some models and some kinds of tasks produce more disconnected reasoning than others — a pattern that could help auditors prioritise where to look.

What it means

For anyone who relies on chain-of-thought outputs to trust a model's answer — and that's increasingly everyone — this paper names a problem you've probably sensed but couldn't articulate. A model can produce a flawless-looking chain of reasoning and still give you an answer that the reasoning doesn't support. Until now, catching that required being in the room with the model, running controlled experiments in real time. Reasoning consistency scanning offers a way to audit after the fact, from the transcript alone.

The distinction between faithfulness and consistency matters because it changes what's auditable. Faithfulness is a question about the model's internals — was the stated reasoning the actual mechanism? That's a research question requiring experimental access. Consistency is a question about the transcript — does the reasoning logically lead to the answer? That's an audit question, answerable from the record. By separating them, the paper opens a practical path for safety teams who don't have access to model internals but do have evaluation transcripts.

The six-subtype taxonomy gives auditors a shared vocabulary. Instead of saying "the reasoning seems off," an auditor can categorise the specific type of disconnect — which makes findings comparable across evaluations and models.

What it means for business

For a two-person AI consultancy running model evaluations for clients, this matters directly. If you're producing safety reports or model assessments, the ability to scan transcripts for reasoning-answer mismatches — without needing to re-run the model with experimental interventions — could become a standard audit step. The scanner is open-source under MIT licence [P3], meaning it can be integrated into existing evaluation pipelines.

For a compliance team at a financial services firm using LLM agents for decision support, the paper highlights a risk that's been hard to quantify: the model's stated reasoning may look sound while being logically disconnected from its recommendation. A related preprint, "Verify Before You Commit," makes a complementary point — in LLM agents, coherent reasoning can still violate logical or evidential constraints, allowing unsupported conclusions to slip through [P4]. Together, these suggest that "the reasoning looks good" is not sufficient assurance.

For a suburban agency building AI-powered tools, the practical takeaway is to treat chain-of-thought outputs as evidence, not proof. A model that shows its work is more auditable than one that doesn't — but only if someone actually checks whether the work supports the answer.

What we don't know yet

The benchmark is small — 60 transcripts, manually adapted from one evaluation framework [S1]. Whether the method generalises beyond the four generator models and three evaluations tested is an open question. The authors themselves don't claim universal validation.

The paper is an arXiv preprint, not peer-reviewed [S1]. The author's claim that this is the first scanner to target logical consistency in safety evaluation transcripts is unverified and may be contested [S1].

The distinction between consistency and faithfulness is technically precise. Consistency doesn't tell you whether the model's reasoning is faithful — only whether it's logically coherent with the answer. A model could be perfectly consistent and still unfaithful, or faithful but inconsistent in a particular case. Conflating the two would misrepresent the paper's scope.

The next thing to watch: whether the open-source scanner [P3] gets adopted by evaluation platforms beyond InspectScout, and whether the six-subtype taxonomy survives scrutiny as other researchers test it against their own transcripts. The broader question — whether consistency scanning becomes a standard audit step or remains a research tool — will be answered by the teams who pick it up.

Sources: [S1] arXiv cs.AI new (official RSS), arXiv:2607.07229v1, 9 July 2026 · [P2] arXiv HTML, arxiv.org/html/2607.07229 · [P3] GitHub, SilviaSantano/Reasoning-Consistency-Scanner · [P4] arXiv, "Verify Before You Commit: Towards Faithful Reasoning in LLM Agents via Self-Auditing"

Sources

More from Not A Tech Guy

Generated from an audited evidence pack with primary-source research. Social-media items are discussion signals, not verified facts. Nothing here is financial, legal or medical advice.