A July 2026 preprint analysing nine AI systems that answer questions about gastrointestinal endoscopy images found that high answer accuracy does not reliably produce faithful clinical reasoning [S1]. The systems that top challenge leaderboards can still give explanations that are incomplete or misleading. The finding raises a question every hospital AI buyer should be asking: if the score looks great but the reasoning is broken, what exactly are you trusting?

The gap between score and reasoning

The paper, posted to arXiv on 17 July 2026 by researchers including Sushant Gautam, Vajira Thambawita, and Pål Halvorsen, sits in the cs.AI and cs.LG categories and has not been peer-reviewed [S1][P2]. It uses the MediaEval Medico 2025 task as a retrospective case study, examining nine documented systems built for visual question answering, or VQA, in gastrointestinal endoscopy [S1].

VQA is what it sounds like: a model looks at an image and answers a question about it. In endoscopy, that might mean "what finding is visible in this frame?" or "is this a normal or abnormal region?" The Medico challenge draws teams that combine vision and language models to answer such questions, and the leaderboard ranks them by answer accuracy.

The preprint's central finding flips that ranking on its head. Fine-tuning large pretrained models, the dominant approach where you take an existing model trained on general data and adapt it cheaply for medical use, delivers strong challenge performance [S1]. But getting the right answer more often does not reliably produce faithful and complete clinical reasoning [S1]. A system can tell you the correct label while giving a justification that is wrong or vague.

This matters because the explanation is what a clinician would rely on to trust the answer. A correct label with a broken rationale is a coin flip wearing a lab coat.

Why structured reasoning helps

Not all nine systems performed equally on reasoning quality. Methods that enforce structured reasoning and explicit grounding, meaning the system must point to specific visual evidence in the image to support its answer, showed more reliable behaviour across different question types [S1].

This connects to a broader pattern in VQA research. A 2025 preprint on VQA reliability noted that vision-language models are prone to hallucinations, producing overconfident yet incorrect answers that erode trust [P4]. Facebook Research's 2022 ECCV paper on reliable VQA took a different angle, proposing that systems should abstain rather than answer incorrectly [P5]. The new preprint adds a third dimension: even when the answer is correct, the reasoning behind it may not hold up.

The authors are careful about what their evidence shows. The link between structured reasoning and reliable behaviour is correlational, not based on controlled experiments that isolate the effect by removing the component [S1]. The nine systems were analysed retrospectively from a shared challenge, not re-run on a newly created test set [S1]. So the paper identifies a pattern, not a proven cause.

What it means

For a reader with no background in medical AI, the takeaway is simple. A high score on a medical AI benchmark tells you the system gets the right answer often enough. It does not tell you why the system arrived at that answer, whether the reasoning is sound, or whether the same logic would hold on a different patient.

The preprint recommends four changes to how these systems are built and evaluated [S1]:

  • Evaluation beyond lexical overlap, meaning scoring should check whether the answer is semantically correct rather than only whether the text matches a reference string word for word.
  • Standardised evidence-linked explanations, where every answer comes with a pointer to the specific visual region or feature that supports it.
  • Leakage-aware data governance, to prevent training data bleeding into test sets and inflating scores.
  • Lightweight robustness and calibration checks, to test whether the system's confidence actually matches its accuracy.

These are design principles, not validated protocols. The authors frame their findings as supporting trustworthy multimodal healthcare AI based on data fusion, explainability, and resilient evaluation [S1]. But the word "trustworthy" describes an aspiration and a set of recommendations, not an empirically verified property of any system [S1].

What it means for business

For a small medical imaging startup or a two-person AI consultancy building tools for endoscopy clinics, this preprint is a warning about where to spend effort. Fine-tuning a pretrained model to get a high leaderboard score is the easy part. The hard part, and the part that determines whether a clinician will actually use the tool, is the reasoning layer.

A suburban radiology practice or a regional hospital considering an AI tool for endoscopy triage should ask the vendor a specific question: can the system show me what it is looking at? If the answer is a confidence score and a label, that is the lexical-overlap trap the preprint describes. If the answer includes a highlighted region and a structured justification, that is closer to what the authors recommend.

For vendors, the cost implications are real. Building structured grounding into a VQA system requires more than attaching a language model to a vision encoder. It means engineering a reasoning pipeline, curating training data with linked visual evidence, and running calibration tests that most challenge benchmarks do not require. That is engineering time, not API calls.

What we don't know yet

The preprint has not been peer-reviewed, and its claims may change during review [S1]. The evidence for structured reasoning is correlational, so we do not know whether forcing a system to ground its answers causes better reasoning or whether better-designed systems simply tend to include grounding [S1].

The findings are specific to GI endoscopy VQA. Whether the same pattern holds in dermatology, radiology, or pathology is not established [S1]. The nine systems were analysed retrospectively from a shared challenge, which limits how strongly the comparisons can be drawn [S1].

The next concrete event is the peer-review process, which will either confirm or challenge the correlational findings. Until then, the preprint is a signal, not a verdict. If you want to follow how AI evaluation is shifting from scores to reasoning, subscribe for the next update.

Sources

Generated from an audited evidence pack with primary-source research. Social-media items are discussion signals, not verified facts. Nothing here is financial, legal or medical advice.