Video-DeepResearch-35B-A3B, an open-weight model described in an arXiv preprint posted August 4, scored 64.0% average accuracy on a new video reasoning benchmark, beating Claude 4.5 Sonnet (59.0%), Gemini 2.5 Pro (57.5%), and GPT-5 (52.5%) S¹. A model whose code sits on GitHub, outscoring the most expensive proprietary systems on the planet. Whether that number holds up depends on something the preprint's abstract does not mention.

My read: This is the first video reasoning agent I've seen that forces a model to exhaust visual grounding before it can touch web search. The staged tool-access idea is a genuine fix for a real problem: the authors found that current agents skip looking at the video and just search text instead. But I don't buy the headline comparison yet. The team built the benchmark, wrote the 200 questions, and graded their own model against it. That's standard for a preprint, but it means 64% is a claim, not a verdict. The number I'd watch is whether a third party replicates this on an external benchmark.

Why current video agents cheat

The authors diagnosed two failure modes in existing multimodal agents when faced with video S¹. First, modality bias: when you ask a model a question about a video, it tends to skip the visual tools and just search the web for text answers. Second, parametric knowledge leakage: the model answers from its training memory rather than actually using tools to investigate S¹. Both are shortcuts. Both produce answers that sound right but are not grounded in what is actually on screen.

Think of a student who, asked to analyse a film clip, Googles the plot summary instead of watching the scene. The answer might be correct. It might also be wrong, and you would never know which.

How Video-DR forces the model to watch

The fix is what the authors describe as a pipeline that separates visual analysis from web search, granting access to tools only in stages S¹. In plain terms, the agent cannot access web search until it has exhaustively grounded its answer across multiple frames of the video. Tools unlock in stages. First, visual perception. Then, only after the model has done the visual work, web retrieval becomes available.

The training recipe has two stages S¹. Supervised fine-tuning teaches the model the basic tool-use patterns. Then Group Relative Policy Optimization (GRPO), a reinforcement learning method, lets the agent explore autonomously. The authors argue this pushes past the limits of imitation-based training S¹. That ceiling is their own framing, not an established industry term, but the idea is that pure imitation learning caps how creative an agent can be. Reinforcement learning lets it discover strategies the training data never showed it.

The numbers on Video-DR-Bench

The team curated Video-DR-Bench, a benchmark built through a mix of human and AI effort, consisting of 200 complex questions that require chaining multiple pieces of evidence together S¹. Sometimes that evidence comes from the video, sometimes from the web, sometimes both.

Video-DR-Bench accuracy by model

The 30B-A3B variant, a smaller version, reached 59.3%, roughly matching Claude 4.5 Sonnet S¹. That matters because it suggests the architecture scales down without collapsing.

The self-graded problem

Every number above comes from one source: the team that built Video-DR also built Video-DR-Bench and ran the evaluations S¹. The paper is an arXiv preprint, not peer-reviewed S¹. The model names in the comparison (Claude-4.5-Sonnet, GPT-5, Gemini 2.5 Pro) are the authors' test labels and may not correspond to currently shipping commercial products S¹. Nobody outside the team has replicated these results.

This is not unusual for a preprint. It is the normal first step in research. But it means the 64% figure is a self-reported score on a self-authored test, and the gap between Video-DR and the proprietary models could narrow or vanish under independent evaluation.

A more capable video reasoning agent that autonomously browses the web while analysing footage expands the attack surface for video-based AI systems. The security implications of agents that watch, reason, and search deserve as much scrutiny as the accuracy numbers.

What to do about it

If you build systems that analyse video, the modality bias problem the authors identified is worth checking in your own stack. A media monitoring firm that tracks broadcast news for corporate clients, for instance, might ask its current tool to "find every mention of our CEO across last week's broadcasts and summarise the context." If the tool returns an answer that reads like a web search result rather than something grounded in the actual footage, you have modality bias. The Video-DR code is on GitHub S¹. You can run the benchmark locally and see whether your own video queries return visually grounded answers or text-search shortcuts.

One practical step this week: take five video clips your team already works with, ask your current AI tool a specific question about each one, and check whether the answer references something only visible in the frame. If it does not, your tool is doing what Video-DR was built to fix.

What we don't know yet

The results have not been peer-reviewed S¹. No third party has independently replicated the 64% score. The benchmark has 200 questions, which is small, and we do not know how it generalises beyond the authors' test set. The proprietary model names in the comparison may not match shipping products S¹. The "imitation-learning ceiling" is the authors' own concept, not a recognised industry framework.

The code is public on GitHub S¹, which means independent reproduction is possible. The next signal: the first independent reproduction attempt posted to the Video-DR GitHub repository. We will check the claimed 64% against it.

If you want to follow this story as it develops, subscribe and we will keep you posted.

Sources: S1 — Video-DeepResearch: Towards the Next-Generation Multimodal Deepresearc · P2 — Video-DeepResearch: Towards the Next-Generation Multimodal Deepresearc · P3 — Paper page - Video-DeepResearch: Towards the Next-Generation Multimoda · P4 — Watching, Reasoning and Searching: A Video Deep Research Benchmark on · P5 — QuantaAlpha/VideoDR-Benchmark

More from Not A Tech Guy

Generated from an audited evidence pack with primary-source research. Social-media items are discussion signals, not verified facts. Nothing here is financial, legal or medical advice.