Google DeepMind researchers built a benchmark that asks AI agents to do what a radiologist or an electrical engineer does every working day: read a visually dense file, cross-reference it with others, and produce a finished deliverable. Even the strongest model they tested, Claude Fable 5.2, satisfied just 56.6% of the hardest grading criteria S¹. That number matters because the benchmarks already in use barely test visual reasoning: when the researchers removed image-viewing tools from Gemini 4 Carbon on GDPval, its score did not change S¹.
My read: The GDPval detail is the tell. If stripping image tools doesn't move the score, the benchmark wasn't requiring the model to look at anything. The 56.6% Climb-WSR figure is honest about how far multimodal agents still have to go, but I'd want to see absolute token counts behind the 5.7× efficiency claim before calling Gemini 4 Carbon the practical pick. The autorater's 96.2% agreement with human experts is measured on individual criterion judgments, not full deliverables, so the grading pipeline is less proven than the headline number suggests.
The gap MKA fills
Single-format evaluations like GDP.pdf and GDP.xlsx test whether a model can work within one file type. They do not require reasoning across files or across modalities S¹. An agent that reads a CAD blueprint, pulls a specification from a separate PDF, and exports a cost estimate as a spreadsheet is doing something fundamentally different from answering questions about a single document. That gap is what Anima Singh, Suhas Yogin and Weicheng Kuo, researchers at Google DeepMind, set out to close with the Multimodal Knowledge Agent benchmark P³. Google's announcement, from senior product manager Rohan Doshi and principal engineer Ben Caine, says the benchmark was developed with Artificial Analysis P².
What the numbers say
MKA contains 200 tasks spanning 10 economic sectors S¹. Each task feeds an agent multiple visually dense files: CAD blueprints, electronics schematics, radiographs, complex charts and geospatial maps S¹. The agent must extract and combine information across those files, then produce a standalone deliverable in a format a professional would actually use, whether .xlsx, .pptx, .pdf, .docx, .html or .png S¹. Deliverables are graded against expert-weighted rubrics by a sandboxed agentic autorater, which agreed with human domain experts on 96.2% of criterion judgments in an expert-graded study, corresponding to a Cohen's κ of 0.91 S¹. Tasks are scored on two metrics: weighted satisfaction rate (WSR), the share of rubric criteria the agent satisfies, and Climb-WSR, which focuses on frontier criteria that prior-generation models found difficult S¹.
The researchers evaluated four frontier models: Gemini 4 Carbon, Claude Fable 5.2, Claude Opus 5.5 and GPT-6 Astra S¹. Claude Fable 5.2 led with 56.6% Climb-WSR and 73.3% Overall WSR, edging Gemini 4 Carbon at 54.3% Climb-WSR and 71.9% Overall WSR S¹. Gemini 4 Carbon, however, used 5.7× fewer tokens than Claude Fable 5.2 to reach a near-identical score S¹. Even the strongest model stayed below 57% on Climb-WSR, which the authors describe as substantial headroom for multimodal agents S¹.

The paper is not peer-reviewed.
It appeared on arXiv on November 20, 2026 S¹. Ten sample examples are available on HuggingFace as google/multimodal-knowledge-agent, under an Apache 2.0 licence P⁴.
Who should pay attention
A construction firm evaluating whether an AI agent can draft a quantity survey from CAD drawings and a separate bill of materials would find MKA more relevant than any text-only benchmark. The same goes for a medical imaging team testing whether an agent can pull findings from a radiograph and format them into a structured report. The benchmark's insistence on real deliverable formats, rather than multiple-choice answers, is what separates it from the agent evaluations we have seen so far, as we found when OpenAI's BioBasicsBench tested agents on 100 wet-lab tasks and when OpenAI coding agents accelerated AI research, unevenly.
The practical first step: download the 10 sample examples from HuggingFace P⁴ and run one against your own agent stack. If it cannot produce a valid .xlsx from a CAD file and a PDF spec sheet, the 56.6% ceiling is not your problem.
Your agent is not even reaching the grading stage.
Sources: S1 — MKA: Evaluating Multimodal Knowledge Agents on end-to-end professional · P2 — Multimodal Knowledge Agents benchmark: November 2026 · P3 — MKA: Evaluating Multimodal Knowledge Agents on end-to-end professional · P4 — google/multimodal-knowledge-agent · Datasets at Hugging Face · P5 — FastLM/MKA
Related reading
- OpenAI's BioBasicsBench tests AI agents on 100 wet-lab tasks — our technology desk, 2026-11-07
- AI email agents comply with 45% of adaptive phishing attacks — our technology desk, 2026-10-24
- OpenAI coding agents accelerate AI research, unevenly — our technology desk, 2026-11-19
Written from 5 sourced items, 4 of them primary.