# MKA Benchmark Explained: Choosing AI Agents for Visual Document Work The new evaluation makes document-heavy agents easier to compare, provided you understand what its score measures and inspect the work behind it. By Paras Madan | 2026-11-11 Google DeepMind introduced the Multimodal Knowledge Agent benchmark on November 10 to evaluate agents working with visually complex files. Artificial Analysis runs the evaluation. For teams choosing an agent for reports, drawings or spreadsheets, its most useful contribution is a closer look at deliverables rather than another number to repeat. The [MKA announcement](https://blog.google/innovation-and-ai/models-and-research/gemini-models/multimodal-knowledge-agent-benchmark/) describes 200 tasks across ten industry verticals and 83 occupations. Tasks combine input files, a professional brief and an expert-written grading rubric. A separate set of ten public tasks lets developers inspect the format without exposing the private evaluation set. This makes MKA relevant to a concrete buying question: can an agent turn difficult source material into work that someone else can use? Answering that still requires more than choosing the model at the top of the chart. ## What Climb WSR actually measures Artificial Analysis reports Climb WSR as the headline score. WSR means weighted satisfaction rate: the weight of satisfied rubric criteria divided by the total weight of those criteria. “Climb” selects the harder criteria, while a separate Regression category checks baseline competence. The [evaluation methodology and results](https://artificialanalysis.ai/evaluations/mka) explain that scores are averaged across tasks. A percentage here therefore describes rubric satisfaction. It does not tell you the percentage of jobs completed perfectly, nor the probability that your next client report will need no correction. That difference changes how you should read the leaderboard. A report can satisfy many requirements while getting one important conclusion wrong. Conversely, an output might contain sound analysis but lose credit for missing a requested deliverable. Both cases matter, but they suggest different fixes. **Translate the score into its measurement before translating it into a purchase decision.** If your workflow has a few mistakes that would make the entire output unusable, identify those separately instead of assuming the benchmark average already represents your tolerance. ## The harness is part of the result Artificial Analysis runs the models in Stirrup with a 250-turn limit, a Linux sandbox and tools for code, web access and image viewing where supported. The common setup makes comparisons more interpretable, but it also defines the environment being compared. Your production agent may have different tools, permissions, prompts or time limits. It may need to retrieve documents through an internal system, work with incomplete files or produce output in a strict company template. Those differences can change the work even when the underlying model is identical. Use MKA to form a shortlist rather than treating its ordering as a deployment configuration. Before paying for a large evaluation, write down which parts of your environment differ from the benchmark. A team that removes image access from a visual-document workflow, for example, should not expect the published multimodal result to transfer unchanged. ## Inspect a task before comparing models The [public MKA dataset](https://huggingface.co/datasets/google/multimodal-knowledge-agent) provides a useful starting point. Choose an example close to your work, read its instructions and inspect the available input files before looking at the generated output. Otherwise, a polished document can create an impression of correctness before you know what it was supposed to establish. Consider an illustrative internal task: compare a set of product diagrams and prepare a short report identifying inconsistencies. Before asking any agent to do it, decide what counts as evidence. Every reported mismatch should identify the relevant source, distinguish an actual conflict from missing information and explain why the difference matters. Then inspect the deliverable with those requirements in mind. Can you trace a conclusion to the drawing? Does the report acknowledge an unreadable label? Did the agent quietly choose one source when two disagreed? These checks expose whether the output can support a decision, rather than merely resemble a finished report. This is a proposed evaluation workflow, not a hands-on test of the published models. Public examples can help you design the exercise, but your own results require actually running and reviewing it. ## Compare the cost of usable work MKA’s reported model cost excludes judge calls. That is a useful boundary to know before treating the chart as the complete price of an evaluation pipeline. In a business workflow, add the costs the chart cannot know: document preparation, infrastructure, review time, retries and corrections. An inexpensive first draft can become expensive if a specialist must reopen every source to find out whether the conclusions are defensible. For the diagram example, record how long a reviewer needs to reach an accept-or-reject decision. Separate formatting repairs from factual repairs. Moving a caption is different from discovering that a supposed mismatch came from comparing different versions of a drawing. A simple comparison sheet can track the task, required deliverable, model configuration, model spend, review minutes, critical errors and final acceptance. Keep those fields consistent across candidates. The point is to compare the cost of reaching usable work, not to manufacture a single impressive score from incompatible trials. ## Keep your own acceptance gate Google’s benchmark includes visual design alongside factual criteria. Your weighting may be different. A customer-facing deck needs readable presentation, while an internal audit may care far more about traceable evidence and explicit uncertainty. Set a few non-negotiable requirements before reviewing results. For a document comparison, these might include no invented source references, no unsupported claims of agreement and a clear list of unresolved questions. A visually attractive report should not compensate for failing those checks. MKA offers a more relevant way to investigate agents that produce professional files. The useful next step is to move from its shared benchmark to a small set of your own representative tasks, with an acceptance standard that reflects the consequences of being wrong. I’m Paras. I work at the intersection of AI, engineering and distribution. I help AI teams turn technical evaluations and product trade-offs into clear writing that developers and decision-makers can use. [Book a 30-minute call](https://cal.com/paras-madan/30min).