# OmniDR-Bench: How Video Deep Research Agents Are Evaluated

A new benchmark separates polished video research reports from the evidence needed to trust them.

By Paras Madan | 2026-10-23


Researchers from Peking University and Alibaba introduced OmniDR-Bench on October 22 to evaluate video research agents. Tested systems handled presentation more reliably than content and citations.

A report may read well while leaving the reader unable to check its claims.

## What OmniDR-Bench measures

The [released benchmark](https://github.com/Multimedia-Semantic-Analytics-Lab/OmniDR-Bench) contains 50 English and Chinese tasks requiring video evidence, external research and HTML reports with citations and timestamps.

Evaluation uses 6,414 binary criteria, assessed through multimodal judging, deterministic checks and browser probes.

Consider a video demonstrating an engineering process. Repeating the narration is different from checking whether the visible setup supports the explanation. A useful research report would also help a reader find the relevant moment and inspect the outside evidence. That is the kind of distinction worth testing before presenting a summary as research.

## What the benchmark results show

Across [18 model-and-agent configurations](https://arxiv.org/html/2610.31158v1), the authors report average scores of 81.8 for presentation, 45.4 for content and 47.8 for citations.

The small benchmark uses a fixed layout and reference-skill-influenced evaluation directions. **Its findings should inform product-specific tests, not replace them.**

## What developers can actually use

The evaluation code is released under MIT. Dataset texts and arrangement use CC BY-NC 4.0, and source videos and audio are not distributed with it. Developers need to check the dataset notice and obtain source material under the applicable terms rather than assume everything in the workflow has the code's license.

The evaluator requires Python, Node.js and a text-and-vision judging endpoint. It supports a validation-only mode for checking report structure before model judging, which makes it possible to separate broken artifacts from weak research.

**A working citation link and a supported claim are different checks.** Product teams can adopt that distinction even if they do not use the full benchmark. For example, a report review can ask whether the linked source actually supports the nearby sentence, not merely whether the link opens.

The next useful improvement in video research may be less visible than a better report layout: making it easier to trace each conclusion back to what the agent saw and verified.

I'm Paras. I work at the intersection of AI, engineering and distribution.

I help technical founders explain AI products and research clearly, so developers can understand what changed and why it matters.

[Book a 30-minute call](https://cal.com/paras-madan/30min).