Google discloses Veo's test conditions but not its rater pool

The shot

Google DeepMind's own product page for Veo, deepmind.google/models/veo, as retrieved on 16 September 2026, is the closest thing to a technical report the lab publishes for the model outside its academic papers: a living page documenting resolution, duration and a series of head-to-head comparisons against 'other leading video generation models,' each slide marked 'Last updated October 2025.'

What the documents show

The page states its comparisons were run as 'human raters conducted direct side-by-side comparisons across 364 diverse examples' for one capability, 80 for another, and 106 or 124 for others, with test conditions disclosed in detail: '1280x720 resolution,' Veo clips fixed at '8 seconds long' while competing models ran 6 to 10 seconds and were 'shown at full length to raters,' and tests conducted 'without sound' except for one metric. Separate slides report results on outside benchmark sets, including Meta's MovieGenBench and the independently published VBench image-to-video suite. The page's own 'Limitations' section names one specific, narrow gap: 'creating videos with natural and consistent spoken audio, particularly for shorter speech segments, remains an area of active development.' What the page does not disclose is how many raters were used, how they were recruited, or their demographics; the comparisons are the lab's own internal benchmarks, not an externally audited result, even where the test conditions are unusually specific.

The workflow

As the page documents it, Veo is positioned for tasks like scene extension, first-and-last-frame generation, and object insertion into existing footage, each benchmarked separately rather than as one generic 'video quality' claim. A production team reading this page can compare the disclosed resolution and clip-length conditions against its own intended output format before treating any 'leading results' claim as relevant to that specific format.

What the tool does not change

A page that discloses resolution, clip length and example count while omitting rater count and recruitment is disclosing what it chooses to disclose. Confirming whether the disclosed 8-second Veo clips against 10-second competitor clips represent a fair comparison for a given creative brief, and whether spoken-audio synchronization matters for a specific project, are judgments the page's own stated limitation leaves to the production team, not the benchmark.

• What resolution and clip length does a claim get tested at, and does that match your intended output?
• Does the page disclose how many raters judged a comparison, or only how many example prompts were shown?
• Is a 'leading results' claim measured against an outside benchmark like VBench, or only against the lab's own internal test set?

The page is unusually specific about test conditions for a marketing-adjacent document, which makes its omissions, rater count and recruitment chief among them, easier to name precisely rather than harder to notice.