2 min read
Add as a preferred source on Google

VibePrevisBench Measures Coding Agents on 242 Blender Scene Tasks

Tasks use scripts, photos, scene continuations and reference videos across seven scene domains and 33 locations.

A miniature camera faces a carefully staged model set / TokenPost.ai
A miniature camera faces a carefully staged model set / TokenPost.ai

VibePrevisBench, introduced Nov. 17, gives coding agents 242 Blender scene-building tasks to test whether they can turn varied inputs into executable 4D scenes and produce visually legible proxy videos.

The tasks use four input types: 94 start with scripts, 39 with photos, 23 continue existing scenes and 86 use reference videos. Together, they cover seven scene domains and 33 locations. The range is designed to test how agents handle different briefs while building scenes that can run in Blender.

The benchmark has two evaluation checks. One runs an agent’s code again and compares the resulting scene with hidden reference facts across nine areas, including spatial layout, actions, cameras and physics. The other uses a vision-language model to assess whether those facts are visible in the proxy video. The two checks cover 84,406 scorable facts, measuring both what the scene contains and what the video shows.

Results from 11 coding agents showed that basic camera work and staging were generally handled well. Agents differed in their understanding of spatial relationships and camera motion, and complex requests proved difficult. They sometimes identified problems in their own renders but failed to fix them.

The project released its code and data Nov. 11. Its site also presents evaluation scores, a leaderboard users can explore, and 14 case studies. The Blender-focused work joins other efforts to evaluate generative world models, including WorldFoundry’s framework spanning video, 3D and 4D reconstruction, embodied action and interactive systems.

Simon Yoon

Reporter

Simon Yoon reports on blockchain technology for TokenPost. Send corrections or tips to info@tokenpost.com.

Loading…