2 min read
Add as a preferred source on Google

FrontierCS 2 Finds No AI Agent Passed 16 Research-Coding Tasks

The 48-task benchmark spans 13 computer science domains. Claude Opus 5.5 led the initial evaluation with a 58% pass rate.

A small research robot faces a precision test fixture / TokenPost.ai
A small research robot faces a precision test fixture / TokenPost.ai

Researchers led by UC Berkeley released FrontierCS 2, a computer science benchmark measuring whether AI agents can improve software tied to academic research. In its initial evaluation, no agent passed any of 16 tasks, while Claude Opus 5.5 recorded a 58% pass rate overall.

The benchmark contains 48 open-ended tasks across 13 computer science domains, including machine learning, databases and robotics. Each challenge starts with a research codebase, and an agent has five hours to rebuild and improve its implementation. To pass, the revised software must outperform the original implementation.

Evaluators measure performance on multiple criteria and include workloads that agents cannot see in advance. The project also uses a capability index intended to compare the typical difficulty of its tasks with the abilities of the AI systems being tested.

November updates addressed concerns about reward hacking.

The first evaluation covered 12 agent-model pairings. A later leaderboard addition included Claude Sonnet 5.5, which recorded a 50% pass rate. The results leave 16 challenges that no tested agent passed, even as the strongest systems cleared more than half of the task set.

FrontierCS 2 builds on an earlier benchmark introduced at ICML 2026. That version focused on algorithm challenges selected by experts, while the newer benchmark uses codebases connected to academic research. The shift tests agents on existing research software rather than only on curated algorithm problems.

About 60 researchers contributed to the project, led by UC Berkeley Ph.D. student Qiuyang Mang with collaborators from institutions including Princeton. The paper appeared on arXiv Oct. 21, 2026. The team also has a private pool of more than 100 additional tasks that it plans to announce in the future.

Simon Yoon

Reporter

Simon Yoon reports on blockchain technology for TokenPost. Send corrections or tips to info@tokenpost.com.

Loading…