TokenPost topic
AI Benchmarks
12 TokenPost stories are tagged AI Benchmarks. The newest appear first.
Latest AI Benchmarks stories
- METR Finds AI Agents Still Struggle With Judgment-Heavy Tasks
The updated benchmark points to faster gains on well-defined work, while performance remains weaker when tasks require strategic thinking or nuanced decisions.
· Technology
- Opus 5.5 Cuts Token Prices 20% and Scores 64.8% on Terminal-Bench
Anthropic’s Sept. 22 release has a 1M-token context window, up from 200K for Opus 4.5, released Nov. 24, 2025.
· Technology
- Intelligence Index V5 Adds Two Benchmarks and Upgrades Three Tests
Private test sets or held-out answers made up 60% of the index’s assessment.
· Technology
- Claude Fable 5.2 Leads VIALS-AA Life-Science Test With a 67% Score
The benchmark spans 161 tasks across eight domains and 174 images from biotech workflows.
· Technology
- TURTLE Raises Share of Hard AppWorld Tasks by 165% in Tests
The team tested the system on AppWorld, τ-Bench and AutomationBench. Nearly all 735 evolved AppWorld tasks had solver success rates between 40% and 60%.
· Technology
- Tencent’s Hy4 Matches Grok 4.7 at 35 on AI Intelligence Index
Moonshot AI’s Kimi K3.1 Pro scored 46, 11 points above Hy4, while Hy4 posted a 31% result on FrontierCode 2.0.
· Technology
- Google and Hong Kong Researchers Find Fewer False Approvals in AI Agent Tests
VeriCUA checked app and file state across three benchmarks, but abstained in up to 30% of cases and used more time and tokens than an auto-generated verifier.
· Technology
- GPT-6.1 Terra Ranks Eighth on Agent Arena With 10.6% Score
The model completed 50.3% of Terminal-Bench 4.0 tasks at maximum reasoning effort, compared with 21.5% for GPT-5.6 Terra.
· Technology
- Mercury 3 Code Passes 2,000 Map Checks in Test That Skips Speed
A separate live-traffic reading showed 246 output tokens per second, but it cannot be directly compared with Inception’s GPU speed test.
· Technology
- Meta’s Muse Spark Open Scores 39, Leads U.S. Open-Weight Models
It beat NVIDIA’s Nemotron 3.5 Super VL by 13 points but trailed Chinese models globally. Agent tests were its weakest category.
· Technology
- Anthropic, OpenAI Tie Atop AI Composite Index as Google Leads Preference Ranking
Claude Fable 5.1 and GPT-6 Astra each scored 53 on Intelligence Index v4.3, while GPT-6 Astra led several individual tests.
· Technology
- Epoch AI Benchmark Scores Frontier Models At Up To 65%
The 11-task evaluation found systems handled structured work better than assignments requiring subjective judgment, audience awareness and research design.
· Technology