1 min read
Add as a preferred source on Google

Larger AI Models Complete More Steps in Long-Horizon Execution Tests

Earlier mistakes made models more prone to further errors, while thinking mitigated that effect and enabled longer tasks in a single turn.

Blank cards form a long path beside several fallen pieces / TokenPost.ai
Blank cards form a long path beside several fallen pieces / TokenPost.ai

Larger models completed more turns correctly than smaller models, even when smaller models had near-perfect single-turn accuracy.

Researchers supplied the knowledge and plan needed for each assigned task, then measured whether models carried out the steps. Per-step accuracy declined as tasks gained steps.

Self-conditioning occurred when a model became more likely to make further errors after its conversation history included its own earlier mistakes. Model size alone did not eliminate the effect. Thinking mitigated self-conditioning and enabled models to complete longer tasks in a single turn.

The tests covered assigned task execution, not multi-model agent systems.

The tests did not measure whether different model combinations are cost-effective.

Simon Yoon

Reporter

Simon Yoon reports on blockchain technology for TokenPost. Send corrections or tips to info@tokenpost.com.

Loading…