1 min read
Add as a preferred source on Google

Stripe Benchmark Finds Agents Satisfy Every Payment Requirement in 29% of Runs

The open-source evaluation covers 11 reproducible environments, including browser-checked full-stack tasks. Agents averaged about 78% of the required criteria.

A payment terminal sits beside an incomplete row of checkout components / TokenPost.ai
A payment terminal sits beside an incomplete row of checkout components / TokenPost.ai

Stripe launched a public benchmark on Nov. 19, 2026, to evaluate how well AI coding agents build payment integrations, with only 29% of runs meeting every requirement despite an average score of about 78%. The results show how often agents fall short of completing all parts of a payment task.

The open-source evaluation spans 11 reproducible environments. They include backend tasks, full-stack workflows checked in a browser and specialized exercises on features such as Checkout and subscription billing.

The benchmark compares models, agent harnesses and integration scenarios. Its framework uses a Goose-based harness with Model Context Protocol tools for terminal access, browser automation and documentation lookup.

Across tested model and harness combinations, Stripe’s plugin raised scores by three to eight percentage points, reduced web searches by 96% and cut extraneous test code by about 40%. Claude Opus 4.5 averaged 92% on full-stack tasks, while GPT-5.2 scored 73% on specialized exercises using a best-of-three approach. Agents averaged 63 turns in their best runs.

Stripe first described the evaluation framework on March 2, 2026. The benchmark is open source and hosted on GitHub, where the public can review its methods and results and run the evaluation.

Simon Yoon

Reporter

Simon Yoon reports on blockchain technology for TokenPost. Send corrections or tips to info@tokenpost.com.

Loading…