# Stripe Benchmark Finds Agents Satisfy Every Payment Requirement in 29% of Runs

By Simon Yoon

Canonical URL: https://www.tokenpost.com/news/technology/61625
Published: 2026-11-19T19:33:33.000Z
Updated: 2026-11-19T19:33:33.000Z
Section: Technology

> The open-source evaluation covers 11 reproducible environments, including browser-checked full-stack tasks. Agents averaged about 78% of the required criteria.

Stripe launched a public benchmark on Nov. 19, 2026, to evaluate how well AI coding agents build payment integrations, with only 29% of runs meeting every requirement despite an average score of about 78%. The results show how often agents fall short of completing all parts of a payment task.

The open-source evaluation spans 11 reproducible environments. They include backend tasks, full-stack workflows checked in a browser and specialized exercises on features such as Checkout and subscription billing.

The benchmark compares models, agent harnesses and integration scenarios. Its framework uses a Goose-based harness with Model Context Protocol tools for terminal access, browser automation and documentation lookup.

Across tested model and harness combinations, Stripe’s plugin raised scores by three to eight percentage points, reduced web searches by 96% and cut extraneous test code by about 40%. Claude Opus 4.5 averaged 92% on full-stack tasks, while GPT-5.2 scored 73% on specialized exercises using a best-of-three approach. Agents averaged 63 turns in their best runs.

Stripe first described the evaluation framework on March 2, 2026. The benchmark is open source and hosted on GitHub, where the public can review its methods and results and run the evaluation.


