OpenAI's GPT-6.1 Sol Scores 86.6% on COBOL Migration Test

OpenAI released Oct. 19 test results showing large gains for its newer coding models on COBOL migration and other legacy-language tasks, which matter to companies maintaining software used for financial records.

In MultiLangBench, models had to create a database engine compatible with SQLite in each of 20 programming languages. At maximum reasoning effort, GPT-6 Astra passed an average of 87.4% of tests, compared with 26.8% for GPT-5.6 Sol.

The gap extended to three languages used in the evaluation. The COBOL pass rates were 72.1% for GPT-6 Astra and 10.9% for GPT-5.6 Sol. The respective scores were 83.5% and 21.4% for Fortran, and 89.4% and 27.5% for Visual Basic.

A separate evaluation tested whether models could translate COBOL programs into other languages while preserving their behavior. GPT-6.1 Sol passed 86.6% of those tasks at low reasoning effort, versus 5.4% for GPT-5.6 Sol. The newer model's estimated API cost was 39% lower in that test. At higher reasoning effort, its pass rate reached 97.9%.

To pass a migration task, the replacement program had to produce output identical to the original on every hidden test. That measures a stricter result than whether the translated code runs: a program can execute without errors yet return a different answer.

Such differences matter in software that handles financial records. Changes to rounding or the treatment of missing records can alter a result even when the replacement appears to work. The original program may also depend on business rules and other systems that a coding test does not capture.

The 39% cost comparison covers estimated API use for the COBOL evaluation. It does not measure the total expense of moving a production system. A live deployment requires expert checks of the software's business rules, dependencies and behavior.

IBM offers a tool that converts COBOL to Java and generates tests to compare the programs. Anthropic has also outlined a way to analyze and migrate COBOL systems with Claude Code. OpenAI has released the MultiLangBench setup and tests for the defined coding tasks.