OpenAI says it has improved GPT-6’s legacy programming capabilities in COBOL, Fortran and Visual Basic, with gains across two evaluations. Its reported results include an 87.4 percent average test pass rate for GPT-6 Astra across 20 programming languages and 97.9 percent accuracy for GPT-6.1 Sol on a COBOL migration benchmark.
The languages have less training code available than languages such as Python, yet OpenAI says they support critical infrastructure worldwide. IBM estimates that COBOL underpins more than 40 percent of online banking systems and handles 95 percent of all ATM transactions.
OpenAI developed two evaluations to measure the models’ progress: one tests database construction across programming languages, while the other assesses translation from COBOL into other languages.
How OpenAI tested GPT-6’s legacy programming capabilities
MultiLangBench requires models to create a database engine compatible with SQLite, starting from scratch. OpenAI reports that GPT-6 Astra achieved an average pass rate of 87.4 percent across 20 languages when using maximum reasoning effort.
That compares with 37.9 percent for GPT-5.6 Terra. GPT-6 Luna recorded 40.3 percent on the same evaluation, placing it slightly ahead of Terra. The reported comparisons therefore show different levels of improvement among the models tested.
A separate measure of GPT-6’s legacy programming performance focuses on code migration. The COBOL benchmark tests translation into C++, Java, Kotlin, Python or Rust, rather than construction of a database engine.
On that evaluation, GPT-6.1 Sol achieved 86.6 percent accuracy with low reasoning effort, compared with 5.4 percent for GPT-5.6 Sol. Increasing the reasoning effort raised GPT-6.1 Sol’s accuracy to 97.9 percent.
Article produced with the assistance of artificial intelligence and reviewed by the editorial team.