The company’s models are now better at working with COBOL and other old languages.
Casimiro PT/Shutterstock
OpenAI's models are now better at building and migrating software in old programming languages with a dwindling number of programmers, including COBOL, Fortran and Visual Basic. As the company explains, 40 percent of world's banks still rely on COBOL systems, and migrating to ones that use newer languages would not only cost them millions to hundreds of millions, but also a lot of time. Programmers that work with COBOL are aging out of the workforce, however, so the banks that still use it will have to migrate within the next few years.
The company said its work on improving its models' COBOL capabilities is guided by developers and enterprise customers who submitted feedback. Coding models are apparently struggling when it comes to working with old languages. OpenAI created an AI evaluation suite called MultiLangBench, which asks models to create SQLite-compatible database engines in a specified programming language. It said GPT‑6 Astra had an average test pass rate of 87.4 percent across 20 languages, making it a lot of effective than GPT-5.6 Sol that had a 26.8 percent pass rate. COBOL remains the hardest language to crack, with Astra's pass rate being 72 percent.
OpenAI also evaluated its models' capabilities by having them migrate COBOL programs to ones that use C++, Java, Kotlin, Python or Rust, while also preserving what the programs do. For OpenAI's models to effectively help banking institutions migrate to modern languages, they must be able to preserve COBOL programs' behaviors, such as how it rounds numbers with excess decimal digits and how it handles missing records. In this case, GPT-6.1 Sol achieved the highest accuracy, which reached 97.9 percent, with high reasoning effort. At low reasoning effort, it achieved 86.6 percent accuracy.
In its post, OpenAI admits that a migration is only successful if "all tests pass." Programs that handle banking tasks, after all, would need to be highly accurate. "Behavior-preserving capabilities like these are useful for teams maintaining and modernizing older systems, though applying it to production software still requires expert review and testing against the system's business rules and dependencies," it wrote.