CodingSoftware & developer tools
OpenAI introduces COBOL-migration and 20-language coding evaluations, reporting GPT-6.1 Sol at 86.6% on COBOL translation
OpenAI introduced two coding evaluations for low-resource languages. On a COBOL-migration test, GPT-6.1 Sol passed 86.6% at low effort, compared with 5.4% for its predecessor. On MultiLangBench, models build a SQLite-compatible engine in 20 languages.
From the OpenAI News article of October 19, 2026 and cited sources: On OpenAI's COBOL-migration test, models translate COBOL programs into C++, Java, Kotlin, Python or Rust, and hidden tests require byte-for-byte identical output. [2] [3] [7] [9]
SOURCES[2] FourWeekMBA: GPT-6.1 Sol scores 86.6% fourweekmba.com[3] KuCoin News: OpenAI launches new benchmarks kucoin.com[7] RuntimeWire: OpenAI reports benchmark gains for COBOL and… runtimewire.com
AT Pulse research deskPublished · Updated · 4 min read · AI-assisted analysisAI-assisted, not human-edited. Methodology Corrections Who owns this site
- What happened
- On OpenAI's COBOL-migration test, models translate COBOL programs into C++, Java, Kotlin, Python or Rust, and hidden tests require byte-for-byte identical output. [2] [3] [7] [9]
- Original source
- OpenAI News article, company source
- Brief by
- AT Pulse research desk AI-assisted; publisher and contact on the desk page
- When
- October 19, 2026 article published
- Covered
- Oct 20 edition
- Evidence
- 11 sources across 10 sites
- 1OpenAI: Advancing coding in low-resource programming…openai.com
- 2FourWeekMBA: GPT-6.1 Sol scores 86.6%fourweekmba.com
- 3KuCoin News: OpenAI launches new benchmarkskucoin.com
- 4Slashdot/CNBC: IBM shares crater after Anthropic COBOL posttech.slashdot.org
- 5Gigazine: IBM stock drops 13%gigazine.net
- 6arXiv: COBOL-Coderarxiv.org
QuietHow it’s measured
Hacker News · 8 likes
Reach blends public engagement (views, likes, comments, shares, stars) from the platforms where the story or its original source lives: YouTube, Hacker News, GitHub, Substack and similar. A video and the article written from its transcript share one score, taken from the video. Platforms with no data are left out rather than counted as zero, and the age of a story never lowers its score.
On OpenAI's COBOL-migration test, models translate COBOL programs into C++, Java, Kotlin, Python or Rust, and hidden tests require byte-for-byte identical output. GPT-6.1 Sol passed 86.6% at low reasoning effort and 97.9% at high effort, up from 5.4% for GPT-5.6 Sol. It also reportedly costs 39% less. [2] [3] [7] [9]
MultiLangBench asks a model to build a SQLite-compatible database engine from scratch in 20 languages, from Fortran and OCaml to Zig and Lean. It scores the share of test cases passed, not whether the engine works end to end. GPT-6 Astra averaged 87.4%, up from 26.8%. COBOL remained its weakest language at 72.1%, and the lower-cost GPT-6 Luna scored 17.1% on COBOL. [3] [8] [9]
The results suggest general frontier models may be closing the gap with specialist approaches, such as the fine-tuned COBOL-Coder model. However, the two use different benchmarks, so their scores are not directly comparable. The announcement follows Anthropic's February push on COBOL modernization, which coincided with a roughly 13% drop in IBM shares. [4] [5] [6] [11]
The evidence shows large reported gains on legacy languages that banks and governments still run.
Read the full assessment
The implication is that pilot migrations look worthwhile where the outputs of existing programs can serve as test oracles, provided expert review and regression testing stay in place.
Executive brief
OpenAI reports that GPT-6.1 Sol passed 86.6% of its internal COBOL-migration evaluation at low reasoning effort. The previous model, GPT-5.6 Sol, scored 5.4%. GPT-6.1 Sol also reportedly costs 39% less (FourWeekMBA; KuCoin News). The October 19, 2026 post also introduces MultiLangBench, which asks a model to build a SQLite-compatible database engine in 20 languages. OpenAI itself built and ran both evaluations. All coverage found traces back to OpenAI's post, and no independent replication was found. OpenAI says production migrations still need expert review.
What changed and event timeline
Anthropic moves first on COBOL
Anthropic published guidance on using Claude Code to modernize COBOL. IBM shares fell about 13% that day, which framed legacy migration as a market-moving AI use case (;).
Academic COBOL specialist model
The COBOL-Coder paper fine-tuned a model on compiler-validated COBOL data. It reported large gains over GPT-4o on COBOLEval and introduced a COBOL↔Java translation benchmark ().
OpenAI announces low-resource language evaluations
The post introduces MultiLangBench and a COBOL-migration evaluation, with results for GPT-6 Astra, GPT-6.1 Sol and GPT-6 Luna. Coverage describes it as a capability update, not a product launch (;).
Press frames it around banks
Coverage cites a figure that 40% of the world's banks rely on COBOL, and repeats OpenAI's note that production use requires expert review ().
Capabilities and access
- GPT-6 Astra (flagship): reported 87.4% average on MultiLangBench across 20 languages (Engadget). RuntimeWire labels this result as coming from a "batch" setting (RuntimeWire).
- GPT-6.1 Sol: reported 86.6% (low effort) and 97.9% (high effort) on COBOL migration.
- GPT-6 Luna (lower cost): reported 40.3% on MultiLangBench at about $0.77 per sample, against $3.48 for GPT-5.6 Sol (KuCoin News).
Read the full section
- GPT-6 Astra (flagship): reported 87.4% average on MultiLangBench across 20 languages (Engadget). RuntimeWire labels this result as coming from a "batch" setting (RuntimeWire).
- GPT-6.1 Sol: reported 86.6% (low effort) and 97.9% (high effort) on COBOL migration.
- GPT-6 Luna (lower cost): reported 40.3% on MultiLangBench at about $0.77 per sample, against $3.48 for GPT-5.6 Sol (KuCoin News).
- RuntimeWire reports the tests were run through OpenAI's API. No new product or access tier was announced.
Technical analysis for researchers and developers
- MultiLangBench task: build a SQLite-compatible engine from scratch. The score is the share of test cases passed, not whether a complete engine works (CellCog).
- COBOL migration task: translate COBOL programs into C++, Java, Kotlin, Python or Rust. Hidden tests require byte-for-byte identical output (CellCog; RuntimeWire).
- Reproducibility: CellCog says the MultiLangBench code is on GitHub but the COBOL test is not public.
Read the full section
- MultiLangBench task: build a SQLite-compatible engine from scratch. The 20 languages are C, C#, C++, COBOL, Common Lisp, Fortran, Go, Java, JavaScript, Lean, OCaml, PHP, PowerShell, Python, Ruby, Rust, Scala, Swift, Visual Basic and Zig. The score is the share of test cases passed, not whether a complete engine works (CellCog).
- COBOL migration task: translate COBOL programs into C++, Java, Kotlin, Python or Rust. Hidden tests require byte-for-byte identical output (CellCog; RuntimeWire).
- Reproducibility: CellCog says the MultiLangBench code is on GitHub but the COBOL test is not public. No repository could be found to confirm the GitHub release.
Claims and evidence
All figures are vendor-reported, from evaluations OpenAI designed itself. No independent corroboration exists, and there are no head-to-head comparisons with other vendors' models (RuntimeWire).
Read the full section
| Claim | Evidence |
| COBOL migration rose from 5.4% to 86.6%, and to 97.9% at high effort | FourWeekMBA |
| MultiLangBench average rose from 26.8% to 87.4% | Engadget |
| Per-language gains: COBOL 10.9→72.1%, Fortran 21.4→83.5%, OCaml 25.6→85.8%, Visual Basic 27.5→89.4% | KuCoin News |
| 19 of 20 languages scored above 83% | KuCoin News |
All figures are vendor-reported, from evaluations OpenAI designed itself. No independent corroboration exists, and there are no head-to-head comparisons with other vendors' models (RuntimeWire).
Context and prior work
- Research on cross-lingual transfer has long found that code models perform worse on languages with little training data (arXiv 2310.16937).
- COBOL-Coder took the specialist route: curated domain data plus fine-tuning. It reported 73.95% compilation success against 41.8% for GPT-4o (arXiv).
- Anthropic's February COBOL push set the competitive frame (AI News).
Read the full section
- Research on cross-lingual transfer has long found that code models perform worse on languages with little training data (arXiv 2310.16937).
- COBOL-Coder took the specialist route: curated domain data plus fine-tuning. It reported 73.95% compilation success against 41.8% for GPT-4o (arXiv). OpenAI's results suggest general frontier models may now close much of that gap, but the benchmarks differ, so the numbers are not directly comparable.
- Anthropic's February COBOL push set the competitive frame (AI News).
Limitations, safety and contested findings
- COBOL is still the hardest language on MultiLangBench: 72.1% even for Astra, and 17.1% for Luna (KuCoin News).
- Coverage stresses that the pass rates do not mean the outputs are production-ready.
- Cost data for Astra on MultiLangBench is incomplete (CellCog).
Read the full section
- COBOL is still the hardest language on MultiLangBench: 72.1% even for Astra, and 17.1% for Luna (KuCoin News).
- Coverage stresses that the pass rates do not mean the outputs are production-ready. Real legacy systems contain undocumented business rules and dependencies that the benchmarks do not cover (Engadget).
- Cost data for Astra on MultiLangBench is incomplete (CellCog).
- Sources disagree slightly on the date: Engadget lists October 20, while other outlets give October 19.
Business and practitioner implications
- Banks, insurers and government teams: the results justify piloting migrations where existing outputs can serve as a test oracle.
- System integrators (IBM, Accenture, Cognizant): pressure on the analysis and translation work they sell keeps growing (AI News).
- Developers: the reported cost gap between Luna and Astra suggests sending low-resource-language work to the stronger tier.
Read the full section
- Banks, insurers and government teams: the results justify piloting migrations where existing outputs can serve as a test oracle. That is the setup the COBOL evaluation rewards. Keep human review and regression testing against business rules.
- System integrators (IBM, Accenture, Cognizant): pressure on the analysis and translation work they sell keeps growing (AI News).
- Developers: the reported cost gap between Luna and Astra suggests sending low-resource-language work to the stronger tier.