AgentsAutonomy & tool use
OpenAI and Ligo release BioBasicsBench, a 100-task test of AI agents on raw wet-lab data
OpenAI and enzyme-design firm Ligo built 100 tasks from real experimental data, including gels, colony plates and microscopy, to test whether AI agents can read messy evidence, catch errors and withhold unsupported conclusions. GPT-6 Astra earned 66.7% of rubric credit.
From the OpenAI News article of November 6, 2026 and cited sources: BioBasicsBench uses raw lab outputs instead of polished paper figures. [1] [2] [3] [5]
SOURCES[1] Measuring how AI agents handle everyday wet-lab science… openai.com[2] OpenAI Introduces BioBasicsBench to Test AI Agents on… unite.ai[3] LAB-Bench (arXiv) arxiv.org
AT Pulse research deskPublished · Updated · 4 min read · AI-assisted analysisAI-assisted, not human-edited. Methodology Corrections Who owns this site
- Original source
- OpenAI News article, company source
- Brief by
- AT Pulse research desk AI-assisted; publisher and contact on the desk page
- When
- November 6, 2026 article published
- Covered
- Nov 7 edition
- Evidence
- 9 sources across 7 sites
- 1Measuring how AI agents handle everyday wet-lab science…openai.com
- 2OpenAI Introduces BioBasicsBench to Test AI Agents on…unite.ai
- 3LAB-Bench (arXiv)arxiv.org
- 4OpenAI on Xx.com
- 5OpenAI introduces 100-task lab benchmark as AI still…superpowerdaily.com
- 6OpenAI Releases BioBasicsBench for Wet-Lab AI Evaluation…emergent.sh
QuietHow it’s measured
Hacker News · 6 likes
Reach blends public engagement (views, likes, comments, shares, stars) from the platforms where the story or its original source lives: YouTube, Hacker News, GitHub, Substack and similar. A video and the article written from its transcript share one score, taken from the video. Platforms with no data are left out rather than counted as zero, and the age of a story never lowers its score.
BioBasicsBench uses raw lab outputs instead of polished paper figures. These include gel and blot images, colony plates, microscopy stacks, sequencing reads, plate-reader exports and protocol videos, across five domains from protein methods to molecular cloning. Expert rubrics give partial credit for conclusions, measurement accuracy, reasoning and spotting when evidence falls short. This is a shift away from the multiple-choice format of earlier benchmarks such as LAB-Bench. [1] [2] [3] [5]
Across 37 model and reasoning-effort settings, all from OpenAI, GPT-6 Astra at the highest effort earned 66.7% of available credit (95% CI 60.3–73.1). GPT-5.6 Sol earned 43.8% (95% CI 37.6–50.0). These are OpenAI's own results on a benchmark it co-built, with no independent replication. The figure is a share of rubric points, not a pass rate. [2] [5]
The documented failures involve judgment as well as perception. In one cloning task, Astra grouped the gel lanes correctly but then overrode what the controls showed, a mistake that could lead a scientist to purify the wrong DNA. It also overcounted colonies by about 12%. Models noticed brightness differences but discounted an interfering dye, and some accurate observations still led to wrong conclusions. [2] [5]
even the top setting lost a third of available credit and made errors such as overriding gel controls. Implication: lab teams adopting agents should keep human sign-off on controls and counts, and test on their own raw data, including failed runs.
Executive brief
OpenAI's best agent in its own new benchmark of everyday lab data earned about two-thirds of the available credit, and its documented mistakes could send a scientist to purify the wrong DNA. On November 6, 2026, OpenAI and the enzyme-design company Ligo released BioBasicsBench. It has 100 tasks built from raw data out of real experiments: gels, colony plates, microscopy and instrument readouts. The tasks test whether an agent can read messy evidence, spot errors and hold back a conclusion when the data doesn't support one. GPT-6 Astra scored 66.7%, against 43.8% for GPT-5.6 Sol (Unite.AI). OpenAI itself says higher scores do not yet mean reliable lab judgment.
What changed and event timeline
LAB-Bench sets the earlier bar
FutureHouse released more than 2,400 multiple-choice questions on biology research skills, including figure reading, protocol troubleshooting and cloning scenarios ().
Scores and failure cases reported
Trade coverage gives the scores for 37 model and reasoning-effort settings, plus specific failures by GPT-6 Astra (;).
Coverage disagrees on public release
One outlet says the dataset and evaluation scripts are openly available. Another says the article gives no information on public access (;).
OpenAI announces BioBasicsBench
The post describes 100 tasks built with Ligo from real experiments. They test reading raw lab data, finding errors and recognizing when the evidence falls short (;).
Capabilities and access
- Models tested: seven OpenAI models across 37 model and reasoning-effort settings. No non-OpenAI models are named (Superpower Daily).
- Top results, both at "xhigh" effort: GPT-6 Astra 66.7% (95% CI 60.3–73.1) and GPT-5.6 Sol 43.8% (95% CI 37.6–50.0) (Unite.AI).
- Access: unclear. Emergent says the full release is on OpenAI's research portal (Emergent), but no other reviewed source confirms it.
Technical analysis for researchers and developers
- Inputs: gel and blot images, colony plates, microscopy stacks, sequencing reads, plate-reader exports, bench photos and protocol videos.
- Five domains: protein methods; imaging and cytometry; quantification and assays; nucleic acid methods; molecular cloning.
- Scoring: experts wrote rubrics that give partial credit for conclusions, measurement accuracy, reasoning, and recognizing when evidence is insufficient.
Read the full section
- Inputs: gel and blot images, colony plates, microscopy stacks, sequencing reads, plate-reader exports, bench photos and protocol videos. Tasks also include reference material such as ladder charts and plasmid maps (Unite.AI).
- Five domains: protein methods; imaging and cytometry; quantification and assays; nucleic acid methods; molecular cloning.
- Scoring: experts wrote rubrics that give partial credit for conclusions, measurement accuracy, reasoning, and recognizing when evidence is insufficient. The headline figure is "available credit earned," not a pass rate (Superpower Daily).
- Statistics: five attempts per task across all 37 settings gives 18,500 results. Confidence intervals come from resampling whole tasks.
- Review: each task was checked by a Ligo scientist who didn't write it, then by an external PhD-level wet-lab scientist, then in a final internal review.
Claims and evidence
- Claim: curated paper figures don't reflect real lab work, which includes failed runs and ambiguous evidence. This is OpenAI's design rationale (Unite.AI).
- Claim: Astra outperforms Sol. These are OpenAI's own results on a benchmark OpenAI co-built. No independent replication was found.
- Claim: specific failures. These are disclosed by OpenAI itself, which makes them a self-critical data point (Superpower Daily).
Read the full section
- Claim: curated paper figures don't reflect real lab work, which includes failed runs and ambiguous evidence. This is OpenAI's design rationale (Unite.AI).
- Claim: Astra outperforms Sol. These are OpenAI's own results on a benchmark OpenAI co-built. No independent replication was found.
- Claim: specific failures. These are disclosed by OpenAI itself, which makes them a self-critical data point (Superpower Daily).
- Claim: open access. Only one outlet reports it (Emergent). That outlet also says no scores were given, which conflicts with other coverage. Treat it as unconfirmed.
Context and prior work
- LAB-Bench (FutureHouse, 2024): multiple-choice questions, including FigQA, ProtocolQA and cloning scenarios.
- What's different: BioBasicsBench uses open-ended, rubric-graded tasks built on raw, uncurated experimental data instead of multiple-choice questions.
- Recent related work: other 2026 preprints evaluate agents on reasoning about wet-lab protocols (BenchBench-Protocol) and on biosecurity-relevant capabilities (ABC-Bench).
Read the full section
- LAB-Bench (FutureHouse, 2024): multiple-choice questions, including FigQA, ProtocolQA and cloning scenarios. About 80% is public, with 20% held back to detect training contamination (GitHub; arXiv).
- What's different: BioBasicsBench uses open-ended, rubric-graded tasks built on raw, uncurated experimental data instead of multiple-choice questions.
- Recent related work: other 2026 preprints evaluate agents on reasoning about wet-lab protocols (BenchBench-Protocol) and on biosecurity-relevant capabilities (ABC-Bench).
Limitations, safety and contested findings
- Overriding controls: in a cloning task, Astra correctly grouped the gel lanes but then overrode the comparison the controls supported.
- Colony counts: Astra overcounted colonies by about 12% (Unite.AI).
- Dye interference: models noticed brightness differences but discounted the effect of an interfering dye.
Read the full section
- Overriding controls: in a cloning task, Astra correctly grouped the gel lanes but then overrode the comparison the controls supported. That error could lead a scientist to purify the wrong DNA (Superpower Daily).
- Colony counts: Astra overcounted colonies by about 12% (Unite.AI).
- Dye interference: models noticed brightness differences but discounted the effect of an interfering dye.
- Observation vs. conclusion: accurate observations still led to wrong conclusions in some cases.
- Narrow comparison: 100 tasks from one partner company, only OpenAI models tested, and no human expert baseline reported.
- Biosecurity: no reviewed source describes biosecurity controls.
Business and practitioner implications
- Treat scores as partial credit.
- For gel calls, cloning decisions and colony counts, require human sign-off on the controls and quantitative checks.
- Biotech teams evaluating lab agents should build tests from their own raw data, including failed runs, in the style of this benchmark.
Read the full section
- Treat scores as partial credit. 66.7% is a share of rubric points, so it doesn't mean two out of three analyses are trustworthy end to end.
- Keep controls in human hands. For gel calls, cloning decisions and colony counts, require human sign-off on the controls and quantitative checks.
- Test with messy data. Biotech teams evaluating lab agents should build tests from their own raw data, including failed runs, in the style of this benchmark.
- Reasoning effort matters. Scores vary across effort settings, so test cost against accuracy at different levels before buying.
- Confirm access first. Check whether the benchmark is actually public before planning to compare against it.