BioBasicsBench tests whether AI can read messy laboratory evidence
OpenAI and Ligo have introduced 100 tasks built from routine wet-lab data. The benchmark rewards checking controls and recognizing when the evidence does not support a conclusion.
Read the story
Published on November 6, OpenAI’s report describes tasks covering protein methods, imaging and cytometry, assays, nucleic acid methods and molecular cloning. Inputs include raw gel images, microscopy, sequencing files and instrument exports rather than only polished figures. Scientists supplied grading criteria and reviewed the tasks. OpenAI reports that GPT-6 Astra at xhigh reasoning earned 66.7% of the available rubric credit, compared with 43.8% for GPT-5.6 Sol. Those figures are the developer’s benchmark results, not an independent IDAKIM Digital evaluation, and mean reward is not the percentage of experiments completed correctly. The report also documents failures: assumptions overriding controls, contradictory observations being discounted and colony-counting errors. It explicitly cautions that higher benchmark scores do not yet establish reliable laboratory judgement.
Why it matters
For researchers evaluating an assistant, the practical lesson is to test control interpretation, uncertainty and raw-data handling alongside final answers. Keep a scientist responsible for experimental conclusions. A benchmark score can help choose evaluation cases; it does not justify unattended laboratory decisions or establish performance on your own samples.