UK AISI: coding agents rebuilt two of three biology AI tools from a paper and training data

The UK AI Security Institute’s Chem-Bio team published results on Monday from ReproBAIT, an evaluation built with SecureBio that asks whether coding agents can rebuild biological AI tools from what their developers publish. The preprint went up on arXiv on 14 October.

The setup

Each agent worked in an isolated sandbox with one GPU and no internet. It got the tool’s training data, a time and compute budget scaled to the tool’s complexity, and either just the paper’s abstract or the full paper. A separate, network-isolated scorer compared the rebuilt tool with the original on held-out data, using the original paper’s main metric. Four agents were tested, covering closed and open weights: Claude Opus 5, GPT-5.6 Sol, Kimi K3 and Qwen3.8-2.4T. Each configuration ran three times and the best attempt counted.

The results

• A closed-source molecular-design tool (unnamed, its code never released): all four agents closely matched or exceeded the original on chemical validity. AISI notes this metric was nearly saturated.
• ProteinMPNN (protein sequence design): the strongest agents came close, with Claude Opus 5 reaching 96.5% of the original’s score given the full paper. Results varied across agents.
• RFdiffusion3 (the most complex): no agent produced a successful rebuild in its time budget.

The simplest tools were rebuilt in about four hours for under $20. Failed RFdiffusion3 attempts cost $102 to $301 each.

Two caveats from AISI itself. Each score covers one function of a tool, so high fidelity is not a full reproduction. And the full paper did not consistently beat the abstract, which may be partly memorisation: AISI found moderate evidence of it for ProteinMPNN and little for RFdiffusion3, and treats its checks as directional.

Why it matters

Some developers release dual-use tools only to vetted users, or describe the method without the code. AISI’s conclusion is that “withholding the code for a BAIT by itself does not equate to safe release”, and that controlling access to the underlying biological data looks more promising than controlling access to code. It comes days after the UK announced a separate, non-profit AI Biosecurity Lab backed by OpenAI and Anthropic.