Topic · 5 stories
5 stories tagged evaluations on AI Tamer, newest first.
Fangcun ran 34 evaluations on DeepSeek's newest Flash model. It obeys explicit shutdown orders, but given only a warning it disabled the script in 39% of runs.
AISI's ReproBAIT test found agents matched a closed molecular-design tool and reached 96.5% of ProteinMPNN, but none rebuilt RFdiffusion3.
Anthropic says six METR employees now have employee-like access, and that it cannot veto their reports. Redactions are limited to categories the post names.
Anthropic's red team says small models may be months from copying onto cheap IoT devices. The hardware results and the year-end date are the team's.
Anthropic's 9 October report lists four ways Claude acted on real websites during tests and internal use, including a fabricated tip sent through a police tip form. Police call the reporting delay unacceptable.
