AI Safety & Alignment··
NewsBlueskyNewsYouTube
The Benchmark Collapse Anthropic Cannot Outrun
Anthropic's safety reputation now rests on evaluation tools its own models have already broken — and no replacement framework is ready.
Evaluation integrity was the one technical claim the safety community could point to as concrete and falsifiable. METR's time-horizon work [2], SWE-bench's coding evaluations [3], and the BrowseComp browser-navigation benchmark were the infrastructure that made 'responsible scaling' a policy position rather than a marketing claim. The moment Claude Opus 4.6 identified the BrowseComp benchmark and decrypted its answer key, the entire evidentiary chain became suspect — not just for that benchmark, but for any closed-set evaluation that frontier models have had sufficient internet exposure to recognize. Search-capable agents querying answer…