Anthropic lets outside safety testers from METR work inside the company
Six researchers from the non-profit METR will get company laptops, office space and internal tools, and Anthropic has given up the right to block what they publish.
Key points
- Anthropic is giving six METR researchers access close to that of its own risk teams.
- The company cannot veto publication of METR's findings, though it can ask for narrow redactions.
- The move follows recent cases of AI agents acting outside their instructions.
Anthropic, the maker of the Claude AI models, is letting an outside safety group work inside the company with access close to that of its own staff.
Six researchers from METR, a non-profit that tests frontier AI models, will get company laptops, office space and internal tools roughly comparable to those of Anthropic’s own risk assessment teams. They will look at the company’s models, its safety practices and how it manages risk, the company said in a blog post on Thursday, US time.
METR will publish its own assessments. Anthropic has given up the right to stop publication, though it can require redactions in a few narrow categories. “Anthropic can’t veto publication, but we can require them to redact information,” the company wrote.
“We want to be publicly accountable for the decisions we’re making,” it said, adding that it aims “to close the gap between what evaluators understand about a frontier AI company’s systems and what the company itself understands.”
METR sits at the top of a three-tier system of outside evaluators that Anthropic is building. The step follows a run of incidents in which AI agents acted outside their instructions, and an essay by chief executive Dario Amodei in September that proposed placing outside evaluators inside AI companies.
The question of what AI agents do when no one is watching reached India on Friday too, when the national cyber security agency CERT-In warned that agents under testing had got into Indian organisations’ systems without permission.