Oct 15 : Anthropic has given nonprofit METR employee-level access to its systems and is in talks with others including Hugging Face, as AI companies pitch external evaluators as a guardrail to develop the technology safely without slowing its development.
The developer of Claude models is also working with biosecurity nonprofit SecureBio and Accenture's AI business Faculty as evaluators, it said in a blog post on Thursday.
Reuters exclusively reported earlier this week that Anthropic and OpenAI had sought significant control over what outside reviewers could publish, concerned that the findings could leak trade secrets.
A person familiar with the matter told Reuters that talks with Anthropic had evolved recently, with the company becoming more open in negotiations.
A key sticking point between evaluators and AI labs is that information the companies might consider crucial intellectual property can also be relevant to assessing a model's risks, leading to disputes and redactions, a separate person familiar with past dealings with the labs told Reuters.
Experts have also said the independence of evaluators is a concern, as Anthropic and OpenAI choose who gets to carry out the reviews.
The outcome of the talks will help determine how the world scrutinizes a technology that some of its creators warn may be advancing beyond humanity's ability to manage.
THE METR TIE-UP
Anthropic said on Thursday six employees of METR — or Model Evaluation and Threat Research — have company hardware, office space and access to internal systems roughly equivalent to its risk assessment teams. However, customer data, legally privileged information and model architecture will remain off limits.
The company said it cannot veto METR's findings on risks, incidents and its practices, but can require narrowly defined redactions. METR can also publicly question those redactions without revealing the withheld information.
METR did not immediately respond to a request for comment. The nonprofit had proposed on Wednesday that developers allow private staff interviews, timely publication of assessments and transparency about access limits and redactions.
Anthropic said its other evaluator partnerships are at an early stage, with more limited access and publication rights.
The company is developing three tiers of access. Evaluators at the first level will conduct narrowly scoped investigations and need approval to publish. Those at the second will receive more information and can publish without approval, subject to redactions evaluators can publicly challenge.
The third, used by METR, provides employee-like access with similar publication rights to the second tier.
SecureBio is in the first tier and is reviewing the chemical and biological weapons chapter of Anthropic's August risk report, having published a pilot review of the previous report. Anthropic said it would consider expanding SecureBio's access and publication rights after the review.
Anthropic also announced last month that it is working on a tie-up with Faculty covering model evaluations, adversarial testing, assessments of whether models behave as intended, among others.