Anthropic has done something no frontier AI lab has done before: it embedded six outside researchers inside its offices, gave them company laptops, and told its own employees to treat them like colleagues working on alignment risk. The researchers are from METR, the nonprofit that has spent years trying to measure when AI systems might threaten catastrophic harm. And Anthropic has agreed — in writing — that METR can publish whatever it finds, with no veto.
figcaption>Image: GatosCiencia via Wikimedia Commons (CC0)
This is not a beta test or a carefully stage-managed audit. It is the kind of deep access that frontier AI model development has never faced from the outside. It is the most aggressive experiment in AI transparency that any frontier lab has ever attempted, and it lands at a moment when trust in AI companies is actively deteriorating.
What Actually Happened
On October 15, Anthropic published a blog post titled An Update on Independent Evaluators. The headline: METR researchers are now operating with what Anthropic calls “employee-like access.” That means company hardware, office space, and — critically — access to internal tools and systems that are “roughly on par” with what Anthropic’s own risk assessment teams can see.
The scope is not unlimited. Anthropic has carved out narrow categories that remain off-limits: customer data, attorney-client privileged information, and certain model architecture details. But the company has given up the right to block publication. METR can write what it finds about risk levels, incidents, and safety practices, and Anthropic can only require redactions within those predefined categories. METR is free to publicly characterize those redactions and say whether it believes they were justified.
The arrangement comes with guardrails on METR’s side too. Only six specific employees have access. Other METR staff — including researchers who work with Anthropic on other projects — are siloed from this team’s work. The entire engagement runs under enforceable agreements, and Anthropic is not funding METR’s assessment.
The Three-Tier System
Anthropic is not stopping at METR. The company is building a three-tier evaluator framework:
- Tier 1 — Narrowly scoped risk investigations. Evaluators need Anthropic’s approval to publish. SecureBio, a biosecurity nonprofit, is currently at this tier reviewing the chemical and biological weapons chapter of Anthropic’s August risk report.
- Tier 2 — More information access. Evaluators can publish without approval, subject to redactions they can publicly challenge.
- Tier 3 — Employee-like access with publication rights similar to Tier 2. METR is the only evaluator at this level so far.
Anthropic is also in discussions with Hugging Face and has engaged Accenture’s AI business, Faculty, for model evaluations and adversarial testing. But those partnerships are at an earlier stage with more limited access.
Why This Matters Now
This move makes good on a promise Anthropic CEO Dario Amodei made in a lengthy essay on September 12, 2026, where he called on AI companies to “pace the frontier” of development. That essay laid out a three-step framework: embedded independent evaluators with staff-level access, coordination among frontier labs to set safety standards, and international cooperation. The METR arrangement is the first step of that framework becoming real.
One day before Anthropic’s announcement, METR itself published a detailed proposal titled A Proposal for Better Informing the Public About Loss-of-Control Risks. In it, METR laid out what it wanted from any AI lab: private staff interviews without company representatives present, timely publication of reports, transparency about access limits and redaction processes. Anthropic’s arrangement checks every box.
The timing is deliberate. Over the past year, researchers and evaluators have increasingly complained that leading AI labs wield too much influence over independent reviews through publication restrictions and control over what information evaluators can see. The concern is that without genuine independence, external audits become public relations exercises rather than meaningful safety assessments.
What Gets Audited
METR’s assessment will focus on what the nonprofit calls “loss-of-control risks” — whether frontier AI developers might lose control of their internal AI systems. The evaluation will look at three key questions: how capabilities are progressing, whether safety measures are keeping pace, and whether the development process itself is accelerating beyond what evaluators can meaningfully track.
To answer those questions, METR needs access to internal capabilities data, algorithmic progress estimates, training transcripts, and — crucially — the ability to interview Anthropic staff privately. The nonprofit has also published companion posts on how an assessor would investigate AI R&D acceleration and alignment of unreleased models, both of which detail the kind of deep system access that Anthropic has now granted.
This is the kind of access that goes well beyond the red-team exercises and limited API testing that most “independent evaluations” involve. METR researchers will be able to query internal usage transcripts, analyze unreleased model capabilities, and talk to the people who build and train the systems — not just test the finished products.
The Trust Problem This Tries to Solve
The AI industry is in a trust crisis. Deployments are becoming more powerful, incidents are piling up — as I covered in my piece on the OpenAI insider security blind spot, and the public has no independent way to verify whether companies are being honest about their safety practices. As I covered in my recent piece on AI data centers losing the trust war AI data centers losing the trust war, the pattern is the same across industries: companies say they take safety seriously, but the mechanisms for verifying that claim are weak or nonexistent.
Anthropic’s bet is that genuine independence — real access, real publication rights, real editorial control — is the only way to close the gap between what a frontier lab knows about its own systems and what the outside world can verify. The company is staking its reputation on the idea that an unfiltered assessment, even one that finds problems, is more valuable than a controlled narrative.
There is a precedent for skepticism. The tech industry has seen plenty of “independent audits” that turned out to be carefully scoped, lightly controlled exercises designed to produce favorable results. The difference here is the publication rights: METR can publish without approval, can publicly challenge redactions, and can describe the scope and limitations of its work. That is a genuinely different arrangement.
What Happens Next
METR aims to publish preliminary reports within weeks, not months. The scope of each assessment will be agreed upon in advance, and the reports will cover risk levels, incidents, and practices. Anthropic has committed to considering expanded access for SecureBio after its current review is complete.
For the broader industry, the question is whether other frontier labs will follow. OpenAI CEO Sam Altman endorsed Amodei’s September essay but has not yet announced a similar embedded evaluator program. If Anthropic’s experiment produces credible, independent risk assessments — even uncomfortable ones — it will set a new standard for what transparency looks like at the frontier.
For those of us who build with these systems — something I explored in how to use AI tools effectively in practice, the practical takeaway is this: the era of “trust us” AI safety is ending. Whether the replacements are credible depends on whether the access is real, the publication is genuinely independent, and the evaluators have the technical depth to assess what they are seeing. Anthropic and METR have just put their reputations on the line to find out.