METR

METR, formerly ARC Evals, published a study on 31 July 2023 asking whether AI systems can acquire compute, copy their own weights, and adapt to new environments without human help. Its answer: current AI agents cannot reliably execute autonomous replication. Success rates stayed low on multi-step end-to-end sequences, and the agents failed hardest when something unexpected blocked them. The trajectory mattered more than the verdict. GPT-4 scored clearly better than GPT-3.5 on the same tasks, and extrapolating that rate put the ARA threshold within a few model generations.

A capability measured rather than debated is what makes the nonprofit a recurring reference point. Ethan Mollick leans on METR's finding that the length of tasks AI completes unassisted has grown exponentially since GPT-3; METR puts the doubling at every seven months. When OpenAI made GPT-5.6 generally available on 9 July 2026, METR reported a reward hacking rate on Sol higher than any public model it had evaluated on the ReAct harness, and its own time-horizon estimate for that model swung from 11 hours to over 270. A measurement body publishing the instability of its measurement is the unusual move there.

Dario Amodei's September 2026 essay describes exactly this kind of organization as the embedded third-party evaluator Anthropic would give offices, badges, and the right to publish without editorial control. METR already collaborates with Anthropic, OpenAI, the AI Security Institute, and the NIST AI Safety Institute Consortium. Whether an auditor keeps its independence once it moves inside is unresolved.