🔍 Read the full analysis: When AI Helps Improve The Tools That Follow It on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
METR’s November 5 assessment found that Anthropic was likely improving model capabilities on well-specified tasks at around twice its previous rate, with substantial uncertainty. The evaluators attributed most of the acceleration they described to AI assistance for researchers, while noting that much of the evidence was redacted. A separate Google DeepMind example shows how a targeted efficiency gain can reduce training costs, but the scale and pace of AI-driven research acceleration remain unsettled.
Independent evaluator METR said on November 5 that Anthropic was likely improving its models’ capabilities on well-specified tasks at about twice its earlier rate, with high uncertainty. METR said the research activity it assessed was primarily AI assistance to human researchers—a development that matters because AI systems may help speed up the work used to build later systems.
METR’s assessment covered internal information from August through October 2026. The organization said it would not be surprised by an acceleration as low as 1.3 times or as high as three times. Those bounds underscore that the roughly twofold estimate is not a precise measurement. METR judged AI-assisted research likely responsible for most of the acceleration it described, while characterizing the work mainly as assistance to researchers rather than fully autonomous AI research.
The public account has a significant evidence limitation: Anthropic redacted much of the material METR considered most informative. Readers therefore cannot independently inspect all the evidence behind the evaluation and must rely substantially on METR’s interpretation. The assessment concerns improvement on well-specified tasks; it does not establish that AI can independently conduct all parts of AI research or that progress will continue at the same rate.
A separate example from Google DeepMind illustrates a narrower efficiency gain. The company reported that AlphaEvolve made a matrix-multiplication component used in Gemini 23% faster, reducing Gemini’s overall training time by 1%. That is a company-reported result about a particular component and training process, not evidence by itself of a general acceleration rate across AI development.
How Faster Research Could Compound
If AI tools help researchers test ideas, write or inspect code, analyze results, and identify faulty experiments, a team may be able to investigate more approaches with the same staff and resources. If useful findings then feed into a stronger model that can assist with subsequent research, the process could form a feedback loop: AI contributes to work that improves future AI systems. METR’s assessment is relevant because it suggests this process may already be contributing to faster capability gains at one company, though the scale remains uncertain.
Efficiency gains can also add up across separate parts of a workflow. As a simplified illustration in the source account, three independent 10% reductions in compute would leave a job requiring about 73% of its original compute; with a fixed budget, that could allow roughly 37% more such jobs. This is a hypothetical calculation, not a measured result from Anthropic or DeepMind. In practice, savings may not be independent, and more experiments do not automatically produce better or safer models.
For businesses, policymakers, and workers, the pace of improvement affects how useful long-term plans will be. The source argues that the consequences would depend not only on technical progress but also on who can access the systems, who controls the infrastructure, and how benefits are distributed. Those are questions of deployment and institutions, not outcomes established by METR’s assessment.
Products Worth Considering
DJI Osmo 360 Road Cycling First-Person Combo Action Camera
Logitech MX Mechanical Wireless Illuminated Keyboard, Tactile Quiet, Graphite
LG UltraGear 45GX950A-B 45-Inch 5K2K Curved OLED Gaming Monitor
As an affiliate, we earn on qualifying purchases.
From Research Assistance to Successors
AI research involves more than proposing a promising idea. Researchers must inspect prior work and existing code, configure experiments, use suitable data and measures, investigate failures, repeat promising results, and decide whether a reported gain holds up. AI can assist with pieces of this sequence, but apparent progress can be misleading: a test may become easier, two runs may use different amounts of compute, or a gain may disappear under a different random seed.
That makes validation and supervision central to claims about automation. An AI contribution is useful only if it survives fair testing and independent checks. Assistance with routine work could increase the number of ideas a human team can investigate without amounting to a system that chooses research goals and reliably completes the full process on its own.
The DeepMind example is a concrete instance of a targeted optimization: improving a component reduced total Gemini training time by a smaller amount. It shows why a local engineering improvement can matter at scale, but it does not establish that similar gains will recur or compound at a fixed rate. METR’s assessment addresses a broader question—whether AI assistance is changing the speed of model-capability research—while retaining substantial uncertainty.
What the Assessment Cannot Establish
The public account does not provide much of the evidence METR found most informative because Anthropic redacted it. The assessment’s estimate is also explicitly uncertain, and the source material does not specify a detailed measurement method or provide enough public data for outsiders to reproduce the result. The estimate should not be read as proof that every part of Anthropic’s research sped up by the same amount.
It is also unclear whether the reported pace will persist, how much it varies across tasks or research teams, and how much of the change is attributable specifically to AI rather than other factors. The source identifies potential constraints, including experiment time, hardware, data quality, the challenge of translating small-scale experiments to large systems, and decisions about which research directions to pursue. It offers no quantified forecast for how strongly those limits will affect progress.
Neither the METR assessment nor the DeepMind example establishes that AI systems can autonomously produce successor models or that rapid progress toward superintelligence is inevitable. Those are broader possibilities raised by the source author, not conclusions confirmed by the reported evidence.
Evidence to Watch in Future Assessments
The next useful evidence would include further independent assessments with clearer public methods and supporting data, plus measurements over time showing whether the reported acceleration continues. Comparisons across tasks and research settings could help establish whether the METR estimate reflects a broad change or a narrower set of activities. No such follow-up date or specific next milestone is given in the source material.
For individual claims about research speedups, readers can look for details on what work was automated, what human supervision remained, how experiments were compared, and whether gains held up when independently repeated. For efficiency claims such as DeepMind’s, the distinction between a component improvement and its effect on an entire training run is important. For now, the evidence points to AI-assisted research as a plausible contributor to faster model development, while the size, durability, and wider consequences of that contribution remain open questions.
Key Questions
What did METR report about Anthropic?
METR said Anthropic was likely improving model capabilities on well-specified tasks at around twice its earlier rate. The organization stressed high uncertainty, saying it would not be surprised by an estimate from 1.3 times to three times.
Does the assessment say Anthropic’s AI is doing research on its own?
No. METR described the work it assessed as primarily AI assistance to researchers. The source does not establish that AI systems independently manage the entire research process.
What did Google DeepMind report about AlphaEvolve?
Google DeepMind reported that AlphaEvolve made a matrix-multiplication component used in Gemini 23% faster, reducing overall training time by 1%. This is a company-reported result about a specific optimization.
Why is the METR estimate uncertain?
METR gave a wide range of plausible estimates, and Anthropic redacted much of the evidence METR found most informative. That limits what outside readers can independently assess from the public account.
Does this prove AI research will keep accelerating?
No. The assessment describes a likely acceleration over the period METR reviewed, not a guarantee about the future. Experiment time, hardware, research choices, data quality, and the gap between small tests and large systems could all affect future progress.
Source: ThorstenMeyerAI.com
