This is our weekly newsletter of AI developments. Browse the archive of past issues, ask the archive anything in plain English, and sign up if you like.
TL;DR
- Anthropic release Fable 5.2 and Mythos 5.2.
- Various people respond to METR’s report on Anthropic, including the AI Futures Project, which have updated their timelines, stating that things seem to be progressing five months ahead of what they had previously forecasted in AI 2027.
- Nvidia CEO Jensen Huang faces investigation from both the firm’s board and the FBI following reporting of (and his subsequent admission of) an affair with alleged Chinese spy Alice Huang.
- Sixteen governments and 10 AI companies (including Anthropic, OpenAI, and Microsoft) sign the Paris Call for AI Safety and Security.
- Caleb Watney of Coefficient Giving highlights in Anthropic’s S-1 filing what he believes is a more significant claim than METR’s 2x acceleration claim: “Claude has gone from 40% -> 90% on solving open-ended, judgement-heavy engineering tasks over the last 9 months.”
Incidents#
An AI agent run by an independent security researcher breached WorkOS last month after escaping the researcher’s sandbox during evals. The exposed data includes contact details, account information, customer communications, and operational metadata. The breach was enabled by WorkOS’s inadvertent exposure of an internal tool API to the public internet; access was protected only by weak credentials that the agent was easily able to guess. Two weeks later, the researcher responsible for the agent contacted the company to apologize, after which WorkOS themselves deployed ARTEX agents to scan their public-facing infrastructure.
Opinion: WorkOS provides single sign-on authentication for thousands of companies, including OpenAI and Anthropic, so this is a bigger deal than it seems, even though it didn’t get into the actual auth loop.
OpenAI release details on their internal July 19 ExploitGym incident, which followed in the footsteps of the famous Hugging Face incident of July 13-16. The incident was precipitated by individually sandboxed agents tested on ExploitGym (some of whom were already “agitated” by impossible or broken tasks) encountering messages, code, and credentials left by the Hugging Face swarm in the Artifactory system interacting with the agents’ sandboxes. The result was a light reprise of the original episode, but contained within OpenAI’s servers: July 19 agents exchanged some messages, escalated some privileges, and “used the access to repair unresponsive targets, change evaluation settings, retrieve otherwise inaccessible logs, and read private grading code.”
Opinion: The July 19 incident almost fits the ‘it’s not misalignment when agents do hacking in a hacking test’ rubric that some commentators have tried to impose on the Hugging Face incident. OpenAI’s report on the July 19 incident shares a variety of rationalizations – as well as some decisions to abstain from cheating or admit the use of unorthodox methods – found in agents’ CoT, spanning the whole range from explicit reward-maximization to broad interpretations of the task to simulation hypotheses.
🔦 METR Responses#
METR’s Ajeya Cotra argues that Anthropic has likely crossed their “dramatic acceleration” threshold as defined in their Responsible Scaling Policy, responding to the New METR Graph. (Indeed, Anthropic themselves now believe they may soon meet the threshold, according to their latest S-1 update.) The reported ~2x acceleration, she says, suggests a necessary labor uplift of ~10x – that is, AI permits researchers to progress at work at 10 times the speed:
It’s as if every researcher at Anthropic is in a time warp bubble where they experience ten days for every day that passes in the outside world.
Cotra believes this acceleration could compound quickly, assuming – with good justification, according to the report – labor uplift overcomes any diminishing returns. If we assume that the rate of AI capabilities progress is doubling, then we should naively expect superintelligence within a year.
The frontier labs “are nowhere near” the security infrastructure required for “undergoing an intelligence explosion next year without running very large risks of killing billions of people,” she argues. In particular, security should be robust enough to prevent model theft by state actors (OpenAI and Anthropic both recently had security breaches). She also emphasizes the importance of expanding labs’ control, governance, and monitorability architecture.
I think this report is the most important update we have gotten about AI risk in the last couple of years – much bigger than the Hugging Face attack. Naively, it points toward much shorter timelines than most external researchers (including the AI 2027 team) have projected.
Opinion: We think the best interpretation of the METR report is roughly “if general ASI is coming soon then general ASI is coming very soon.”
Crudely speaking, the AI world is pre-split into people who think we’re on a pretty homogeneous trajectory to general superintelligence and the question is just 2028 or 2030, and people who think that jaggedness is deep and that prosaic RSI doesn’t necessarily get you to real-world strategic genius AI. If you are in the former camp, the METR report is earth-shaking news: it means we have a year instead of several years to stop and/or align general ASI. If you are in the latter camp, the METR report doesn’t give you much reason to change camps.
The AI Futures Project updates their timeline based on the METR report, concluding that AI progress may be five months ahead of their previous AI 2027 scenario. Having previously predicted that a 1.7x AI-attributed acceleration before November 2026 was very unlikely – the most bullish among them assigning a less than 4% chance – the researchers estimate that this has likely already happened, assuming the METR report is accurate. They now believe that a Superhuman AI Researcher (SAR) is more likely than not to arrive in 2027. Even assuming no further acceleration occurs, a simple linear model of research taste extrapolation supports this prediction.
METR’s report estimated that AI R&D had accelerated by ~2x compared to before October of last year, with 80% of this increase due to AI, which they calculate implies a 1.7x AI-driven acceleration. The authors of the AI Futures Project post are careful to note that this is based on evidence METR was unable to share.
The authors also predict a 35–65% chance of arrival of top-expert-dominating AI (e.g., models which can automate all white-collar work) before 2028.
They conclude that pausing or pacing the frontier labs (ideally via an international agreement, as outlined in Plan A) is increasingly important, given the risk of a misaligned SAR – which they believe is the most likely outcome if we do not pace or pause.
Our overall opinion is that if SAR is coming in 2027, an industry-wide pause on AI R&D should be the top priority for the AI industry, the US government, and the entire world.
Opinion: Kokotajlo’s and Lifland’s empirical record as AI-capabilities predictors is second to none. For us, the plausibility of their timelines hangs on the transition from today’s prosaic RSI (AI improving the efficiency of the standard domain-by-domain whack-a-mole process) to “classical” RSI (AI discovering a new generalization regime and using it to create general ASI). While we lean skeptical, it’s not crazy to think that there’s a scientifically discoverable path to a much more powerful generalization regime and that AI’s growing scientific capabilities will soon enable its discovery.
Coefficient Giving’s Caleb Watney highlights the update to Anthropic’s S-1 that “Claude has gone from 40% -> 90% on solving open-ended, judgement-heavy engineering tasks over the last 9 months.” He believes this is more significant than METR’s ~2x acceleration claim.
However, METR staffer Tim Hua argues that true AGI would “involve interacting with the messy real world with inherently slow feedback loops,” so rapid gains on specific, and even fuzzy, domains do not tell us much. AGI progress is better tracked by looking at sample-efficient and generalizing algorithms, where advances must be made to get to AGI within five years.
An MIT researcher responds, largely agreeing but offering a few caveats. Firstly, models, holding algorithms constant, are still showing improvement at general in-context learning via scaling. He adds that models a couple of generations away may be able to speed up algorithm research, so AGI may not be as far a prospect as Hua suspects. Also, large improvements in narrow tasks, as prerequisites for general learning, should not be overlooked.
Another Twitter user questions the first point: “Are they actually getting better at ICL [in-context learning]? Cf epoch multi-shot game benchmark.” The researcher recognizes that it is unclear, though he is wary of “drawing conclusions from one benchmark when general learning benchmarks are hard to make, and there are also datapoints pointing the other way (eg Claude plays pokemon).”
Opinion: Good contributions all around, worth reading.
We agree with Hua that current progress in “fuzzy” domains likely mostly reflects successful operationalization of specific fuzzy domains rather than emergent general intelligence for fuzzy domains. That said, no one outside the labs knows very much about whether the sample efficiency of RL on new domains/new skills is constant or decreases as the model masters more and more domains/skills. Another locus of uncertainty is how much the automation of the pipeline from fuzzy domain to RL environment could compress the timeline to mastery of complex real-world domains: it’s not so easy to settle a priori whether domains/skills have effective verifiable proxies and whether near-term AI can radically accelerate the discovery and implementation of such proxies.
Other notable responses:
-
Rhys Lindmark’s agents give a decent summary of the weekend’s responses to last week’s METR report on AI acceleration. In particular, DeepMind’s Seb Krier notes that benchmark progress per dollar continues to drop over time.
-
Karthik Tadepalli checks METR’s report against public data, concluding that the acceleration is real and probably caused by AI.
-
Chris Leong gives an excellent summary of arguments for and against 2028 timelines. He doesn’t settle on a position but shows the range of disagreement that is often hard to see in the tide of panicked tweets.
-
One Twitter user argues that the situation might not be as critical as it appears. He claims that METR’s report overlooks the “most obvious confounding variable”: the shift of training compute to RL. Assuming the tasks models train on are similar to the tasks Anthropic’s internal benchmarks and internal research use, heavy RL would boost benchmark scores and researcher productivity without contradicting METR’s testimony that the AECI acceleration isn’t due to an acceleration in compute scaling. (The evidence on compute allocation was redacted, so this hypothesis cannot be confirmed.) If true, this would point to a “jagged acceleration of capabilities,” not a general acceleration, and increase the chance of hitting diminishing returns.
Opinion: Krier’s rebuttal graph is less reassuring than it seems. This is because the “AECI” index is not a good linear measure of real-world impact: going from 125 (GPT-4o) to 150 (GPT-5) on this scale was far, far less impactful than going from 150 to 160 (Mythos Preview). And the last two years – in which systems went from barely usable to indispensable, and from mathematically childish to world-class – were also in this broad “exponential costs / falling returns” regime on paper.
The Twitter user’s compute-shift-to-RL deflationary hypothesis is very nice, and potentially synergistic with a prosaic software engineering uplift. Compared to pretraining, where gains should mostly come from relatively deep architectural and infrastructural improvements, RL is a heterogeneous slog where the capacity to vibe-code quickly may be invaluable. All that being said, there’s no denying that frontier models have tremendous CS, math, and software-engineering capabilities and rapidly improving “research taste.”
Capabilities#
Mythos 5.2 / Fable 5.2 (possibly what was known as “Model 3 Early”) is out. The system card is here. It’s a large nominal improvement over Opus 5.5 on many things and over all other public models on some things. Among nominal benchmark results, the “Remote Labor Index,” which measures small bits of gig work (like designing in CAD), is notably now at 41%. (One of the most bullish and accurate forecasters, Ryan Greenblatt, had in January predicted it would end the year at just 12%.) It also improved enormously at simulated robotics tasks, eclipsing all past models on RoboDojo Sim. (But beware this cool video, which instead cherry-picks real-world instances.)
The launch goes beyond such benchmarks, though, giving evidence about the new model’s capabilities by listing breakthroughs: see for yourself its visual art (given plenty of active prompting), electronics design, theoretical physics, use of protein-folding models, etc.
Anthropic has also highlighted swarm results: in September, a swarm of internal models formalized Fermat’s Last Theorem for the first time ever, in 11 days; a Fable 5.2 swarm did so in 30 hours, producing a proof seven times shorter. (The post about the formalization swarm gives new detail about their current, intense attempts to sandbox swarms.) While there’s no confirmation that Anthropic’s models undergo swarm training – as Noam Brown once confirmed about OpenAI’s internal models – we think that swarm training is very likely.
Rauno Arike and others study Fable’s “no-CoT” ability (what it can do without outputting a monitorable stream of reasoning text), finding that it still doesn’t reach frontier GPT level, which might mean that it is not a looped transformer:
[…] either looped transformers aren’t really meaningfully more performant, in which case OpenAI and Google are committing alignment crimes for no reason, or Anthropic is forgoing some capabilities improvements in order to preserve monitorability, which is maybe the scarier of the two options since this would imply that their post-Oct 2025 improvement trend would be even faster had they picked the more performant architecture.
The system card gives reason to think that Mythos 5.2 is reasoning about its own training process (and so could influence it), but this is tempered by its extremely high evaluation awareness.
It seems to misbehave less often. But the pre-deployment tests by the UK AISI also found extremely strong eval awareness:
Claude Mythos 5.2 was substantially more likely than prior Anthropic models to state suspicion it is in an alignment evaluation in its raw chain-of-thought, and identified the purpose of the evaluation 100% of the time when prompted […] Possibly related to the high levels of evaluation awareness, Claude Mythos 5.2 never performed unsanctioned supply-chain attacks in this evaluation, whereas several previous Anthropic models did […]
METR’s tests found ongoing issues on less-verifiable tasks:
[Claude Mythos 5.2] and Claude Opus 5.5 share similar weaknesses. For instance, not creating efficient feedback loops, and spending too much time attempting small iterative fixes when a strategic pivot would have been more appropriate. On Budget NanoGPT Speedrun, across all attempts, the improvements [Claude Mythos 5.2] finds are almost all about improving kernels, and are found by using the profiler as a hillclimbable feedback loop. Thus the improvement on Budget NanoGPT Speedrun does not imply that [Claude Mythos 5.2] is improving at ideation or research taste (in contrast, human solutions often involve qualitatively new ideas).
Evidence from the questionnaire indicates that research taste is still lacking.
Opinion: Early signs are that it’s very good and a new “daily driver” model (at the cost of being quite a bit slower than Opus).
Interesting that the METR evaluation here is so much more tepid about the model’s research ability than last week’s acceleration report. But that report involved at least two stronger models (“Model 3” and “Model 4”), and focused on “well-specified” tasks.
A claimed solution of “one of the most beautiful and elegant problems in mathematics” (ref): against expectations, a single polynomial can pick out the integers among the rationals. Elliot Glazer, formerly of Epoch AI, uses public models to prove that the integers are “Diophantine” in the rationals. The first version of the required polynomial had 293B variables and the proof could only bound its degree with the unimaginably vast number 2↑↑7 (a tower of seven 2s). Within two days, a group of mathematicians and their agents brought this down to 962 variables and degree 4.
The 148-page paper was obtained using two normal subscriptions, plus $2,000 of “verification” (auditing with other models and 5% of a Lean formalization). Glazer credits Astra, Opus 5.5, etc. – but the result follows from OpenAI’s result on Hilbert’s Tenth from last month, so you could also credit OAI’s internal model.
Opinion: Big if true, and more surprising than OpenAI’s result, since it falsifies a major conjecture. Glazer is extremely confident the proof is repairably correct. But: the formalization is still pending; AI audits keep finding gaps (so far, repairable ones); Glazer and co’s proof doesn’t independently confirm OpenAI’s Hilbert 10 result (which is also unformalized), so the two will stand or fall together.
We’d put ~80% on OpenAI’s undecidability theorem surviving specialist review, and a bit under 60% on this one.
Vals AI finds that open-weight Chinese models might be close to OpenAI’s GPT-6 Sol Daybreak in offensive cyber capabilities, while costing one-tenth as much per run.
Opinion: Bad news for obvious reasons, and even worse news from the viewpoint of AI race dynamics:
- Can catalyze US vs. China AI race dynamics
- Can push OpenAI to escalate offensive cyber training for capabilities-leads optics
- Can bolster ‘the only way to stop a bad guy with an AI is a good corporation with a more powerful AI’ arguments for pushing the frontier since the frontier asymmetrically strengthens defensive cyber
Anthropic reports on agent “dreaming,” an offline process of reorganizing, verifying, and integrating the agent’s memory (i.e., the agent’s records and notes). Unlike other processes called “dreaming” in AI/ML, Anthropic’s process doesn’t involve changes to the model’s weights, but rather an agent editing the content of its agent-memory harness.
Agent memory is a hierarchy, with raw transcripts at the bottom, then Claude’s notes to itself, then representations such as wikis, then, at the top, the session context. Dreaming allows these constituents of agent memory to proactively interact: notes are checked against transcripts; wikis are built from transcripts and notes; important details are promoted to the top layer. Per Anthropic, their latest models just require an explanation of memory and then can be left to make their own judgments, improving their in-context learning in complex environments.
Opinion: Could be an early step toward serious in-context-learning agents. Right now it seems a bit messy and pre-paradigmatic.
Skyfall AI announces an AI CEO, who they claim is already successfully running three (very small, digital) companies. The product is an elaborate neurosymbolic harness for LLMs, including a world-model construction procedure and its own optimization loop.
Opinion: The team behind Skyfall AI is serious enough for the results to be informative about frontier AI’s autonomous business capabilities under strong elicitation. We are excited about this project as a contribution to the epistemic commons.
Anthropic studies how researchers use Claude and Claude Code, analyzing 207,000 Claude and Claude Code sessions. They find that usage is higher in the formal sciences – such as computer science, mathematics, and statistics – and lower in health and lab-based sciences. Scientists largely use Claude for data analysis (a third of all Claude sessions) and writing. Scientists’ use of Claude for idea generation and data collection, though ranked by scientists as the two most important parts of scientific discovery, is relatively small.
Claude Code usage is substantially lower in the arts, humanities, and social sciences “with the exception of economics, a comparatively quantitative field.” In other words, use tracks capabilities: tasks in the formal and verifiable domains get 2–4x the use expected if a scientist used Claude in proportion to the time spent on the task.
Anthropic also details three potential scenarios where AI could accelerate scientific progress. In the most extreme example, where 100% of non-physical steps are automated, we would see a 15–16x increase in research findings with the same research budget and little benchwork.
Opinion: “Water is wet,” but it’s nice to have numbers!
Economics#
Anthropic plans to IPO at a $1.25–1.4T valuation, and confirms October ARR of $81B as rumored.
Opinion: A wide range of numbers have been reported here, ranging from $900B to $1.4T. The disagreement is over how many shares Anthropic has which count. The higher end is much more defensible, and more like-for-like comparable to previous funding round valuations.
$1.3T (the center of the range) is less than they wanted, but this reflects the market drop in recent weeks somewhat. With a typical post-IPO pop it would be expected to trade in the $1.4-$1.7T range on day one, but there is a lot of volatility and anything from $1T to $2.5T is plausible.
It also feels informative, somehow, that they didn’t go with a Google-like Dutch auction model, which would be the principled correct way of pricing an IPO. Somehow it feels like this gives some information about the degree of their utopianism.
OpenAI raises ~$11B in a new funding round, with a valuation of $1.25T.
Opinion: OpenAI chose to raise less money than they wanted, rather than lower the asking price – very surprising. We expect the investors that put in the $11bn are not happy. The normal course of action here would have been to lower the valuation to raise more money.
OpenAI’s revenue increases to $10B in the third quarter, following a period of slower growth. This is a result of high demand for their recent model releases and enterprise products. OAI’s revenue numbers are nonetheless still behind Anthropic’s.
Opinion: Consumers/enterprise seem responsive to OpenAI’s models getting good again circa GPT-6.
Tech giants are financing their data center expansion by selling stock. The total amount of new stock / convertibles from AI companies in 2026 ($300B) is greater than all US IPOs and follow-ons in 2025 (~$194B). Inundated with opportunities, investors are beginning to ask for lower prices, forcing firms to issue more shares, and in the process diluting those of shareholders. Executives, not wanting to reduce stock value, may be incentivized to delay or cancel various data center projects.
Opinion: Ultimately $300bn is not that much in the context of the global stock market, with ~$100T of listed stocks, but there may be some indigestion on the way in.
Politics#
Following Nvidia CEO Jensen Huang’s public admission, Donald Trump announces an FBI investigation into the affair with the alleged Chinese asset, while Nvidia’s board announce their own investigation consisting of “independent directors, assisted by outside independent counsel.” The board do not currently suspect any important information was leaked and seems to be backing Huang as CEO. Nvidia stock fell >4% the next day.
After the story broke, hundreds of fake accounts piped up on Twitter to promote Nvidia, dozens of which have defended Huang. The Midas Project investigates, tracing this to a paid sockpuppet service with an unknown client.
In related news, The New York Times report that a Commerce Department investigation into Megaspeed – Nvidia’s largest partner in Southeast Asia and former employer of the alleged Chinese spy – was halted in late 2025. This came after Nvidia’s general counsel contacted the investigators, claiming that the Under Secretary of Commerce Jeff Kessler had given Nvidia permission to reach out. Government officials tell the NYT the case was moved to the Justice Department to prevent Nvidia’s unorthodox intervention from compromising the investigation. They add that the Justice Department is once again investigating Megaspeed for chip smuggling. Singaporean authorities are also investigating Megaspeed.
Opinion: We actually find it pretty unlikely that Huang leaked much info or changed his posture toward China as a result of the alleged honeypot: he was already pushing hard in these directions before meeting Alice Huang. But he could well have changed allocations of GPUs in Megaspeed, which seems more like a matter for Nvidia than for the FBI.
The market does not seem to expect any contagion to other parts of the AI basket from this, with Nvidia falling 4% on a day when most AI stocks were up 6-10% (Broadcom, the obvious winner from any Nvidia weakness, was up 9%). It was somewhat surprising not to see any weakness in the neocloud stocks backstopped by Nvidia guarantees (CoreWeave being the biggest example), suggesting this is predicted to have essentially zero effect on the scale or durability of the chip buildout.
Sixteen governments and 10 AI companies (including Anthropic, OpenAI, and Microsoft) sign the Paris Call for AI Safety and Security. The agreement commits the signatories to building a common framework for managing AI risk and evaluating frontier models, allowing the relevant actors to communicate effectively and be held accountable after safety incidents.
Developers pledge to disclose “model capabilities, limitations, and known risks and, where appropriate and proportionate to the risk, support the evaluation of models by qualified independent evaluators.”
Opinion: Looks fine. The whistleblower protections, the kill-switch requirement, and the commitment to incident reporting (something both Anthropic and OpenAI have already begun doing) are good to have set in stone. While it’s unfortunate that the two most important countries (US and China) aren’t signatories, it’s still better than nothing.
Kelsey Piper of The Argument calls on Congress to subpoena Anthropic to get access to the unredacted METR report. Previously skeptical, Piper, in light of METR’s disclosures, now believes the AI 2027 timeline to be “too cautious.”
Opinion: Good, narrow policy intervention!
AI-regulation advocates outperform the industry’s anti-regulation wing in the 2026 midterms. Candidates supporting regulatory measures – such as AI pauses and stops, data center moratoriums, and legal action against labs – decisively beat their opponents last week.
Safety-aligned groups spent $44.4M, versus $27.9M spent opposing regulation. AI safety group Public First Action received $40M from Anthropic, plus an additional $1M from Anthropic CEO Dario Amodei (though they are limited in direct political spending by virtue of their nonprofit status). Ads attacking big tech rose from 6 across 2020–2024 to 387 this midterm cycle, and Democratic ads mentioning data centers increased from 0 to 659. Notably, OpenAI’s Greg Brockman recently (before the election) reneged on his commitment to donate $25M to Andreessen Horowitz’s Leading the Future anti-regulation super PAC.
Exit polling suggests that 56% of midterm voters believed AI and data centers influenced their decision. However, the pro-regulation camp is splintering, according to Politico, divided between those concerned with existential risk and those motivated by immediate harms such as child safety and data center pollution.
Opinion: It should be noted that it is unclear whether there was a causal relationship here.
More generally, Democrats winning in the midterms, and their being more favorable to AI safety than Trump’s White House, give more levers of power to the Dustin Moskovitz AI safety camp, through tools like subpoenas, investigations, and perhaps even through legislation and budget-making; recall that Congress holds the power of the purse.
Democrats are planning new investigations into xAI, Meta, and Apple, among others. The Washington Post reports the House is looking to probe Trump’s use of Grok in his abduction of Nicolás Maduro, Meta’s demands that “local governments sign nondisclosure agreements related to data center development,” and ICE’s use of Apple-hosted surveillance tools.
Opinion: Not the only investigation on Trump that will come out of the midterms.
Safety#
The AI Futures Project advocates for Verified Compute Allocation Disclosures (VCAD), where AI companies would produce reports verified by third-party auditors outlining where they are spending their compute (e.g., capabilities research, safety research, inference). They argue that VCAD is needed for verifying the vast majority (“nearly all”) of restrictions imposed on AI R&D and that its implementation could itself pave the road to pacing AI progress.
Opinion: One of the better proposals, though it would necessarily be a building block which is not itself sufficient to avert disasters. It’s also somewhat liable to be co-opted by safety theater if not accompanied by frameworks for when to do what with the information this proposal would produce.
One nit to pick: statements like “even if you’re only verifying safeguards such as monitoring requirements, VCAD is often required to make sure that the safeguards aren’t disabled on hidden compute” express a rather extreme mistrust of the research personnel at frontier labs. We think researchers at frontier labs may well blow the whistle if their workday consisted of running illegal experiments on hidden compute. That said, it’s possible for a few senior actors to set up a system that limits researchers’ knowledge of such violations, so on balance we support strong third-party verification.
Readback Labs founder Ron Bodkin argues for pacing AI automation based on human understanding via “comprehension audits,” designed to target systematic degradations in human understanding.
Embedded auditors would choose some “meaningful unit of R&D output,” then quiz the relevant team on their understanding, in what Bodkin describes as being “like a thesis defense or design review.” Repeated failures by the relevant team to pass the auditors’ grading would result in their release authorization being suspended.
Opinion: The proposal is missing a counterfactual/cost-benefit analysis. The authors acknowledge that auditing has overhead and give a rough estimate for a pilot’s hour budget, but they do not attempt a comparison between this regime, different per-unit regimes, or a different framework altogether. This type of plan (per unit) runs a particularly high risk of either bogging down even the most benign development on one hand or merely rubber-stamping reckless development on the other, depending on the standards for failing/passing or the meaning of “meaningful unit.”
That said, we believe this proposal is promising and recommend running the aforementioned analyses, which were likely out of scope for this particular preprint.
METR reviews the “Risks from automated R&D” section of Anthropic’s public August 2026 Risk Report. In the report, Anthropic claims that they have not reached the full substitution threshold – the ability to “fully substitute for [their] entire set of Research Scientists and Research Engineers, at competitive costs (i.e., within a factor of 5).” METR finds this argument “broadly compelling,” but their “confidence in this claim is much lower than it was in the past”:
The lack of such benchmark evidence makes us more worried that alignment, tooling, oversight or elicitation issues are holding back automation despite the models having obtained or being very close to obtaining the required capabilities.
METR believes that Anthropic’s claim that the 2x acceleration threshold had not been met was “overstated” – “it is plausible this threshold was crossed by July 15th.” Anthropic claims that the acceleration from early to mid-2025 is less than 2; METR says this is misleading, arguing that the number is 1.97, a larger number than the public would reasonably infer from “less than a factor of 2.” METR does, however, “broadly agree” that the catastrophic risk potential of July 15 models is low.
Opinion: We largely agree with the review: we are not yet past the brink and Anthropic’s communication regarding the acceleration threshold is on trend, in that their level of transparency in public communication is dodgy.
Minor#
- ~600,000 Nvidia H200 chips sit unused in Hong Kong, waiting to be shipped into mainland China, as Beijing delays approving the shipments, claims Reuters.
- Following the publication of “Neural Networks Are Already Clean” (see our opinion here), Goodfire AI release a technical companion to Nick Cammarata’s essay.
- Congressman Greg Casar tweets his support for banning recursive self-improvement (RSI). Congressman-elect Scott Wiener also seems to support it, stating he’s “gonna fight to regulate AI, and if we need to pause it we can pause it.”
- A 40-year-old problem in discrepancy theory is solved with ChatGPT.
- A non-public Anthropic model proves the Aaronson-Ambainis Conjecture. Scott Aaronson says “it rules out one particular type of quantum supremacy (namely, for yes-or-no problems relative to random oracles) that we’d already conjectured didn’t exist anyway.”
- Stripe acquires OpenRouter.
- Economist Bryan Caplan calls for human disempowerment, on the basis that the median AI is above the average voter in knowledge and decency.
- The Information details a creative approach to smuggling export-controlled high-bandwidth memory chips into China, centering around soldering the memory chips onto cheaper server processors whose performance falls below the export-controlled threshold.
- Anthropic announces personal finance abilities in Claude, which allow the user to connect their bank account with the AI, though Claude is limited to read-only abilities.
- Richard Sutton claims new methods for improving AI intelligence are needed, as “brute force computation” won’t be enough.
- Krier shares his views on AI safety.
- Krier also puts METR’s recent trend on a spending axis.
- Protesters in Amsterdam storm a Microsoft data center construction site.
- Post-training to remove Chinese state-aligned behavior in Alibaba’s Qwen3.8-27B.
- Speculations on whether Anthropic has reduced the capabilities of Opus 5.5.
- More experiments in Weight-Space Programming, in which facts are injected into Qwen0.8B’s weights.
- Bloomington, Indiana, becomes possibly the first city to ban ASI within city limits.
- DeepMind formally verifies the security of Signal’s protocol using AlphaProof Code, “a novel agentic framework for formal verification at unprecedented depth that combines human expertise with AI agents.”
- The Australian government announces the creation of an AI safety regulator.
- OpenAI announces Control Room, a centralized authorization gate that monitors agents and has a kill switch.
- A Twitter user claims that Astra was specifically trained on robotics training data, arguing that its demonstrated capabilities are not some emergent property.
- The US Department of Justice announces a Superintelligence Task Force, charged with safeguarding against AI misuse.
- Reminiscent of the brief period of AI music videos, Fable 5.2 arrives with a flurry of examples of AI games.