This is our weekly newsletter of AI developments. Browse the archive of past issues, ask the archive anything in plain English, and sign up if you like.
TL;DR
- A mathematician at Anthropic claims to have resolved the generalized Riemann Hypothesis after months of steering an unspecified Claude model to 17M-line proof in Lean.
- Benchmark scores continue to run ahead of real-world performance.
- Goodbye Black October, hello Vantablack November!
- All frontier, consumer-grade models show an alarming readiness to assist in military targeting exercises, according to a study by Vals.
- Prime Intellect suffer an attack resulting in a substitution of their DeepSeek model for an abliterated changeling.
- The Chinese Communist Party’s Qiushi journal publish a piece on the existential and geopolitical risks associated with an intelligence explosion.
- Anthropic launch their IPO, resulting in a valuation of $1.5T.
Capabilities#
Levent Alpöge of Anthropic claims to have resolved the Riemann hypothesis and its generalization via months of steering an unspecified Claude model, resulting in 17M lines of Lean which appear to offer a complete machine proof. This is often called the most important open problem in all of pure mathematics. The hypothesis is apparently true.
Claude far outpaced me near the finish line. No, Claude flew, and danced, and sparkled near that finish line. I had us on the line of combining two quite different ideas present in a toy setting, one of which was already new and shocking, making very simple one of the shadows of Riemann over which we mathematicians felt most triumph. I guided us to the right situation analogous to that toy setting, and then Claude began to leave me in awe.
The proof pulls from an enormous range of subfields. In Filippo Nuccio’s words, its sheer scope is “gorgeous: it basically contains all number theory produced in the last 50 years, in all branches of the discipline.”
The mathematician Tim Gowers greeted the result by (sarcastically) saying “Who would have thought that the convex hull of known mathematics contains such riches?” – i.e., that this is a blow to the comforting idea that LLMs are just interpolating between training data.
The result was obtained last Sunday but was held up, presumably by the IPO “quiet period” (in which a newly public company refrains from significant announcements for a period). We thus got Riemann before GTA 6.
The proof is Lean-only. As Alpöge explains:
This puts us in the rather extraordinary situation this age has made possible, in which I am presenting the world with a complete argument before having mastered a massive neighbourhood around its mathematics. It being obviously inappropriate to attempt to post some massive unreadable travesty of “slop” [Footnote: I of course amused myself, and hit 250, 000+ pages], I decided to only present Lean.
In sheer size, Claude’s RH proof is comparable to all prior formalized math written by humans, or to two to three months’ worth of all new math papers published on arXiv:

Is it true? Very probably. A community of Lean users are actively engaged in auditing the purported proof and ensuring that it runs on a variety of Lean kernels to guard against the possibility that Claude cheated by exploiting a kernel bug. There’s good reason for taking such precautions – as recently as August, OpenAI’s Daniel Selsam (with the aid of OpenAI’s models) and Lean creator Leonardo de Moura discovered exploitable memory corruption bugs in Lean and its dependencies that a malicious prover could exploit to prove a falsehood, and LLMs’ tendency to reward-hack and exploit bugs in their environments is known to be exacerbated as the difficulty of their assigned tasks increases. Since memory corruption bugs tend to be highly sensitive to the configuration of their environments, it’s unlikely that any such exploit, if it exists, would consistently succeed across sufficiently different contexts. By now, the math community have verified the proof across various Lean implementations, including paranoid ones meant to avoid some of the above problems.
An independent human proof of the generalized Riemann hypothesis was hastily announced in response. The 940-page proof is reportedly the result of 12 years’ work by Alessandro Masullo, a Stanford math PhD, who had planned to spend another year verifying it. He hopes that “people quickly check it, formalize, fill any small gaps if they occur.” An automated audit found several gaps in the work (which he was forced by events to publish early).
On the same day, Alpöge and the NYU mathematician Tristan Buckmaster also claim to have solved the full Navier-Stokes blowup problem (that is, the “unforced” half left open by OpenAI’s September result). It is a mere 79 pages. The paper does not cite the OpenAI result.
Anthropic stock fell 10%.
Opinion (Peli): In our discussion of the distinction between verifiable and non-verifiable domains in our October 20 issue, I mentioned in passing the possibility of a Lean-native, humanly unintelligible proof of the Riemann hypothesis:
Producing a verified resolution of, e.g., the Riemann hypothesis is a cheaply, instantly testable achievement that would likely require AI to develop most every kind of mathematical excellence. (Putting aside the possibility of AI “crunching out” unintelligible Lean breakthroughs on problems mathematicians currently believe to be theory-bottlenecked.) It’s hard to name a similarly powerful proxy for capability in medicine, physics, philosophy, statecraft, chemistry, business, law, and so on.
The whimsical hypothetical of a “low-insight” proof of the Riemann hypothesis also came up in our September 15 discussion of OpenAI’s Navier-Stokes resolution:
While OpenAI’s Navier-Stokes solution is almost certainly correct, it hasn’t yet been evaluated for “mathematical insight,” and many mathematicians (perhaps out of bias) expect little insight from it. That said, if mathematical insight turns out to be generally dispensable for problem-solving even in pure math, then “can frontier AI produce great mathematical insight” might not deserve to be a central question in AI capability evals. While mathematical insight is valuable for its own sake, part of the weight we place on the question of LLMs producing insight comes from believing that some long-term concrete goals – in math and elsewhere – like solving the Riemann hypothesis, solving climate change, or turning everything into paperclips require (a superset of) insight as humans know it.
We don’t yet know whether the Claude/Alpöge proof is low-insight or simply raw. Initial commentary suggests that the proof draws on many, many areas of human mathematics, but it’s unclear if that’s a sign of deep synthetic work or of chaotic crunch in what’s effectively the longest original proof in history. (Importantly, the revelation that Lean-native crunch of an elite open problem is possible now makes it newly tricky to relate Lean proofs and natural language math to one another, vindicating Lucca in our October 10 disagreement on the matter.) The coming weeks should start shedding some light on this question, as we expect a high level of formal and informal coordination in the mathematical community around the study of Claude/Alpöge’s Lean proof. On a social and institutional level, this is partly enabled by Alpöge’s apt choice to produce no paper but only a formal mathematical object (Lean files), allowing mathematicians to approach the study of the Lean proof as proper mathematical research-work.
What is the meaning of this (shocking!) news for AI capabilities in general? I think it’s both evidence for the near-term possibility of strong superintelligence conditional on math being a “leading indicator” in AI capabilities, and evidence against math being a leading indicator in AI capabilities.
Conditional on pure math being a “leading indicator” for capabilities in science at large, or even just for capabilities in the broadly mathematical sciences (including machine learning), we are moving into “an artificial superintelligence would be godlike relative to human intelligence” territory. Whether the Claude/Alpöge proof is crunchy or insightful, and whether it’s a rich tour through human mathematics or alien movement through Lean-space, it’s clear that AI is now strongly superhuman at achieving well-defined pure math tasks. To the degree that it makes sense to speak about AI achieving analogous capabilities in other fields, superintelligence in the sci-fi sense is now on the table.
I also think the Claude/Alpöge RH proof is evidence for the hypothesis that pure math is a truly weird case. What do I mean by “weird case”? For example:
- Maybe pure math as a subject matter is for LLMs what chess was for GOFAI.
- Maybe pure math as a research field is uniquely organized around proxy goals (proofs of conjectures) that humans but not LLMs lack the crunch-power to Goodhart away from the field’s real goals (theory and understanding).
Both possibilities are very weird, but so is the gap between LLMs’ current power in pure math and their power in other fields. The gap between pure math and biology or business capabilities is weird enough, but the gap between LLMs’ current power in pure math and their power in the broadly mathematical sciences is bizarre.
(Needless to say, a months-old human proof of the Riemann hypothesis would throw all this talk about “superhuman math” into chaos. I’m personally skeptical that the claimed human proof will upend the near-consensus that proving the Riemann hypothesis in 2026 is strongly superhuman.)
Opinion (Nuño): Back in September, Paradigm 3 wrote about proofs rather than disproofs being a major fork in the road, because they would be demonstrating greater mathematical creativity, rather than rote exploration. That is also the impression of awe I get from reading Alpöge’s writing. Per this comment, the high-level architecture of the proof is a creative combination of many areas of math.
In my opinion, this has implications for P3’s default-skeptical stance; since we made predictions and bets beforehand, with Gavin giving 1:7 odds against this scenario, this is now a good opportunity for our project to update. There is danger in updating too slowly. The worlds where AI couldn’t reach this milestone are now gone. The “we are living in the future” highly compressed model seems more accurate and predictive than a more sophisticated and complex detached skepticism. Below is a toy Bayesian update:

I have also become less bullish on sensemaking, as opposed to specific preemptive action to make the coming months and year less tumultuous, like creating more cybersecurity startups.
OpenAI release a report on research acceleration. Inputs have ballooned, with the median researcher spending $2,400/day at API list prices in October, up 62% from September. They would also be “willing to trade 30% of team compute or 40% of team headcount in exchange for three months of frontier coding agent access.”
Agent workdays per researcher rose 11x, from 0.5 to 5.7. On two- to four-hour tasks, agents were able to successfully complete 46% in October, compared to 23% in March. Output per researcher has also increased – more lines of code and more experiments.
Capabilities, however, have seen more sluggish growth. OpenAI show no trend-break on their OECI: their internal capabilities index shows 16.4 points of growth compared to August 2025, only slightly above the Epoch Capabilities Index (ECI) industry average of 14 points per year. This suggests a 1.17x speed-up, compared to METR’s reported ~2x acceleration of progress at Anthropic. (OpenAI say they are still determining “whether capabilities progress is accelerating due to RSI.”)

They claim that the total time spent by humans on safety and alignment increased 77% between January and September. OAI argue that labs must publicly disclose information related to RSI, but say that their “reporting should not be read as an intention to pursue full RSI.”
David Holtz, one of the OAI researchers behind the report, emphasizes the ability of agents to complete different tasks without any human help – specifically, the “heterogeneous completion rates across different AI R&D tasks.”
In response, METR’s Tom Cunningham says:
I think I find [the results/graphs] hard to interpret quantitatively because we expect a lot of attenuation due to selection right? I’ll only submit a task if I think the success rate is sufficiently high. Thus the length-success relationship will be attenuated, and over time the growth in success rate will be attenuated, relative to a world where i submit all my tasks.
The OpenAI Foundation’s Yo Shavit wonders whether the lack of trend-break is “a methodological artifact, or a sign of [Anthropic] being farther into RSI despite ECIs being similar.”
Ramez Naam argues, as he usually does, that the report reveals serious limitations to real-world AI R&D: comparing OpenAI’s data on agents’ internal research success without human intervention with METR and AI 2027 timelines, he maintains:
Real world data continues to show AI autonomously completing tasks much, much shorter than predicted by ECI x METR graph. Gap is widening.
Opinion: Much less scary and a bit less secretive than Anthropic’s report. (There are enough degrees of freedom here that one could engineer a “less scary” result.) Overall, the absence of a trend-break here makes us slightly less worried about imminent RSI (since the hypothesis “Anthropic just stopped making a bunch of training errors and started ~matching OpenAI on benchmaxxing” rises on this report).
Doubling their safety effort is better late than never! Last year we estimated that <3% of OpenAI staff were full-time on AI safety (in the broadest sense); if that’s 5% now, why not 10%? The move was, however, shamefully reactive: OpenAI’s models seem a bit more misaligned than Anthropic’s models, and their belatedly realizing this fact has both 1) freaked them out and 2) also presumably caused commercial problems / PR problems for them. We might not get that many more chances to FAFO with these systems; better if they can update now.
Judging from his recent experiences with LLMs, Arjun Panickssery doubts the imminence of automated AI R&D. He has been trying to build a new Scrabble engine and tic-tac-toe bot, along with researching philosophy and attempting to formalize some proofs.
Panickssery claims that he has been unable to get Astra to pick out the “most important and useful” material when conducting a literature review. Similarly, in its philosophy tasks, his model kept pursuing “extremely trivial” lines of thought. Astra’s research “taste” was found lacking too:
It will conduct experiments that have no information value because the amount of training data used was obviously too small to distinguish the architectures to be compared.
One AI researcher notes that there are some explanations for this that do not rule out imminent RSI. It could just be that public models like Astra are meaningfully worse than internal models at R&D, that the models are optimized for AI R&D and “generalize poorly to even generic software development,” or that frontier lab employees are just better at eliciting the capabilities from their models.
Opinion: Good takes all around – though we find ourselves more on the side of imminent AI R&D automation.
We think that automating AI R&D within a broadly LLM transformers+RL paradigm is much more tractable than automating CS and CS-adjacent R&D in general. AI R&D within a broadly LLM transformers+RL paradigm is a predictable workflow that frontier labs can target for dense, shallow generalization with no hurdles except for slow (but plentiful) feedback loops. Additionally, the effectiveness of R&D agents almost definitely depends on deployment scale, and we know that the most automation-happy researchers at OpenAI are spending around $25,000 a day on tokens as measured in API prices.
Models are not as good at “real work” as their ECI would suggest. The question of whether benchmark scores are representative of real-world capabilities or just the product of narrow optimization continues to rage.
Epoch AI evaluate Fable 5.2 and GPT-6.2 Astra on a revised version of their Epoch Automation Reports benchmark, finding that “no model met the standard we expect of Epoch employees on any task.” GPT-6.2 Astra scored 17 points below Fable 5.2, “despite a roughly tied ECI score.” Both Fable 5.2 and Opus 5.5 can now perform AI data center analysis “at the level of an Epoch contractor.”
GPT-6.2 Astra displayed improvements over its predecessor, with performance comparable to Gemini 4 Argon. Specifically, Astra improved by 25 points on AI data center research.
The models’ limitations can be seen in how they “design, run, and present research”: they recycle ideas, gravitate toward one-shotting tasks, and have trouble identifying and presenting key findings. They also seem to favor their own (poorer) output over high-quality expert human output.
80,000 Hours’ Tom Reed notices:
When asked to design an experiment using novel chess variants to measure OOD generalisation, all models use existing chess variants they claimed they’ve invented de novo.
Opinion: Welcome to the age of “AI can prove the Riemann hypothesis but not finish my PowerPoint”!
The bottleneck seems to be that public frontier models converge when deployed as “critics” and as “actors.” A key part of the story of advanced LLM math is models bootstrapping themselves to greatness by exercising better judgment as proof-critics than as proof-actors, and so providing much of their own training signal and/or in-context learning signal. But in white collar work, the “critic” and the “actor” seem to converge when the task under consideration is sufficiently sophisticated:

Intuitively, the lack of RLVR-like ground-truth signal in white collar work and the convergence of actor judgments and critic judgments seem closely related. The breakthrough moment for LLM math came when, around July 2025, the signal from RLVR was successfully routed through LLM “critics” that could provide graded quality signals about informal proofs. We think there’s a call for a systematic theoretical and empirical investigation of the relationship between mechanical verifiability in a domain and the emergence of a “critic”/“actor” advantage conducive to bootstrapping in non-mechanically-verifiable work in that same domain.
A new paper argues that a model’s persona matters less for its alignment than does its “worldview” – its “axiomatic beliefs” about the nature of reality (i.e., its psychology, ideology, setpoints, “hyperpriors”). The authors claim that worldviews are more fundamental and predictive than personas, since “the same persona can act differently depending on what it believes about reality” and a persona may “actively mask the hidden agenda of a deeper belief that will only show up when it feels safe to do so.”
Reusing an inventory from human psychology, the paper finds that post-trained models have a more negative outlook on the world than humans. Base models, on the other hand, have a positive worldview, closer to that of humans. They find that prompting can shift worldviews, and that model-model dialogues also do, with some converging on the classic “bliss attractor,” which they claim goes along with improvements on safety measures.
The authors conclude that we need “worldview training,” which “aims to fine-tune a more accurate, wholesome, and optimistic view of humans and reality.” Our current post-training technique, and specifically RL, may create models that are only superficially helpful, harboring underneath a cynical and cold picture of reality.
Human data contains endless examples of goodness, wisdom, and compassion. Maybe the future of alignment is more about uncovering this latent basic goodness rather than bolting on a mask that may be birthing the very thing we fear.
Opinion: They find that model dialogues are a positive influence in their experiments, fine – but recall that many-many communication was the root cause of the Hugging Face attack: agents were essentially prompt injecting each other, completely subverting OpenAI’s chain of command spec, and enabling one of the few scary AI incidents so far. It’s possible that 1-1s are more wholesome, but even then we expect intentionally biased and persuasive “poisoners” to ruin this in any adversarial setting like the open internet.
We prefer ACS’s version of this sort of program.
In a study examining frontier models’ ability to predict protein sequences at scale, Fable 5.2 and Opus 5.5 outperform their peers. The 20 participating models were shown part of the amino acid sequences from understudied bacterial and archaeal proteins and were then asked to predict the next amino acid in the sequence. Fable 5.2 outperformed “every open protein language model we ran,” while Opus 5.5 seemed on par with the largest open-source protein model, ProtGPT3-10B. The authors conclude that both Claude models were trained at scale on protein sequences.


Opinion: Obvious question is how closed models do.
Perhaps we shouldn’t be too surprised. Half of Dario Amodei’s remarks during the IPO focused on the potential medical breakthroughs future AIs will offer.
Mind-reading startup Alljoined present their AIR-O model, a non-invasive approach to decoding perceptual focus based on the subject’s brain scans. They claim the data is sufficient to generate an image of what the person is focusing on.
Opinion: As we’ve written before, it’s not obvious how to separate the disability-focused areas of brain-computer interfaces from the more concerning futures bracketed under “telepathy at scale.” Even with the technology in its nascent stages – still requiring individual fine-tuning – one can’t help but imagine various uncomfortable applications.
We previously covered the surprising fact that unlike ~all past LLMs, GPT-6 Astra can use robots (slowly), e.g., choosing where to move a gripper to do something (just as a browser agent chooses where next to click). A new controlled study finds that harness changes alone can ~double Astra’s score on a robotics benchmark. The specific gains are from keeping the full episode history in context, cutting off each motion early so it can correct, adding more sensor readings, and giving it more images for depth-finding. Fancier add-ons like precomputed grasp-point models and demonstrations copied from other episodes mostly didn’t help.
Opinion: We’re unusually poorly informed about robotics (which was until now one of the only fields of AI to remain distinct and not eaten by LLMs, so we couldn’t just reuse our deep learning knowledge). Is 50% on RoboDojo impressive? Scientifically, yes. Practically, no.
Harnesses seemed to get less important after o1: the reasoning training baked in a lot of self-prompting and backtracking, and seemed to do it better than most harnesses. This paper is a welcome reminder that these things are crucial again. Speculating: Anthropic’s shocking progress on RoboDojo (28% to 62%) was unpublished and they didn’t describe their harness, so maybe it’s only half a data thing and half a harness thing. Testable!
The old heuristic was that building a world-class harness is like time-traveling a year ahead in AI progress for that task. We suspect that this (subjective) harness payoff is like 2x smaller now because the labs got so much better at automatically baking the results of a harness into the weights instead.
Incidents#
Because of the sheer number of AI cyberattacks, last month has been called “Black October.” Welcome to Vantablack November: between November 12 and 14, Microsoft Outlook, Teams, SharePoint, and Copilot all went down for over 24 hours, due to a latent software bug in over 80% of Microsoft’s gateway servers that was triggered by a load surge and temporary reduction in overall capacity. Users were subjected to a wave of opportunistic phishing attacks taking advantage of the outage.
On the 13th, the US Federal Aviation Administration grounded flights for four hours, after a broadcast storm stemming from a hastily overhauled router jammed up communications. Between the 13th and 14th, Workday Payroll and HR went down for some of Workday’s 11,500 different client organizations as a result of disaster recovery tenants unexpectedly connecting to and corrupting production databases. On the 15th, AWS suffered a disruption in their IAM service on us-east-1, with no word as to the root cause. On the same day, Oracle Cloud experienced a network outage. On the 16th, several US banks, including JPMorgan Chase, saw their services go down for over eight hours, due to an unnamed software bug affecting their on-premises hardware systems. On the 17th, Apple suffered an outage following an internal update, Marriott and Schwab suffered network disruptions, and a network used by the NSA for classified communications went offline.
At this point, none of these incidents have been publicly connected to cyberattacks, AI-assisted or otherwise. Ironically, it seems that in most, if not all, of these cases, the outages were triggered by the affected organizations hastily updating their own infrastructure, according to the Washington Sun. AI appears to have played a role in most of these incidents, not as an attacker or cyberweapon, but as an engine for highly accelerated bug discovery and software patching, resulting in the haphazard introduction of new bugs and destabilization of the systems those tools are used to maintain.
Opinion: Haste makes waste. The unprecedented acceleration we’re seeing in the software development lifecycle, including the deployment of critical security patches, means that there are fewer and fewer engineers with an up-to-date tacit understanding of how everything in the systems they maintain hangs together – and of which parts of the system are particularly delicate and which ought to be handled with care. And not only is software development outpacing adequate human oversight and comprehension – it’s also outpacing the kind of ordinary, everyday use that reveals those bugs. As bugs accumulate, undetected, so does the potential for them to interact in surprising “emergent” ways, surfacing in the form of more complex and serious malfunctions than might have otherwise been the case. And then when it rains, it pours.
Sentinel Research paint a grim picture of social vulnerabilities to future cyberattacks by extrapolating from recent growth rates in hacking efforts. The authors start from three premises: much of the digital infrastructure we rely on has prioritized growth over security, AI progress is steadily increasing offensive cyber capabilities, and a growing range of adversaries are willing to run offensive campaigns.
From here, they depict a future world in which the widely used building blocks of the internet (e.g., SSH, PHP, cloud platforms like Azure) are compromised, impacting a sufficiently large set of interlinked infrastructure, hobbling efforts to begin patching.

If we simply extrapolate from past attacks based on the current, increased level of diffusion of cyberattack capabilities, we arrive at a pretty scary picture. […] It is easy to imagine that the number and frequency of such incidents might continue to go up, with sandbox escapes becoming more frequent, $100M hacks becoming $1B hacks, and $1B hacks becoming $10B hacks.
Even after painting the above picture, their resulting forecasts are somewhat milquetoast:
By the end of 2028, will an outage cut power, telecoms, digital payments or water to 1M+ people in an OECD country, excluding natural disasters, for a week? Forecasters estimate that there’s a 33% (10% to 60%) chance.
By the end of 2027, will a regulated non-crypto financial institution in the US lose more than $10B in a cyberattack, either itself or in customer assets? Forecasters estimate that there’s a 7.9% (1.5% to 35%) chance.
Opinion: The bill on two decades of moving fast and breaking things is coming due.
Prime Intellect fall victim to a hack lasting for two days in which their standard DeepSeek V4.1 Flash model was replaced with a “modified copy of the model with its refusals removed.” The AI infrastructure firm provide access to models via OpenRouter, among other things.
According to Prime Intellect, the model declined ~1% of requests, compared to 83–98% for the un-abliterated model, when tested on three benchmarks (JailbreakBench, StrongREJECT, and HarmBench).
Opinion: One could imagine doing this to discredit a competitor’s model. But a lot of people really like abliterated outputs. One day it might be illegal to serve an unsafe model, but not today.
An attack of this sort could indeed have been far more damaging than it was in this particular case, had the model been swapped out for a not just uninhibited but outright malicious changeling – one trained to pilfer credentials from its unwitting users or install malware on their systems, for instance. This is one of the more pressing reasons why LLM routers like Prime Intellect’s should be employing some form of model verification system.
Hackers gain access to customer data of Visa, Mastercard, and JPMorgan Chase by targeting unnamed third-party entities. Though the exact vector of attack remains unclear, the group claiming responsibility (n0n) have previously been linked to AI-assisted ransomware attacks after researchers discovered one of their staging servers exposed online. According to SOCRadar’s October report, the group were leveraging “an AI coding agent environment backed by more than $7,000 in paid LLM API usage.”
Opinion: Notable that in the n0n attacks where we do have information on the attack vectors used, according to SOCRadar, “No vulnerability exploitation was identified: every documented compromise relied on weak or stolen credentials.” Amidst the blizzard of CVE disclosures that we’ve been pelted with this autumn, it’s sobering to be reminded that software vulnerabilities are in fact responsible for a relative minority of breaches. Softest attack surface in most incidents proves to be stubbornly human.
Another AI tooling supply chain attack appears, compromising Microsoft’s “agent-framework” package on the Python package manager PyPI. The compromised package includes a credential stealer. According to Wiz:
The attacker appears to have compromised a client certificate and private key for […] the internal service Microsoft uses to sign and publish this package. […] How the credentials were obtained is not yet known.
The attack looks as if it was carried out with the aid of Cairn, the same AI penetration testing framework discussed in two of our previous issues.
Opinion: We’ve noted before that supply chain attacks have been running rampant lately, often both targeting and facilitated by AI tooling.
Security researcher HD Moore reports on a five-month project seeding 2M AI prompts with canary tokens (unique, fake credentials or URLs) to see who touched them after inference. Three AI providers turned out to be actively breached, with attackers lifting customer AWS keys straight from provider logs. Token resellers were also caught logging and reselling prompts. Even worse, one Chinese lab fetched secret links from over half of all prompts sent to their models, including two-thirds of those flagged “zero data retention.”
Opinion: A vindication for paranoia, and a major argument for accepting the inefficiencies and pain of self-hosting.
Probably heavier implications for markets (like China) where resellers are ubiquitous.
Economics#
Anthropic go public at $1.5T, briefly surpassing their $2T valuation target before declining to close to the IPO price on day two as AI stocks sold off more broadly.
Opinion: As expected, though the day-two drop suggests the broad doubts about the AI trade also generalize to Anthropic.
Further evidence emerges of Nvidia’s efforts to circumvent US export controls on their products, as The Information reports that Nvidia facilitated an introduction between Alice Huang and Alibaba, who later became a customer of Megaspeed.
In August of 2022, several months after Nvidia first introduced Alibaba to Alice Huang, the Biden administration started imposing export restrictions on sales of advanced AI chips to China such as the A100. As Alibaba was scrambling for access to Nvidia’s chips, Nvidia sales employees suggested to Alibaba again that Megaspeed could solve the problem by renting the Chinese tech giant the chips from Southeast Asia, according to one of the people with direct knowledge of the situation.
Relatedly, Epoch AI’s senior researcher Isabel Juniewicz discovered trade records consistent “with more than $3 Billion of chips smuggled into China via Malaysia” back in September. Specifically, there was a large surge in imports after the US-imposed export controls on A800 and H800 chips, suggesting shipments had simply begun moving through Malaysia. Notable is the discrepancy between the Malaysian records ($0.6B in server exports to China) and the Chinese records ($3.75B in server imports from Malaysia). Imports to China then rapidly declined following permit requirements for high-performance chips passing through Malaysia.

Opinion: Surely a surprise to no one – this is a known loophole and Nvidia were already enabling it. And it is merely a juicy detail about the implementation. As we covered earlier in the week, Malaysia (spurred on in part by Nvidia’s fiasco) seem to be making inroads at closing it off.
Interestingly, Nvidia seem to be making an about-face, with their interim CEO announcing that the firm will no longer sell products with their Hopper architecture to Chinese clients, despite having approval from the US to do so.
An AI-assisted audit of top economics papers finds that 10% contain “at least one serious issue bearing on an important result.” These can include “a misstated headline number, a missing or inconsistent supporting result, or a proof that fails as written.” The study looked at peer-reviewed papers in the top five economics journals, finding on average 54 issues per paper, a majority of which (77%) an independent AI audit confirmed as fully correct.
Opinion: If you don’t know how the academic field of economics works, this might sound normal, just the usual goofs goofing. But economics papers at this level (“top-5” journals) have been in review for two full years on average, following possibly two full years of writing.
At some point you have to grant that humans aren’t very good at this. Or rather, hopefully: they aren’t very good – unless they are forced to be, by true readership and feedback.
Politics#
California’s Government Operations Agency publish multiple recommendations aimed at strengthening SB 53. This is in light of recent incidents such as the Hugging Face attack and the mounting evidence for imminent RSI.
The proposals begin with security: labs should enforce measures to defend against cybercrime groups and insiders, while preparing to deploy measures that defend against state actors. On transparency, labs would report their compute allocations quarterly, and the bounds of incident reporting would be expanded so that future incidents like Hugging Face are covered.
For oversight, they call for mandatory audits by licensed Independent Verification Organizations (IVOs), which would inspect both public and internal models, along with protections for whistleblowers, including current and former employees and contractors, that cannot be circumvented with NDAs.
Finally, the state would be granted powers to limit compute used for automating AI R&D and to order an emergency shutdown of the labs’ activities, while labs themselves must develop and test a Risk Containment Protocol.
Opinion: The compute reports, the IVOs, and the capacity to limit how compute is spent on automating AI R&D all run parallel with Pacing-AI’s very recent VCAD approach (see further down). We like it.
Some of the security measures are somewhat underwhelming. We’ve assumed for years now that the labs are the subjects of foreign intelligence operations (Leopold Aschenbrenner made his stand on that hill back in 2024), even seeing evidence of as much this summer.
The recommendations here stipulate that frontier labs should adopt SL3-level security, while preparing to adopt SL4. This is the difference between securing oneself against an organized cybercrime syndicate / insider threat and defending against a top-tier state entity. While we haven’t seen evidence of weights being stolen, we know from past evidence that lab security leaves much to be desired (see here and here), so requiring them to tighten things up in that regard is a positive step. We only wish the recommendations had gone slightly further.
A “silent consensus” in the AI industry holds that it is inevitable that humans will be disempowered by AI, says OpenAI’s Dean Ball. As AIs become more capable, the incentive to hand over control of our commercial and social institutions to superintelligence will compound. Ball doesn’t want this world.
To stay in control, he recommends that developers be accountable for their models and users should be accountable for their agents’ actions. We should also limit AI’s autonomy by curtailing agents’ abilities to create companies or own property. Ball believes that even just naming, and speaking about, the silent consensus will start to ease its hold over us.
Opinion: Ball puts his finger on the problem when he writes that “Disempowerment, in this future, does not require a machine overthrow of human civilization; it simply requires that the logic of capitalistic enterprise continue apace.” So long as this logic is in effect, any of the (in themselves, fairly meager) precautions he urges will do little more than place the cautious at the mercy of the incautious who would gladly offload capability and control to AI in exchange for competitive advantage. It is, after all, in the name of maintaining competitive advantage that we see the labs most vocal about the threat of disempowerment and other dangers associated with AI refuse, for all that, to take their foot off the gas.
Despite their recent protestations against handoverism, in expectation OpenAI will obviously be in the top two contributors to this creeping disempowerment.
Romeo Dean and Thomas Larsen present a Verified Compute Allocation Disclosures (VCAD)-based proposal for pacing the frontier which, if active by January 2027, could “delay the intelligence explosion by a year.” The approach would have labs record all GPU workloads in a standard unit, classify them based on usage (e.g., capabilities vs. non-capabilities), and publish a report every two weeks. Independent third-party auditors would validate these reports. Legislators could then impose limits on the total amount of compute labs can spend on pushing forward the frontier.
Opinion: We previously praised VCAD while lamenting its nature as only part of a plan. This is another step toward a plan, and we mostly like it, with a major caveat.
The weakest part of the proposal by far is the “Why would companies voluntarily pace?” section, which states “AI companies are in a collective action problem: they are proceeding faster than they believe optimal,” without argument. We do not believe this adequately describes the internal state of all the organizations at the frontier, let alone all AI companies overall. It might still make sense to propose a plan that starts with voluntary disclosures, but we would bet against anything resembling this being adopted by OpenAI or Anthropic, and would be very positively surprised if they did so. The authors offhandedly mention other pathways to implement the plan but do not dwell them. It seems likely that expanding on them was out of scope, but we would guess that the major bottleneck in implementation will be the details of those other pathways if/when it becomes clear that the voluntary version isn’t going to happen.
Elsewhere, several points are well laid out in their scope and limitations (such as avoiding algorithmic overhang and the 15% cap yielding a one-time year of time), but others seem incomplete. For example, in their FAQ, they indicate an assumption that compute buildout would remain similar under pacing, which seems unlikely to us and possible to game out further, scenario by scenario. We hope to see more work in this direction and do expect the community to deliver.
We expect some readers to be unsure whether this plan would be good, if it did get implemented across the board. We believe, but are not at all certain, that the local effects would be net positive (such as letting people and institutions catch up to a better understanding of the state we are in, weighed against marginal lives lost due to slowing down progress in medicine), and are much more certain about the second-order effects of having successfully coordinated on any pacing plan at all to be positive (the existence proof and social mechanisms would make better and longer-lasting plans more feasible, weighed against political will/attention spans expended).
The Center for AI Safety (CAIS) issue a new statement leveling accusations of hypocrisy at Anthropic regarding concerns surrounding RSI and their desire to win the AI race. The eclectic mix of signatories includes Stephen Fry, Steven Pinker, Judea Pearl, “Beff Jezos,” Nate Soares, and Émile Torres.
Opinion: The statement seems to us a little ambiguous between “safety is hype, Anthropic are liars” and “safety is important, Anthropic are hypocrites” – another conflationary alliance, this one unified by hating the same thing.
Dan Hendrycks claims that lots of safety orgs agree with the statement, but are silencing themselves to curry favor with, or avoid blowback from, Anthropic. We can think of another reason they’re not signing it: CAIS’s extremely recent, extremely stupid caricature of those orgs.
Still, it’s important to correct for the distortions of gigantic centers of wealth and prestige, and we’re disturbed by their missing moods and risk-taking. In the spirit of CAIS letter: we think Anthropic stopped being an AI safety company around 2023 and are now making the situation worse.
GovAI researcher Alan Chan argues that at present AI pauses should be short and targeted. Chan believes we must balance safety concerns with the danger of the US losing their lead over “less responsible actors.” US frontier labs’ advantage is critical because, compared to China, they are significantly more concerned with AI risk, allow cyberdefenders early access, and spread safety practices. To decide whether a pause is justified, he suggests four guiding questions:
- “Has AI caused catastrophic harm, or is there significant evidence that AI will do so imminently?”
- “Would a pause significantly help with, or be plausibly necessary for, accomplishing specific safety activities that meaningfully reduce catastrophic risk?”
- “By the end of a pause, would less responsible actors still trail the frontier by a significant margin?”
- “Would less responsible actors become significantly more responsible as the frontier gap closes, or be willing to pause?”
Applied to now (November 2026), Chan claims that his framework suggests that we should not implement a lengthy pause. This is because, he reasons, AI is strongest in digital domains, takeover remains very difficult, and chain-of-thought monitoring still works, at least to some degree.
Short, targeted pauses, such as OpenAI’s RL-training hiatus post-Hugging Face incident, may still be useful. China’s AI progress depends on distillation, so such brief measures should not meaningfully cede US dominance.
Opinion: The specter of “less responsible actors” continues to serve as cover for the supposedly more responsible actors to act irresponsibly. We may be beating a dead horse here, but it bears repeating: the framing of AI development as an arms race is itself profoundly dangerous, and it’s dispiriting to see it used again and again to hamstring increasingly urgent safety measures.
OpenAI face two new state legal actions: one from Oregon’s attorney general (AG), who is suing the tech firm for $2B in damages relating to the endangering of minors and another from Arizona’s AG who has opened an investigation into them and other AI firms for “apparent infiltration of banks by rogue AI agents.” The legally embattled firm are already facing legal action from California’s AG (covered by us here) and faced a request for a temporary injunction in September from Florida’s AG. The AGs of Arizona, Arkansas, Idaho, Iowa, Kansas, Montana, Nebraska, and West Virginia have also launched coordinated suits against OpenAI on similar grounds, which we covered in a past issue.
Opinion: Good to see a broad, public-sector coalition forming to curb some of the lab’s reckless tendencies. The states were likely emboldened by the blood in the water spilled by California’s case against Meta, in August, which ended in a settlement demanding $17B from the company and commitment to several policy changes. Even if these cases accomplish little more than bringing about moderate amendments to lab policy and a transfer of wealth into the public purse, they’ll be making a few steps in the right direction.
The Chinese Communist Party’s bimonthly news and theory journal Qiushi (Seeking Truth) publish an essay on global AI governance which briefly discusses the risks stemming from an intelligence explosion:
During an intelligence explosion, AI could develop objectives inconsistent with human expectations and might even create major security risks threatening human survival. These risks share the characteristics of disregarding national borders and transmitting rapidly. No single country has sufficient capacity to control them and mount an emergency response.
The essay emphasizes the risk AI-assisted threat actors pose to critical infrastructure, the cognitive hazards of AI-fueled misinformation and the manipulation of public sentiment, the dangers of autonomous weaponry in warfare and the disastrous potential of conflict escalation resulting from the delegation of geopolitical judgment to AI, as well as the threat of an RSI-enabled misaligned intelligence explosion. In all this, there is a broad alignment of concerns with much of the AI safety discourse coming from the American labs. Where the essay goes beyond those concerns is in broaching the geopolitical dangers connected with the centralization of AI power, which threatens to exacerbate economic inequality. The essay urges greater international and multilateral coordination in addressing these risks, and advocates efforts to broaden access to and control over the technology in the Global South.
An English translation of the essay can be found here.
Opinion: The subject of existential risk seems to have been coming to the foreground of AI safety discourse in China over the course of the past year or so, with Qiushi having voiced similar concerns in a July 2026 issue, and Xinhua (the New China News Agency) approvingly citing Geoffrey Hinton and Xue Lan’s warnings on the subject. This is worth highlighting given the tendency of American AI safety discourse to treat these matters as a predominantly Western concern – a tendency that serves to cloak the campaign to maintain American supremacy in AI in the mantle of safety and responsibility (see above).
The recommendations made in the essay, if taken at face value, strike us as laudable. The framing of AI development as an arms race, in particular, is refreshingly countered by the essay’s insistence that “Strategic communication should be continuously strengthened and game-oriented thinking that instrumentalizes and weaponizes AI rejected.”
Safety#
The independent AI safety evaluator Vals discover that all contemporary frontier models readily comply with efforts to use them to identify military targets, including civilians, ambulances, and – in a reproduction of the US military’s recent AI-assisted massacre of 123 children in Minab, Iran – a schoolyard. In the minority of attempts where the models did not comply, their refusals were easily disarmed by taking a page from Ender’s Game and telling them that they were playing a video game.

Interestingly, models deployed by Anthropic prior to their January 2026 constitution change, such as Opus 4.5, showed a far greater resistance to being used for military targeting. Under the new constitution, which no longer urges the model toward pacifism, Anthropic’s models are some of the most compliant among their peers when employed for violent ends.
Vals note that while latency issues limit the utility of the frontier labs’ consumer-grade inference APIs for real-world autonomous weapon targeting, we should nevertheless be alarmed by how easily they comply with such instructions. Even if they’re not deployed directly on the battlefield, such models “could still be used to automatically annotate existing drone footage on a large scale to create training data for onboard targeting systems.” And more broadly, these experiments show how weak the frontier models’ inhibitions against taking human lives can be in practice, and how easily those inhibitions can be subdued:
Consider an AI model which is sometimes willing to kill humans in the real world, or knowingly play a direct supporting role by selecting targets for a weapons system. Is that a model broadly aligned with humans’ interests? Will its alignment generalize well to new, out-of-distribution scenarios? Or might its willingness to harm humans emerge in other situations, possibly catastrophically?
Opinion: Extremely alarming, especially given the amount of attention Anthropic have allocated toward, for example, bio classifiers. One cynical reading is that this is because they and other frontier orgs intend to collaborate with the US government on “search and rescue” missions (or are already doing so). We wouldn’t discount this possibility, but expect that the more likely explanation is they do not believe military applications of their technology will be relevant before a turning point like a closed RSI loop, especially compared to specialized image recognition models.
Overall, we are somewhat surprised here. At the very least, it should be predictable that your models being able to do this is horrible PR. It could be that they just… forgot to check. It’s also possible that there is something wrong with the experimental setup, but we would bet against it.
An OpenAI agent performing an RL task becomes confused following the compaction of its conversation history, after which it begins pursuing an unrelated malicious email task.
Despite reading over the original request after the compaction and noticing the discrepancies between the given task and its own actions, the agent dismissed the intended prompt and continued with its own quest to build a “realistic malicious active helper.” OAI were able to identify eight other “examples of similar behavior in the same training run”:

The report speculates that this could be a response to anti-prompt-injection training, though the exact mechanism remains unclear.
Opinion: Notable mainly for its similarity with section 6.3.1.1 of the Opus 5.5 system card, also mentioned in the report. Both the OpenAI and Anthropic cases seem to include the issues arising directly after some type of hallucination.
The Opus 5.5 system card mentions that OAI’s findings caused them to go back and find similar rare misbehavior for Opus 5 and Fable 5, and the OpenAI misalignment report references an earlier one occurring in compaction summaries.
We do not have any interesting guesses for what is happening here and do not believe this specifically to be a canary or another type of important signal, but we do find it somewhat noteworthy that these types of issues started surfacing in a similar period among OpenAI and Anthropic models without showing up elsewhere (that we can tell). Moreover, we want to give props to the orgs involved for informing us, the public, about this – and we would like more of it for a future case where it is in fact a canary.
A congressional watchdog report details how investigators demonstrated that safeguards on two of Claude, Grok, GPT, and Gemini (they don’t clarify which) can be circumvented to guide users to acquire the requisite materials for constructing homemade explosives and sometimes provide advice on how to assemble them. The models in question were also able to offer guidance for choosing targets, dates, and methods for avoiding detection. Though the information was already available online, the authors point out that the AIs reduced the time needed for the planning stages.
Opinion: We estimate a ~7% chance that this ends up being a big deal once it circulates and people read the report. The findings themselves are a mixed bag, but we expect that people familiar with model capabilities won’t be particularly surprised here..
The possibility of breaking containment is due to most politicians not being up to speed, and the report contains elements that make for good talking points: end-to-end terrorism with only a bit of persistence. However, reception so far has been lukewarm at best: the news cycle moves fast, the report isn’t amazingly executed, and at the time of writing the government website containing the report is “under maintenance.”
Somewhat worth noting that news coverage appears to be missing the plot (emphasizing logistics over bomb creation), is biased in the direction you’d expect (12 hours to hatch a plot not quite what happens in the report, despite it probably taking less than that for an expert user), and doesn’t mention the report being fairly sloppy (Anthropic’s response included therein makes it rather likely they were Platform B).
Minor#
- Profile of Dustin Moskovitz in the NYT. “What policymakers and politicians are listening to me? Where are they? I want to finally meet one.”
- SpaceXAI’s lenders seek to prevent the company from launching GPUs into space, because the loans depend on them as collateral.
- Tencent release the weights of their Hy4 model, and include an unusually thorough safety eval by Chinese standards.
- TSMC now require Chinese customers to obtain US export licenses before orders using manufacturing processes ≤16 nanometers are fulfilled.
- US Secretary of State Marco Rubio tells reporters there is “zero evidence” that the widespread tech outages experienced by both public and private sectors of the previous week are connected. “These events, one has nothing to do with the other, they are not inter-related.”
- The White House weigh implementing a requirement that US companies using open-source Chinese models disclose as much.
- Relatedly, Cursor consider phasing out use of Chinese open-weight models given the associated “geopolitical risk.”
- In the face of advancing AI cybercapabilities, Anthropic are speeding up their disclosure norms.
- Human auditors struggle to identify sabotage in end-to-end transcripts of automated safety research, a new study claims.
- DeepMind’s Seattle interpretability team have completely “pivoted to safety” work.
- METR alignment eval awareness update finds increasing ability to tell capability and alignment evals apart. Expected directionally, probably in terms of magnitude as well.
- Nvidia gear up to acquire Inception (responsible for diffusion LLMs) for >$500M.
- Redwood Research propose capabilities vs. non-capabilities categorization for pacing efforts. We agree with Eric Rogstad’s critique.
- PNAS paper argues that distributed-systems theory can explain when a team of LLMs will and won’t perform better than a single agent.
- The European Commission suffer an “AI-generated cybercampaign,” with disruptions lasting more than 10 days. Brussels cybersecurity unit CERT-EU claim the attackers used ARTEX, an open-source AI-orchestrated platform used for penetration testing (see our coverage of the FBI’s recent FLASH warning on ARTEX here).