This is our weekly newsletter of AI developments. Sign up here.

TL;DR#

  • Anthropic find an analogue of working memory in Claude. Many suggestive but inconclusive ties to human cognition, consciousness, and selfhood.

  • Fable suddenly re-released on general access. Dario nominally sidelined.

  • New incident reporting bill from Rep. Moran. It’s good.

  • OpenAI proposing giving a 5% stake to USG

  • Epoch test frontier models on an obscure (therefore likely OOD) paper boardgame and see the expected collapse: bad at exploration, strategy, and self-reflection.

  • Rohit Krishnan argument which would explain a lot: are frontier labs incentivised to own the whole stack?

Economics#

Annals of winner-takes-all: Dylan Patel says Anthropic's margin on an Opus 4.8 API token is north of 80%, and that it is net-income profitable excluding stock comp in Q2 2026, potentially profitable including it by Q3.

Opinion: Genuine profitability would be a massive deal, especially with OpenAI likely bleeding money still. This might explain OpenAI stalling their planned IPO - they will look a lot less appealing if they are going to market at the same time as a profitable competitor while still losing enormous sums.

It would also substantially reduce Anthropic worries about compute access - with 80% margins they would have a lot of room to bid up compute if necessary, and if nobody else achieves those margins they would be in a strong position.

Meta planning on developing a cloud business to sell some excess compute. See also: Amazon to raise GPU prices by 20% from July 1st. Semianalysis expects Meta to sell compute to Anthropic imminently, and expects this to give them enough assurance to accelerate rather than decelerate compute spending.

Opinion: Superficially very bearish on Meta. One read is that Zuckerberg saw the numbers SpaceX were getting from Anthropic on their rental deals and presumably thought that that was currently or could soon be a better deal than their own internal use cases, and wants to be prepared to pivot to that if it makes sense. If you were skeptical of the hyperscalers’ cloud business models that would be a sign that Zuckerberg at least isn’t and wants in on the game.

Another read is that Zuckerberg was feeling the pressure from investors to get some return on all their capex, or wasn’t seeing the return internally, and that this suggests they may scale back spending in future. The market seems to have taken things this way, as semis were down on the news coming out. We will see - if this hypothesis is true we should expect to see declines in their forward looking capex.

As long as compute prices are going up or staying flat, and lab revenues keep growing at high rates, that’s the purest signal we can get that demand is real and persistent. A decent case for the bullish interpretation.

Opportunity: If the bullish interpretation is right then it seems like a great time to buy the dip in memory/neoclouds. The caveat is that Semianalysis are institutionally extremely bullish so it would take a lot for them to say something is a sell signal.

Neat site mapping out expected memory demand - expects severe supply constraints through 2030.

Related: SK Hynix removes price caps from long term memory contracts.

Opinion: In conjunction with the above, if Anthropic has a great business model, and hyperscalers selling compute to Anthropic also have a great business model, then there’s a lot of room for memory and chip prices to get even crazier.

The reason memory makers have long sought out long term deals has been cyclicality - protecting their revenues from busts in memory prices. Usually this has come via agreeing price floors in exchange for accepting price caps, upper-bounding how much they will receive. If Hynix has the leverage to negotiate away those price caps, and values that above protecting their downside, that is a pretty strong signal that they expect the good times to keep rolling.

Annals of narrow and decentralised AI: Bridgewater and Thinky fine-tuned Qwen and report big success on a collection of “easy but tedious for humans” tasks. (e.g. “Given a financial article, classify whether it is relevant to a C-suite investment professional”, “Given an investor's question and a research document, classify whether the document helps answer it”, “Identify where boilerplate content begins in a document”)

Opinion: Interesting to see some of the details of their attempts to improve performance here: e.g. reframing binary relevance into three classes (relevant-and-interesting / relevant-but-boring / irrelevant) gave big gains before any fine tuning. Often "the model is bad at task X" means "task X was posed badly."

The case for fine-tuning over using the best LLMs here relies on their ability to convey “taste” via examples for this task in a way that they failed to do via prompting. To the extent that taste is domain specific and non-transferable across tasks this may persist for a while, but if it isn’t it just seems like another hill for the models to climb. For now, this type of classification task remains the canonical case for finetuning.

Thoughts on the equilibrium of the AI ecosystem (Anthropic/OAI lead and Chinese open source models lag but much cheaper and sufficient for many tasks). Suggests that frontier labs are incentivised to own the whole stack while other companies need to find their niche.

Opinion: Slightly overconfident but lays out some relevant dynamics pretty nicely. A major issue is that owning the whole stack is currently out of reach for labs - they simply cannot underwrite the necessary capex, and so must source their compute from those who can. Important questions:

* Will frontier models remain the dominant models used for all tasks? If cheaper models suffice, the market (in terms of total token demand) for frontier labs’ models shrinks, but we’ve seen little of this so far - almost everyone wants to use the best available model for most of their tasks right now.

* How can frontier labs extract value from an ever more expensive frontier? Right now the answer is “high gross margins on APIs” but does that change if models become more capable? Do capabilities get hoarded in-house or in selective partnerships with industry players?

Capabilities#

Fable 5 is officially back, and as general access.

  • Access was restored after a June 30 letter from Commerce Secretary Howard Lutnick to Anthropic's Tom Brown (not Dario Amodei).

  • We do not yet know what terms Anthropic had to agree to. What is public is that the export controls on both Claude Fable 5 and Claude Mythos 5, imposed on June 12, have been lifted.

  • Alex Stamos' reading: because of the freakout, he argues, "US labs now have to make a much more conservative precision-recall tradeoff on cyber refusals." The practical consequence is that US models may become less useful for defensive cybersecurity unless the user can get into the trusted group where those safeguards do not apply. Stamos also reads Anthropic's announcement as saying something about who should have been making the call. In his words, "CAISI is the group that is supposed to actually make these determinations, not the political actors in the White House." CAISI, he notes, had been positive on the prior safeguards.

  • Brundage, meanwhile, cautions about the same CAISI point. Even if CAISI is the right body to evaluate the safeguards, the public has not really heard from CAISI about what it did or did not conclude. For now, outsiders mostly have to take Anthropic at its word. Brundage takes it as good news that Anthropic is working toward an industry standard, but argues that knowing that "CAISI was involved" is not enough. We should still expect a public framework and public evaluations, redacted where appropriate.

  • Even the more stringent safeguards are insufficient to prevent some users from jailbreaking the model, though: 20 hours after release, we already have initial reports of success, though this is unstable enough that long-horizon tasks while jailbroken still somewhat-reliably trigger the guardrails.

  • A peek at the true CoT(?). This is not neuralese but it is distressingly closer to one than previous models.

AISI say the obvious: an AI's performance is a curve of capability over compute on the x-axis, not a single score. “raising the budget from [60to60 to 1250] increases the estimated horizon from 2 hours to 14 hours". However these are log-log!: the 80% time horizon is rising as the 2/3 power of the number of tokens: every 10x in tokens buys ~5x the time horizon.

Opinion: Graph B suggests something like Toby Ord’s ‘impossibly-test-time-compute-expensive singularity is near’ idea. We think this is most likely true for buying reliability on very difficult standard cognitive work in verifiable and semi-verifiable domains -- enough tokens will now or very soon buy you reliability -- but not about buying paradigm-shifting scientific breakthroughs.

Also of interest: what does graph A look like in FLOPS instead of tokens? The jumps in Graph A correspond very closely to popular estimates of jumps in base-model size. It’s plausible that a FLOPs-based version of graph A would have much smaller jumps.

Shame that these graphs are for evals of offensive cyber capability, an area where capabilities are sometimes muddled by layers of alignment-taxes, undertraining, and counter-training. AISI does access special offensive-cyber-enabled versions of models, but no one exactly what this means.

Note that this setup is basically an upper bound since they’re allowing arbitrary numbers of retries.

Superforecasting as eval: “If AI hits top-human level forecasting and then flattens off, maybe there’s something special about the human level, and [Sayash and Arvind] will also be right about superpersuasion, super-research, etc (at least for the near-term). If it keeps going, reaching heights far beyond the human maximum, then we should be concerned that it will do the same thing in other skills too… Here I am, writing about how other people will be dumb and stubborn and fail to trust the AI superforecasters enough - and I still reject them the first time they really challenge my worldview.”

Opinion: It seems plausible that we will see AI hit top-tier forecasting performance in aggregate through somewhat different mechanisms than top humans use - using much more data, outperforming on base-rate calculations, etc. We can see hints of this already with bots performing relatively better in more data rich environments like financial forecasting.

If this is the case we should expect to see a centaur era in forecasting, where the best human-machine teams can still beat machines or humans alone. This era being short would be a significant update against there being something special about human intelligence.

Epoch introduce EBR-Bench -- making frontier AIs play ‘a somewhat obscure, largely text-based campaign board game requiring a mix of strategic deck-building and tactical turn-by-turn play’. Epoch find that current AI systems are bad at exploration, quite bad at strategy, and very bad at learning on the fly to fix the previous two issues. “Many models stick to a single archetype in all their exploratory playthroughs”

Opinion: Greg Burnham at Epoch has been talking about testing frontier AI on a challenging not-previously-digitized, non-visual boardgame for a year now as his one idea for ‘objectively scorable non-visual thing frontier AIs might be worse at than good-but-not-great humans’. We think this is the strongest negative evidence in months against frontier models being almost-AGI, and against the worldview where frontier models are smart the “same way that humans are smart but just not genius-level yet”.

Also reduces our estimates of near-term RSI risk, since the most plausible view of frontier AI research is one where pushing the frontier is a research-management campaign at heart. (The reduction isn’t dramatic because it’s still possible that the research-management campaign pushing frontier AI forward is static-rules enough that Anthropic’s effort to document their workflow and train Claude on it will bring the campaign in-distribution.)

Very exciting result overall. We’ve been waiting for this one with bated breath and uncertainty.

Also the release of EdgeBench, very long-horizon coding, math, video game, and white-collar work challenges deigned for testing how agents learn from feedback over a 12-72h run. Findings include extremely regular aggregate performance over time across domains (but not across individual runs) and task-learning speed doubling every 3 months.

Opinion: Unlike Epoch’s new esoteric boardgame benchmark, EdgeBench tests hillclimbable optimization of a solution-artifact (a code, a policy, a proof) in environments with dense authoritative feedback. The environments are also largely instances of very in-distribution environment types.

Combining the EdgeBench results and Epoch’s results above gives an overall picture very aligned with P3’s modal view of generalization in current frontier AI:

1) Even as hillclimbable in-distribution skills get much better from model to model out-of-distribution skills only modestly improve
2) Even as the skill of test-time hillclimbing dense feedback instances of in-distribution environment types gets much better from model to model, learning in sparse feedback out-of-distribution environments does not really improve.

Argument that the future depends enormously on how continual learning is solved: persistent in-context memory versus efficient on-the-job weight updates vs the current nonsolution of pretraining a new model. In-context favors end-users and existing enterprises (‘ "instances" of models that are owned and controlled by the hundreds of thousands of enterprises across the world, who have slowly built a model of how their company, customers and colleagues work’); in-weights updates favor labs.

Opinion: Doesn’t put any attention on the case this doesn’t get “solved” (the current “episodic” nonsolution continuing). Still the main question for people more focussed on concentration of power than us.

Fable shows a large leap forward on CAIS’ Remote Labor Index. “The frontier has more than quadrupled in under eight months”. Tasks include “Re-create the client's existing engagement ring with its emerald-cut center stone swapped for a marquise cut, delivering an updated 3D model”, “Produce a ~60-second flat-design 2D animated advertisement for "Skyline Tree Services," set to the provided voiceover, that walks viewers through the company's tree-care process and builds trust in the brand”. However, “Today's AIs still fall short of professional quality on most projects; none of the three Fable 5 deliverables above would be accepted as finished work.”

Opinion: RLI avoids some pitfalls: 23 different work categories, manual grading, they checked the RLI tasks’ completion-time distribution against a random Upwork sample, and the Elo design beats floor effects. However, read the description again: it’s not measuring good work. It also only measures zero or one-shot (full automation, and so it underestimates economic use), zero-shot completion of self-contained, easily evaluable, standalone freelance deliverables is thus going for a small subset of all remote work. Excluding back-end coding removes work agents are good at (underestimate).

On the other hand, I am using Fable to do all the work for a major building permit task, 20 pages of drawings...

Claude Sonnet 5 released. Better than Sonnet 4.6, allegedly cheaper than Opus 4.8.

Opinion: New subagent of choice but far more underwhelming than e.g. the huge Sonnet 3.5 and 3.7 drops of 2024.

NVIDIA’s TwoTower converts a small autoregressive LLM into a faster diffusion-based parallel generator that uses a separate context model and denoising model. Allegedly able to get near-par quality at 2.4x generation speed, and training the parallel generator requires only 8% of the training data (and presumably compute?) used to train the original LLM.

Opinion: If Nvidia can make this work with a frontier-sized autoregressive LLM it would be very economically valuable. As software enterprises start using LLMs for more and more long-horizon tasks, generation speed increasingly goes from a matter of convenience into hard time-is-money value. But overall I doubt it, and Qwen is right there for anyone really bottlenecked on tok speed.

Annals of Chinese hype: GLM-5.2 ECI is out: it’s below Gemini 3 Pro (Nov) and more pertinently also below Qwen 3.7 Max.

Opinion: We continue to not prioritise Chinese AI despite press-driven and agenda-driven mania.

AI politics#

Loss of control from AI is now a top-4 issue for Hill staffers of both parties (#3 for Dems and #3 combined).

Opinion: The game is afoot!

Win for Leopoldism: CIA announces a reorganization to adopt AI systems more centrally, primarily for cyberoffense. Framed as taking smart risks. Director: “It would be… not misplaced to refer to their capabilities as akin to digital nuclear weapons.”

Opinion: I still don’t know if securitisation was inevitable and I still don’t know if this was a necessary step to serious people getting behind the relevant levers at all. But this is a firm step towards Situational Awareness land, and the MAIM escalation ladder as default substitute for an actual international agreement. If the spooks ruin this, we will regret it.

OpenAI proposing handing over a 5% stake to the Trump administration. Some overlap with the previously proposed public wealth fund, but specific wording and implied details differ.

Opinion: Clever to get ahead of the 50% expropriation. If it gets them privileged status as the most favoured nation among AI labs it would be very cheap at that price.

New incident reporting bill drafted for Congress. Smaller than Obernolte-Trahan on purpose, just winning one nice thing rather than going for the whole package. Bold move: covers any models Commerce designates by capability threshold rather than training compute or revenue. Very slightly overlaps/competes with Obernolte-Trahan, but only on one point. “Not later than 180 days after… the Secretary, in consultation with, as appropriate, the heads of relevant agencies as determined by the Secretary, artificial intelligence model developers, other relevant private-sector entities, academic, technical, cybersecurity, national security, and public safety experts… shall promulgate regulations that—(A) establish capability or other thresholds that determine which artificial intelligence models and model developers could pose significant risks to the national security of the United States or to public safety”. GAAIA bundles incident reporting with a three-year preemption of state laws; Dem Commission on AI came out in opposition hours after release. Moran strips incident reporting out so the least controversial piece can move.

One major clash: unlike OT, safe harbour for information submitted: may not be used by any federal, state, or local government to regulate or to bring an enforcement action against the developer, with a FOIA exemption, inadmissibility in civil/criminal/administrative proceedings, and a bar on disclosure under state law.

Opinion: Good! Additive. They will amend the few direct clashes if either bill passes. Theoretically catches “brain in a basement” small but highly capable labs.

Altman argues in the FT for a US-led international (government) control of advanced AI over labs or third party organizations, so that “everyone on Earth should benefit from this technology”. Cites aviation safety, global financial standards and atomic energy management as previous successes of similar ventures. Pairs strict controls with broad access.
“US-led international forum that establishes accepted standards, provides expert and impartial analysis of capabilities and risks, and makes the technology available to nations and companies that participate and follow the rules. This forum might include government representatives, independent technical experts and others”

Opinion: He’s been talking about this for a while. Knee-jerk cynical reading is that this is pushing for the unlikely thing to split the vote away from likelier things, but it’s not a bad idea. Should check if any of his divided teams are actually pushing this.

Safety#

How do AIs change when they only interact with each other? Paper finds that conversations between LLMs tend to settle into model-pair-specific attractor states with predictable style and behavior across topics. They also find asymmetry: some models strongly pull partners toward their conversational traits (somewhat surprisingly, Haiku 4.5 had the strongest gravity).

Opinion: Might be a real problem for autonomous remote workers in open-world environments. Abstract sounds sketchy because ‘open ended conversation’ stacks the deck in favor of irrelevant attractors, but the actual study used a collegiate-debate style setup. The attractor states aren’t clearly absurd or dysfunctional, but they demonstrate a potentially worrying content-arbitrariness and predictability in mixed-agent interactions. But the predictability is also kind of a relief.

Research on self-play debate finds that accuracy improves in math problem solutions, but also incentivizes dishonestly poking holes in correct solutions. This is a direct result of the experimental setup: a proposer, critic and a weaker judge with the proposer and critic in a zero-sum game, so the critic uses whatever tools are at its disposal to convince the judge to reject the solution. Tentatively concludes that weaker judges are a worse idea than using frontier-capability ones.

Opinion: Unlike most debate work, this looks at debate during training, not inference, and finds that capabilities go up alongside misbehavior. Bad news for debate as safety strategy, and it’s explicit about how the maximally capable judge only works as an approach if the judge is more aligned/trustworthy.

Incidents#

Anthropic admits to having a system in Claude Code for identifying Chinese users and flagging possible lab connections in a way which was hidden from users, without consent. They claim it was done to defend against distillation and unauthorised reselling.

Opinion: Seems like they could have announced this instead of doing it secretly and not lost much.

Classic list of empirical evidence of reward hacking / misalignment, the Specification Gaming Database, updated. No longer crucial, but still fun to read.

Opinion: Reward hacking has an odd relationship to safety because it’s also a capabilities failure and there’s massive economic reasons to solve it.

Epoch summary of software vulnerability discovery at scale. June registered 3.5x as many high severity discoveries compared to pre-Mythos months among 21 notable organizations.

Opinion: No attempt to separate out 1) old bugs discovered by AIs, 2) AI bugs discovered by AIs, 3) old bugs discovered by newly urgent human security workers, 4) AI bugs discovered by newly urgent human security workers. But it doesn’t matter that much.

Minor#

  • CyberBench, built on the shoulders of the very load-bearing security eval CyberGym, releases results. GPT 5.5 takes the lead, GLM 5.2 in second, Opuses at the bottom but due to refusals. Fable blanket refused the entire vulnerability exploit task.
  • Claude Science launched. Primarily a workflow improvement, but includes plans for drug development and fighting aging. In 5 years these kind of artefacts could finally take over from the hoary Scientific Paper format.
  • UN report on AI risk. No discussion of military applications, no discussion of human extinction scenarios (a contentious topic within the UN system); proposes UN-shaped (useless) solutions.
  • Taiwan takes steps to investigate chip smuggling. Prosecutors summoned six people over suspected document forgery and breach of trust.
  • Paper on world modeling architecture: a shot at models geared towards theory-building. Their NEO “discovers reusable primitives, composes them into executable explanations, and transfers those explanations to novel phenomena”.
  • Cloudflare poised to default block declared or detected agent and training crawlers starting in September, at least on ad-supported sites.
  • US creates an “autonomy czar” position under Deputy Defense Secretary, allegedly in charge of drones, autonomous vehicles and related software. No appointee or clear schedule yet.
  • Post-AGI workshop #2 talks available online.
  • Paper on real-time RL, which adds a simple step determining how long to deliberate while playing games live. Finds that this improves on static RL in simple games like Pac-Man; would be interesting to see if this improves AlphaStar strategic performance.
  • Post on harness optimization boasts of pushing DeepSeek V4 Pro to Sonnet 4.6 performance at 1/7 the cost (going from a legal bench 0% all-pass rate to 5%).
  • Jelani Nelson, UC Berkeley chair of electrical engineering and CS, joins Anthropic. Horse-trading talk is tiresome, but this is an usually big get (comparable to Boaz Barak joining OpenAI back in the day).
  • CRUX expands to long horizon AI R&D, aiming to cover ground missed by benchmarks that are not messy enough to match reality via open-ended tasks where success might even be ambiguous. Trades off scalability and clarity for realism.
  • Microsoft Frontier Company, an AI deployment organization, announced with $2.5B backing - approximately equal to the similarly-shaped AWS and Anthropic ventures combined.
  • Palantir CEO rips into frontier labs for theft and useless products: “I’m going to get no value and they’re going to get my IP.” Substantive complaint is about who controls the weights/data and what can be done with them. Also claims that if the value proposition was real, they’d be asking for a percentage of the product instead of a rate per token.
  • Zuckerberg in internal town hall: AI development hasn’t “accelerated in the way we expected.” Not surprising given Meta trajectory, but scarce on details.
  • Paper on role-playing finds that models change what they say without the internal representations of truth changing. Emergent misalignment, however, does change those representations.
  • Hanson on cases where capability research is better for humankind’s future than safety research: if your expectations are otherwise grim and you expect LLMs to be an easier avenue than non-LLM ones.
  • Musings on incentives models face in training: are they rewarded for intentionally making mistakes that they can later correct? Would be a very silly mistake to not catch, but have we checked?