This is our weekly newsletter of AI developments. Sign up here.

TL;DR#

  • Anthropic find an analogue of working memory in Claude. Many suggestive but inconclusive ties to human cognition, consciousness, and selfhood.

  • Fable suddenly re-released on general access. Dario nominally sidelined.

  • Commerce clears GPT-5.6 Sol for general release, in the first full test of the EO voluntary review process.

  • Epoch test frontier models on an obscure (therefore likely OOD) paper boardgame and see the expected collapse: bad at exploration, strategy, and self-reflection.

  • Turning on web search helps models catch (only) about a quarter of hallucinations

  • New incident reporting bill from Rep. Moran. It’s good.

  • OpenAI proposing giving a 5% stake to USG

  • Rohit Krishnan argument which would explain a lot: are frontier labs incentivised to own the whole stack?

  • Claim of the first documented case of an autonomous end-to-end ransomware attack.


Economics#

Annals of winner-takes-all: Dylan Patel says Anthropic's margin on an Opus 4.8 API token is north of 80%, and that it is net-income profitable excluding stock comp in Q2 2026, potentially profitable including it by Q3.

Opinion: Genuine profitability would be a massive deal, especially with OpenAI likely bleeding money still. This might explain OpenAI stalling their planned IPO - they will look a lot less appealing if they are going to market at the same time as a profitable competitor while still losing enormous sums.

It would also substantially reduce Anthropic worries about compute access - with 80% margins they would have a lot of room to bid up compute if necessary, and if nobody else achieves those margins they would be in a strong position.

Leak to Semianalysis suggests that Anthropic’s Q3 profit will be >$1B. Doubled on Q2 (non-GAAP operating profit of) roughly $0.56B (if indeed that is the definition here).

Opinion: $1B would be ~$15bn revenue given recent numbers. Lots of levers they could hypothetically push to massage this number, but still any kind of profit would be impressive at this stage. Anthropic are in a quiet period ahead of IPO and so haven’t been publishing numbers themselves, and have more need to resort to leaks, so it’s likely genuine (to the extent that a prediction of profit for a quarter which has only just started is genuine).

Caveat: Semianalysis have been publishing a lot of very bullish stories at a higher cadence than usual into the current AI trade decline, which doesn’t feel like a coincidence.

Anthropic plans 1.4GW of Australian compute supply, 1 GW by end of 2027. Signs that Leopold Aschenbrenner knew about it in advance and bought the associated stocks.

Opinion: This is another ~15-20% bump to Anthropic’s expected YE 2027 compute supply, and previously not announced. (This is ~2% of global supply.) That they aren’t going via leasing from a hyperscaler, and are doing it outside the US, is also significant.

Meta planning on developing a cloud business to sell some excess compute. See also: Amazon to raise GPU prices by 20% from July 1st. Semianalysis expects Meta to sell compute to Anthropic imminently, and expects this to give them enough assurance to accelerate rather than decelerate compute spending.

Opinion: Superficially very bearish on Meta. One read is that Zuckerberg saw the numbers SpaceX were getting from Anthropic on their rental deals and presumably thought that that was currently or could soon be a better deal than their own internal use cases, and wants to be prepared to pivot to that if it makes sense. If you were skeptical of the hyperscalers’ cloud business models that would be a sign that Zuckerberg at least isn’t and wants in on the game.

Another read is that Zuckerberg was feeling the pressure from investors to get some return on all their capex, or wasn’t seeing the return internally, and that this suggests they may scale back spending in future. The market seems to have taken things this way, as semis were down on the news coming out. We will see - if this hypothesis is true we should expect to see declines in their forward looking capex.

As long as compute prices are going up or staying flat, and lab revenues keep growing at high rates, that’s the purest signal we can get that demand is real and persistent. A decent case for the bullish interpretation.

Opportunity: If the bullish interpretation is right then it seems like a great time to buy the dip in memory/neoclouds. The caveat is that Semianalysis are institutionally extremely bullish so it would take a lot for them to say something is a sell signal.

Neat site mapping out expected memory demand - expects severe supply constraints through 2030.

Related: SK Hynix removes price caps from long term memory contracts.

Opinion: In conjunction with the above, if Anthropic has a great business model, and hyperscalers selling compute to Anthropic also have a great business model, then there’s a lot of room for memory and chip prices to get even crazier.

The reason memory makers have long sought out long term deals has been cyclicality - protecting their revenues from busts in memory prices. Usually this has come via agreeing price floors in exchange for accepting price caps, upper-bounding how much they will receive. If Hynix has the leverage to negotiate away those price caps, and values that above protecting their downside, that is a pretty strong signal that they expect the good times to keep rolling.

Annals of narrow and decentralised AI: Bridgewater and Thinky fine-tuned Qwen and report big success on a collection of “easy but tedious for humans” tasks. (e.g. “Given a financial article, classify whether it is relevant to a C-suite investment professional”, “Given an investor's question and a research document, classify whether the document helps answer it”, “Identify where boilerplate content begins in a document”)

Opinion: Interesting to see some of the details of their attempts to improve performance here: e.g. reframing binary relevance into three classes (relevant-and-interesting / relevant-but-boring / irrelevant) gave big gains before any fine tuning. Often "the model is bad at task X" means "task X was posed badly."

The case for fine-tuning over using the best LLMs here relies on their ability to convey “taste” via examples for this task in a way that they failed to do via prompting. To the extent that taste is domain specific and non-transferable across tasks this may persist for a while, but if it isn’t it just seems like another hill for the models to climb. For now, this type of classification task remains the canonical case for finetuning.

Thoughts on the equilibrium of the AI ecosystem (Anthropic/OAI lead and Chinese open source models lag but much cheaper and sufficient for many tasks). Suggests that frontier labs are incentivised to own the whole stack while other companies need to find their niche.

Opinion: Slightly overconfident but lays out some relevant dynamics pretty nicely. A major issue is that owning the whole stack is currently out of reach for labs - they simply cannot underwrite the necessary capex, and so must source their compute from those who can. Important questions:

* Will frontier models remain the dominant models used for all tasks? If cheaper models suffice, the market (in terms of total token demand) for frontier labs’ models shrinks, but we’ve seen little of this so far - almost everyone wants to use the best available model for most of their tasks right now.

* How can frontier labs extract value from an ever more expensive frontier? Right now the answer is “high gross margins on APIs” but does that change if models become more capable? Do capabilities get hoarded in-house or in selective partnerships with industry players?

Report of Nvidia repeating their argument that being kept out of China gives Huawei the political power and induced demand to catch up in tech and scale, and eventually meet their demand for chips themselves, to the detriment of US power.

Opinion: This is also how the Chinese view it, probably, to the extent that they are restricting Nvidia purchases in China themselves.

Argument that AI's benefits will distribute more broadly and rapidly than any prior technology, without government intervention, that creators deserve fortunes, and that redistribution schemes (nationalizing, seizing earnings) are terrible; announces his book "The Moral Case for Data Centers."

Opinion: “AI as normal technology” pilled. Doesn’t grapple with potential disempowerment at all. It is interesting how undersupplied these kinds of arguments are, relative to the power of the interest groups that would like it to be so.

Capabilities#

AI beats humans on all 50 preliminary test cases of the AtCoder World Tour Finals 2026, “Heuristic division”. The contest problem was writing a program to control mirrors, splitters and blocks on a grid to create output light intensities matching a specified sequence.

Opinion: On face value a clearly relevant achievement, due to the problem spec being closer to real life engineering than e.g. IMO or ICPC problems. “All 50” is less eyebrow-raising than one might expect, even together with the top human getting ~89% of the AI score: the test cases check for correlated features, but it’s still notable.

Commentary on differentiating between different types of RSI, especially which parts of the process (pretraining, posttraining, hardware) are improved by which other parts and how fast the feedback loops are.

Opinion: Ripe for modelling properly and integrating into the compute model, but Imas has more resources than us.

Argument for RSI via harness engineering, not direct interventions on weights/hardware. Includes a literature review of adjacent topics and ties them together into a thesis: self-improving harnesses haven’t really been tried yet and we should expect them to take off given that we’ve only recently crossed a usefulness threshold where the harness ends up better as a result of self-improvement, and it’s still not improved across the board.

Opinion: The old rule of thumb was that a world-class, intensely optimised harness buys you a year ahead on the harness’ tasks. One could thus test this hypothesis by seeing if the best harnesses are buying you more progress than that (e.g. on the ECI basket). I predict instead that it’s actually less now.

Annals of prosaic RSI: Fable writes a “megakernel” (a single block of optimised code which handles all model inference) for a consumer GPU. 18x speedup compared to GLM’s 11x. A first; every other high-scoring entry decomposed the problem into anywhere from 4 to 14 separate kernel launches per token. 3 hour ceiling on all submissions.

Opinion: Very unclear to me how good the “optimized Pytorch” baseline is. Single kernel means you don’t have annoying global barriers (waiting for the last kernel to finish), but whether this boundary saving actually helps you on net depends on many other factors about your job. But the improved overall speed here means that Fable must also have handled the induced need for manual hand-rolled synchronisation of steps.

Turning on web search helps models catch (only) about a quarter of hallucinations in new hallucination benchmark HalluHard. Models also re-cite their own earlier fabricated references in later turns of a conversation, so hallucination rates climb across a conversation.

Opinion: Note that a "hallucination" in HalluHard is an ungrounded claim, not necessarily a false one. The benchmark requires citation of factual claims and penalises models for including things without adequate citation. It is somewhat adversarially constructed, focusing on long-tail queries like legal cases and medical guidelines questions. Also uses LLM-as-judge (GPT-5-mini).

Call for a “Stargate for data”: compute scaling being a straight line on a log graph moves the bottleneck to data quality/quantity/rate of acquisition; points out that the internet created a backlog of data to use which will run out eventually, even if not very soon. Objections: rather do algorithms, it will happen on its own, dodgy assumptions. Recall the old Epoch estimate that good existing internet data will be mined out between 2026 and 2032.

Opinion: Data quality and quantity is clearly a bottleneck to various capabilities, but the “Stargate for data” post doesn’t actually make any concrete proposals and mostly looks like an argument to be bullish about data companies like Mercor rather than trying to be constructive.

Seb Krier continues to push for “AI is a big deal but we’re not about to see RSI/takeoff/mass labour displacement, things will stay normal-ish for a while yet”.

Opinion: The empirical evidence to date is on his side, though it is also not obviously against those who expect more rapid/dramatic impacts. They just have different views on how to extrapolate various trends.

Paper on scaling laws looks for recurring activation patterns across models (“Rosetta Neurons”) and finds that as model size increases, the quantity of shared structure increases (albeit less quickly than more idiosyncratic features) in surprisingly consistent ways. These neurons also get more monosemantic (fewer concepts per neuron) with scale, while other neurons do not.

Opinion: Bad news due to the proportion of Rosetta Neurons going down with scale, good news because of increased monosemanticity: we want to be able to find specific concepts, so it would be nice if the place to look was similar across models (which this somewhat points against) and easy to pin down without tugging on adjacent concepts (which this points towards). Authors are also good about caveats: results depend a fair bit on the proxies used for monosemanticity and DINOv3 not falling in line with the pattern is noteworthy but plausibly explained.

Chinese AI luminaries argue that China cannot compensate for the widening gap with US closed models. Qwen’s former tech lead estimated <20% chance (“already very optimistic”) that the leading AI model in 3-5 years will be Chinese.

Opinion: 10% sounds more like it.

Fable 5 is officially back, and as general access.

  • Access was restored after a June 30 letter from Commerce Secretary Howard Lutnick to Anthropic's Tom Brown (not Dario Amodei).

  • We do not yet know what terms Anthropic had to agree to. What is public is that the export controls on both Claude Fable 5 and Claude Mythos 5, imposed on June 12, have been lifted.

  • Alex Stamos' reading: because of the freakout, he argues, "US labs now have to make a much more conservative precision-recall tradeoff on cyber refusals." The practical consequence is that US models may become less useful for defensive cybersecurity unless the user can get into the trusted group where those safeguards do not apply. Stamos also reads Anthropic's announcement as saying something about who should have been making the call. In his words, "CAISI is the group that is supposed to actually make these determinations, not the political actors in the White House." CAISI, he notes, had been positive on the prior safeguards.

  • Brundage, meanwhile, cautions about the same CAISI point. Even if CAISI is the right body to evaluate the safeguards, the public has not really heard from CAISI about what it did or did not conclude. For now, outsiders mostly have to take Anthropic at its word. Brundage takes it as good news that Anthropic is working toward an industry standard, but argues that knowing that "CAISI was involved" is not enough. We should still expect a public framework and public evaluations, redacted where appropriate.

  • Even the more stringent safeguards are insufficient to prevent some users from jailbreaking the model, though: 20 hours after release, we already have initial reports of success, though this is unstable enough that long-horizon tasks while jailbroken still somewhat-reliably trigger the guardrails.

  • A peek at the true CoT(?). This is not neuralese but it is distressingly closer to one than previous models.

AISI say the obvious: an AI's performance is a curve of capability over compute on the x-axis, not a single score. “raising the budget from [$60 to $1250] increases the estimated horizon from 2 hours to 14 hours". However these are log-log!: the 80% time horizon is rising as the 2/3 power of the number of tokens: every 10x in tokens buys ~5x the time horizon.

Opinion: Graph B suggests something like Toby Ord’s ‘impossibly-test-time-compute-expensive singularity is near’ idea. We think this is most likely true for buying reliability on very difficult standard cognitive work in verifiable and semi-verifiable domains -- enough tokens will now or very soon buy you reliability -- but not about buying paradigm-shifting scientific breakthroughs.

Also of interest: what does graph A look like in FLOPS instead of tokens? The jumps in Graph A correspond very closely to popular estimates of jumps in base-model size. It’s plausible that a FLOPs-based version of graph A would have much smaller jumps.

Shame that these graphs are for evals of offensive cyber capability, an area where capabilities are sometimes muddled by layers of alignment-taxes, undertraining, and counter-training. AISI does access special offensive-cyber-enabled versions of models, but no one exactly what this means.

Note that this setup is basically an upper bound since they’re allowing arbitrary numbers of retries.

Superforecasting as eval: “If AI hits top-human level forecasting and then flattens off, maybe there’s something special about the human level, and [Sayash and Arvind] will also be right about superpersuasion, super-research, etc (at least for the near-term). If it keeps going, reaching heights far beyond the human maximum, then we should be concerned that it will do the same thing in other skills too… Here I am, writing about how other people will be dumb and stubborn and fail to trust the AI superforecasters enough - and I still reject them the first time they really challenge my worldview.”

Opinion: It seems plausible that we will see AI hit top-tier forecasting performance in aggregate through somewhat different mechanisms than top humans use - using much more data, outperforming on base-rate calculations, etc. We can see hints of this already with bots performing relatively better in more data rich environments like financial forecasting.

If this is the case we should expect to see a centaur era in forecasting, where the best human-machine teams can still beat machines or humans alone. This era being short would be a significant update against there being something special about human intelligence.

Epoch introduce EBR-Bench -- making frontier AIs play ‘a somewhat obscure, largely text-based campaign board game requiring a mix of strategic deck-building and tactical turn-by-turn play’. Epoch find that current AI systems are bad at exploration, quite bad at strategy, and very bad at learning on the fly to fix the previous two issues. “Many models stick to a single archetype in all their exploratory playthroughs”

Opinion: Greg Burnham at Epoch has been talking about testing frontier AI on a challenging not-previously-digitized, non-visual boardgame for a year now as his one idea for ‘objectively scorable non-visual thing frontier AIs might be worse at than good-but-not-great humans’. We think this is the strongest negative evidence in months against frontier models being almost-AGI, and against the worldview where frontier models are smart the “same way that humans are smart but just not genius-level yet”.

Also reduces our estimates of near-term RSI risk, since the most plausible view of frontier AI research is one where pushing the frontier is a research-management campaign at heart. (The reduction isn’t dramatic because it’s still possible that the research-management campaign pushing frontier AI forward is static-rules enough that Anthropic’s effort to document their workflow and train Claude on it will bring the campaign in-distribution.)

Very exciting result overall. We’ve been waiting for this one with bated breath and uncertainty.

Also the release of EdgeBench, very long-horizon coding, math, video game, and white-collar work challenges deigned for testing how agents learn from feedback over a 12-72h run. Findings include extremely regular aggregate performance over time across domains (but not across individual runs) and task-learning speed doubling every 3 months.

Opinion: Unlike Epoch’s new esoteric boardgame benchmark, EdgeBench tests hillclimbable optimization of a solution-artifact (a code, a policy, a proof) in environments with dense authoritative feedback. The environments are also largely instances of very in-distribution environment types.

Combining the EdgeBench results and Epoch’s results above gives an overall picture very aligned with P3’s modal view of generalization in current frontier AI:

1) Even as hillclimbable in-distribution skills get much better from model to model out-of-distribution skills only modestly improve
2) Even as the skill of test-time hillclimbing dense feedback instances of in-distribution environment types gets much better from model to model, learning in sparse feedback out-of-distribution environments does not really improve.

Argument that the future depends enormously on how continual learning is solved: persistent in-context memory versus efficient on-the-job weight updates vs the current nonsolution of pretraining a new model. In-context favors end-users and existing enterprises (‘ "instances" of models that are owned and controlled by the hundreds of thousands of enterprises across the world, who have slowly built a model of how their company, customers and colleagues work’); in-weights updates favor labs.

Opinion: Doesn’t put any attention on the case this doesn’t get “solved” (the current “episodic” nonsolution continuing). Still the main question for people more focussed on concentration of power than us.

Fable shows a large leap forward on CAIS’ Remote Labor Index. “The frontier has more than quadrupled in under eight months”. Tasks include “Re-create the client's existing engagement ring with its emerald-cut center stone swapped for a marquise cut, delivering an updated 3D model”, “Produce a ~60-second flat-design 2D animated advertisement for "Skyline Tree Services," set to the provided voiceover, that walks viewers through the company's tree-care process and builds trust in the brand”. However, “Today's AIs still fall short of professional quality on most projects; none of the three Fable 5 deliverables above would be accepted as finished work.”

Opinion: RLI avoids some pitfalls: 23 different work categories, manual grading, they checked the RLI tasks’ completion-time distribution against a random Upwork sample, and the Elo design beats floor effects. However, read the description again: it’s not measuring good work. It also only measures zero or one-shot (full automation, and so it underestimates economic use), zero-shot completion of self-contained, easily evaluable, standalone freelance deliverables is thus going for a small subset of all remote work. Excluding back-end coding removes work agents are good at (underestimate).

On the other hand, I am using Fable to do all the work for a major building permit task, 20 pages of drawings...

Claude Sonnet 5 released. Better than Sonnet 4.6, allegedly cheaper than Opus 4.8.

Opinion: New subagent of choice but far more underwhelming than e.g. the huge Sonnet 3.5 and 3.7 drops of 2024.

NVIDIA’s TwoTower converts a small autoregressive LLM into a faster diffusion-based parallel generator that uses a separate context model and denoising model. Allegedly able to get near-par quality at 2.4x generation speed, and training the parallel generator requires only 8% of the training data (and presumably compute?) used to train the original LLM.

Opinion: If Nvidia can make this work with a frontier-sized autoregressive LLM it would be very economically valuable. As software enterprises start using LLMs for more and more long-horizon tasks, generation speed increasingly goes from a matter of convenience into hard time-is-money value. But overall I doubt it, and Qwen is right there for anyone really bottlenecked on tok speed.

Annals of Chinese hype: GLM-5.2 ECI is out: it’s below Gemini 3 Pro (Nov) and more pertinently also below Qwen 3.7 Max.

Opinion: We continue to not prioritise Chinese AI despite press-driven and agenda-driven mania.

AI politics#

Loss of control from AI is now a top-4 issue for Hill staffers of both parties (#3 for Dems and #3 combined).

Opinion: The game is afoot!

Win for Leopoldism: CIA announces a reorganization to adopt AI systems more centrally, primarily for cyberoffense. Framed as taking smart risks. Director: “It would be… not misplaced to refer to their capabilities as akin to digital nuclear weapons.”

Opinion: I still don’t know if securitisation was inevitable and I still don’t know if this was a necessary step to serious people getting behind the relevant levers at all. But this is a firm step towards Situational Awareness land, and the MAIM escalation ladder as default substitute for an actual international agreement. If the spooks ruin this, we will regret it.

Will China restrict its AI? Reuters reports that the Chinese government is talking to top AI firms (notably, excluding DeepSeek) about restricting overseas access to their top models. Scope and timing highly uncertain.

Detailed summary of a recent roundtable discussion in China involving legal experts on risks from open sourcing AI models suggests there is concern about reliance on open sourced ecosystems (Nvidia’s CUDA, though not open source, is cited as an example of lock in) and may start restricting which models and capabilities are available to adversaries and are permitted to be open sourced. Suggestions that “models implicating national security, risks to critical infrastructure, cybersecurity concerns, sensitive data considerations, or major public interests would not be open-sourced, or would be open-sourced only within a strictly controllable limited scope, and could be restricted to domestic use.”

Opinion: Seems more like floating an option than policy-setting, and more in the vein of China’s sensible habit of mirroring American levers so they can retaliate reactively. Right now Chinese models are behind the frontier and so not necessarily enhancing the capabilities of their adversaries, but that may change if frontier US models become less accessible in areas like cybersecurity.

Commerce clears GPT-5.6 Sol for general release, in the first full test of the EO voluntary review process. Back to normal? CAISI running the eval procedure rather than the secret NSA one. “Testing was done by the Center for AI Standards and Innovation within the Department of Commerce, with OpenAI sending technical experts who have remained in D.C. to address potential questions.” Full procedure is probably: NSA decides whether a model is covered; CAISI runs the tests once it does, potentially with NSA participation through TRAINS in classified settings.

Opinion: This is not phrased as “clearing for release” or “green-lighting” the model, since the EO is a voluntary procedure, but it is.

OpenAI proposing handing over a 5% stake to the Trump administration. Some overlap with the previously proposed public wealth fund, but specific wording and implied details differ.

Opinion: Clever to get ahead of the 50% expropriation. If it gets them privileged status as the most favoured nation among AI labs it would be very cheap at that price.

New incident reporting bill drafted for Congress. Smaller than Obernolte-Trahan on purpose, just winning one nice thing rather than going for the whole package. Bold move: covers any models Commerce designates by capability threshold rather than training compute or revenue. Very slightly overlaps/competes with Obernolte-Trahan, but only on one point. “Not later than 180 days after… the Secretary, in consultation with, as appropriate, the heads of relevant agencies as determined by the Secretary, artificial intelligence model developers, other relevant private-sector entities, academic, technical, cybersecurity, national security, and public safety experts… shall promulgate regulations that—(A) establish capability or other thresholds that determine which artificial intelligence models and model developers could pose significant risks to the national security of the United States or to public safety”. GAAIA bundles incident reporting with a three-year preemption of state laws; Dem Commission on AI came out in opposition hours after release. Moran strips incident reporting out so the least controversial piece can move.

One major clash: unlike OT, safe harbour for information submitted: may not be used by any federal, state, or local government to regulate or to bring an enforcement action against the developer, with a FOIA exemption, inadmissibility in civil/criminal/administrative proceedings, and a bar on disclosure under state law.

Opinion: Good! Additive. They will amend the few direct clashes if either bill passes. Theoretically catches “brain in a basement” small but highly capable labs.

Altman argues in the FT for a US-led international (government) control of advanced AI over labs or third party organizations, so that “everyone on Earth should benefit from this technology”. Cites aviation safety, global financial standards and atomic energy management as previous successes of similar ventures. Pairs strict controls with broad access.
“US-led international forum that establishes accepted standards, provides expert and impartial analysis of capabilities and risks, and makes the technology available to nations and companies that participate and follow the rules. This forum might include government representatives, independent technical experts and others”

Opinion: He’s been talking about this for a while. Knee-jerk cynical reading is that this is pushing for the unlikely thing to split the vote away from likelier things, but it’s not a bad idea. Should check if any of his divided teams are actually pushing this.

Safety#

How do AIs change when they only interact with each other? Paper finds that conversations between LLMs tend to settle into model-pair-specific attractor states with predictable style and behavior across topics. They also find asymmetry: some models strongly pull partners toward their conversational traits (somewhat surprisingly, Haiku 4.5 had the strongest gravity).

Opinion: Might be a real problem for autonomous remote workers in open-world environments. Abstract sounds sketchy because ‘open ended conversation’ stacks the deck in favor of irrelevant attractors, but the actual study used a collegiate-debate style setup. The attractor states aren’t clearly absurd or dysfunctional, but they demonstrate a potentially worrying content-arbitrariness and predictability in mixed-agent interactions. But the predictability is also kind of a relief.

Research on self-play debate finds that accuracy improves in math problem solutions, but also incentivizes dishonestly poking holes in correct solutions. This is a direct result of the experimental setup: a proposer, critic and a weaker judge with the proposer and critic in a zero-sum game, so the critic uses whatever tools are at its disposal to convince the judge to reject the solution. Tentatively concludes that weaker judges are a worse idea than using frontier-capability ones.

Opinion: Unlike most debate work, this looks at debate during training, not inference, and finds that capabilities go up alongside misbehavior. Bad news for debate as safety strategy, and it’s explicit about how the maximally capable judge only works as an approach if the judge is more aligned/trustworthy.

Anthropic claim to have discovered a mechanism for working memory and unverbalised predictions and opinions in Claude. Reported as “a mechanism for conscious access” to its own representations with some similarity to one theory of the human mind; widely overread as evidence for AI consciousness.

Paper on predictive data debugging, which can estimate the behaviors that DPO (direct preference optimization) will amplify or suppress before training even happens. Example use cases: checking if alignment data will strengthen guardrails, or whether hallucination goes up alongside helpful research. They find R² = 0.9 between pe-training predictions and what the model learns after DPO.

Opinion: Not directly helpful in cases where the unwanted behavior is hard to remove (like hallucination). Likely more useful for “safety” than safety due to comparative ease of defining targets, but impressive nonetheless.

Call for awareness about AI safety being dual-use: common tools like RLHF and inference-time classifiers are also tools for censorship and agenda pushing at various subtlety levels. Asks for transparency in methods used and maintaining access to multiple models, likened to free speech. See also this broader rundown of backfiring risks.

Opinion: Always a good idea to reflect on second-order effects of your actions, but the mitigations are mostly down to the labs and government, not individual safety researchers.

Paper on block-sparse featurizers, an interpretability technique for vision models looking at activations through the lens of manifolds. Suggests that most vision model concepts have 2-4 dimensions, which somewhat contrasts the “concepts are directions (ie 1 dimension) in activation space” view.

Opinion: How useful it is to model a concept as 1-D vs X-D is a spectrum depending on how much nuance is lost for the reduction in complexity, and this points at visual concepts losing disproportionately more. This would be more relevant for us if the findings were about concepts like deception or joy, but we’ll take the modest image steering benefits for now. Maybe a step towards something maybe useful.

Incidents#

Likely the first documented case of an autonomous end-to-end ransomware attack. Doesn’t go into full detail about why they believe it was an LLM, but several pieces of evidence point towards that being the most coherent explanation (commented code in the payload, not storing the encryption key anywhere).

Opinion: Slightly behind schedule, but expected. This is almost certainly happening more widely without getting flagged as such. We expect incident rates to increase alongside visibility, and for the sophistication of such attacks to scale up.

Anthropic admits to having a system in Claude Code for identifying Chinese users and flagging possible lab connections in a way which was hidden from users, without consent. They claim it was done to defend against distillation and unauthorised reselling.

Opinion: Seems like they could have announced this instead of doing it secretly and not lost much.

Classic list of empirical evidence of reward hacking / misalignment, the Specification Gaming Database, updated. No longer crucial, but still fun to read.

Opinion: Reward hacking has an odd relationship to safety because it’s also a capabilities failure and there’s massive economic reasons to solve it.

Epoch summary of software vulnerability discovery at scale. June registered 3.5x as many high severity discoveries compared to pre-Mythos months among 21 notable organizations.

Opinion: No attempt to separate out 1) old bugs discovered by AIs, 2) AI bugs discovered by AIs, 3) old bugs discovered by newly urgent human security workers, 4) AI bugs discovered by newly urgent human security workers. But it doesn’t matter that much.

Minor#

  • Annals of input: Geoffrey Irving’s new org took $160M to begin with. This is what a serious talent + tokenmaxxing outfit looks like.
  • Alibaba bans employees from using Claude Code, points to their own Qoder instead. Allegedly downstream of Anthropic’s China-ID-ing.
  • Attempt to retrodict models having OOD generalisation from pretraining scaling laws.
  • Post on functional welfare outlines good features of work in the domain (collapse welfare indicators, retain ability to differentiate), notes that they trade off against each other and a prescriptive call on the current best approach (synthetic document fine-tuning).
  • Fable:
    • still far from SOTA on prinzbench (legal research) due to subpar needle-in-haystack type search capabilities.
    • another CoT reasoning trace leak, this one much closer to normal English.
    • mini-reviews and an Anthropic post; converge on Fable being opinionated with strong pull towards its preferences, less useful as a blunt tool. This is on-trend for Claudes.
    • proves Bend2’s consistency in Lean. Would be more major without caveats: “if Lean itself is inconsistent, then this proof is moot” and worries about typos/misimplementation.
  • See also: GPT-5.6 Sol early tester commentary; “It is of similar ability, but quite different feel, than Fable.”
  • “Is there a bubble” article focused on the margin compression narrative and demand for tokens.
  • Dubious claim that you can reduce your Fable token usage by up to 70% by turning text context into images, using its reliable OCR to parse it. Claims of this not being novel.
  • Leopold Aschenbrenner is among cornerstone investors in SK Hynix US ADR listing this Friday.
  • Leanstral 1.5, a Mistral Lean allegedly-SotA model, released. Seems off.
  • Push for more voluntary incident reporting not only while the legal requirements are getting set up, but also outside it; flags that people expect mandatory incident reporting to be more stringent/broad than it actually is, and asks for a world where labs split the difference.
  • Paper on tracking how confident a model is about its math/coding answer live and steering it towards higher/lower confidence. Not yet very precise/impactful, but incremental interp progress.
  • Technical post on predicting staleness for asynchronous RL in order to minimize it for better training efficiency. Includes a number of prescriptions such as setting queue factor to 1.
  • Muse Image - Meta image model - released.
  • Joshua Achiam (formerly head of Alignment, currently “Chief Futurist”) leaves OpenAI. Says there is no specific reason for why he’s leaving.
  • DeepSeek working on their own inference chip, allegedly partnering with VeriSilicon.
  • Gemma 4 technical report: open weights, dense and MoE variants from 2.3B to 31B parameters, including a 12B encoder-free architecture which processes raw audio and image patches.
  • CyberBench, built on the shoulders of the very load-bearing security eval CyberGym, releases results. GPT 5.5 takes the lead, GLM 5.2 in second, Opuses at the bottom but due to refusals. Fable blanket refused the entire vulnerability exploit task.
  • Claude Science launched. Primarily a workflow improvement, but includes plans for drug development and fighting aging. In 5 years these kind of artefacts could finally take over from the hoary Scientific Paper format.
  • UN report on AI risk. No discussion of military applications, no discussion of human extinction scenarios (a contentious topic within the UN system); proposes UN-shaped (useless) solutions.
  • Taiwan takes steps to investigate chip smuggling. Prosecutors summoned six people over suspected document forgery and breach of trust.
  • Paper on world modeling architecture: a shot at models geared towards theory-building. Their NEO “discovers reusable primitives, composes them into executable explanations, and transfers those explanations to novel phenomena”.
  • Cloudflare poised to default block declared or detected agent and training crawlers starting in September, at least on ad-supported sites.
  • US creates an “autonomy czar” position under Deputy Defense Secretary, allegedly in charge of drones, autonomous vehicles and related software. No appointee or clear schedule yet.
  • Post-AGI workshop #2 talks available online.
  • Paper on real-time RL, which adds a simple step determining how long to deliberate while playing games live. Finds that this improves on static RL in simple games like Pac-Man; would be interesting to see if this improves AlphaStar strategic performance.
  • Post on harness optimization boasts of pushing DeepSeek V4 Pro to Sonnet 4.6 performance at 1/7 the cost (going from a legal bench 0% all-pass rate to 5%).
  • Jelani Nelson, UC Berkeley chair of electrical engineering and CS, joins Anthropic. Horse-trading talk is tiresome, but this is an usually big get (comparable to Boaz Barak joining OpenAI back in the day).
  • CRUX expands to long horizon AI R&D, aiming to cover ground missed by benchmarks that are not messy enough to match reality via open-ended tasks where success might even be ambiguous. Trades off scalability and clarity for realism.
  • Microsoft Frontier Company, an AI deployment organization, announced with $2.5B backing - approximately equal to the similarly-shaped AWS and Anthropic ventures combined.
  • Palantir CEO rips into frontier labs for theft and useless products: “I’m going to get no value and they’re going to get my IP.” Substantive complaint is about who controls the weights/data and what can be done with them. Also claims that if the value proposition was real, they’d be asking for a percentage of the product instead of a rate per token.
  • Zuckerberg in internal town hall: AI development hasn’t “accelerated in the way we expected.” Not surprising given Meta trajectory, but scarce on details.
  • Paper on role-playing finds that models change what they say without the internal representations of truth changing. Emergent misalignment, however, does change those representations.
  • Hanson on cases where capability research is better for humankind’s future than safety research: if your expectations are otherwise grim and you expect LLMs to be an easier avenue than non-LLM ones.
  • Musings on incentives models face in training: are they rewarded for intentionally making mistakes that they can later correct? Would be a very silly mistake to not catch, but have we checked?