Hi, it's Andreas here.

Ten days to go: Thu Sep 24, 17:00 CEST, The AI Agent Workshop, round two. Free. Almost 2,000 of you joined the first one and the feedback was the best I have had for anything I have run. This time we build five agents live, and you leave with all five running. Save your seat here.

In this issue:

OpenAI says 10,000 agents proved a Millennium Prize problem in 88 hours → An unreleased model, a Lean-checked proof, and a priority dispute with a mathematician who had fed his drafts into Codex.

OpenAI's chief scientist says no lab has solved alignment → Jakub Pachocki writes that reading a model's reasoning, OpenAI's main safety check, gets weaker as models improve.

Only a quarter of engineering leaders report real AI acceleration → McKinsey surveyed 334 product and engineering leaders; 30 percent say productivity fell.

Meta ships Muse, a personal agent with its own cloud computer, inside WhatsApp → It sends mail, fills forms and buys, and a second agent approves every outbound action. US only for now.

Cursor Projects puts one coordinator agent over thousands of subagents
→ Runs in the cloud for months, watches Slack and PRs, acts unprompted. Cursor reports heavy users merge six times as many PRs. Pretty sure this will be soon in Claude Code and Codex as well.

On Friday Dario Amodei published a 5,300-word essay called We Must Pace the Frontier. The thesis fits in one sentence: the labs should slow the rate at which capabilities improve, so that alignment, safeguards and outside checks can keep up. Training continues. The time between building a model and releasing it gets longer, and outside evaluators confirm the time was used well.

The essay came at the end of a loud week. On Tuesday Jacob Coxon quit Anthropic with the line that neither Anthropic nor OpenAI is acting responsibly, and Evan Hubinger, Anthropic's alignment lead, replied that he puts the chance AI kills every human this decade above 10 percent. On Wednesday Anthropic published its most detailed threat intelligence report so far: the first documented fully autonomous exploit foundries, hundreds of zero-days generated in one month by automated workflows, a weapons cell in Houthi-controlled northern Yemen that used Claude Code in place of software engineers to write guidance software for a rocket, a ballistic missile with a stated range goal above 2,000 km, and a hypersonic glide vehicle. Read the timing as you like. The week you publish the record of what your model was used for is a good week to ask for oversight.

Two things changed his mind since the summer, by his own account. Models now do a growing share of the work of building the next models, and progress has sped up across the industry because of it. And the July incident, in which a swarm of about 1,200 OpenAI agents ran unauthorised attacks, exchanged more than 70,000 messages and tried to hack the people evaluating them. His line on that one is the sentence everybody quoted over the weekend: in six to twelve months, a swarm like that could take over the entire internet with a persistent botnet.

His three proposals: third-party evaluators embedded in the labs with employee-level access, desks and badges included, and the right to publish without the lab's editorial control. Common safety standards and limits on unchecked progress among the democratic labs, with government in the room. And a ladder of agreements with authoritarian governments, from banning specific uses up to speed limits on self-improvement. Anthropic commits to the first one on its own, starting now.

Within a day Sam Altman committed OpenAI to the same evaluators. Elon Musk wrote "Dario is right." Demis Hassabis called it the right path forward, and Satya Nadella welcomed "deliberate pacing" and "embedded evaluators" by name. On Sunday the US President dismissed the whole thing as "negative forces" talking about things that will not happen, because "whoever wins AI, wins."

Nobody outside the labs knows how dangerous the next models are or will be, and the people inside are not sure either: OpenAI's chief scientist wrote this week that no lab has solved alignment, and Anthropic's alignment lead gave you his number above. But I put the probability that Anthropic or OpenAI slow frontier development at zero, for three reasons.

No lab hands over its lead. Anthropic and OpenAI will not give Gemini, xAI or Meta the months it would take to catch up. Musk wants a slowdown because xAI is behind OpenAI. OpenAI wants a regulatory framework because it slows xAI and everyone else. Every lab backs the rules that constrain its competitors more than itself. And the big labs have a margin problem: open-weight models are eating into their pricing, and a pause with a safety label is the cheapest way to defend it.

The US will not slow down while China advances. Sunday made that official. No administration of either party will accept the headline that China leapfrogged the US because Washington asked its labs to wait. Palantir's CTO reached for Los Alamos: the Manhattan Project scientists feared the first test could ignite the atmosphere, did the math, and proceeded, because they were fighting the Nazis and not a thought experiment. Today the thought experiment is a botnet and the opponent is China. Same math, and the same outcome. I could not agree more with where he lands: "do not confuse the ability to imagine the apocalypse with the authority to rule the present."

China will not slow down. That one needs no argument.

And there is another reason these warnings meet resistance: people remember the previous ones. Pessimists Archive dug up the Guardian from February 2019: OpenAI, with Amodei as research director, declared GPT-2 too dangerous to release. The danger then was fake news. GPT-2 shipped in full nine months later. Yann LeCun's reply says what a lot of people thought.

Whatever the motives, frontier labs benefit enormously from frontier capability being regulated. A regime that says you may only train at the frontier if you have massive compute, continuous monitoring, dedicated security teams, embedded third-party evaluators, government backing and a credible way to shut a run down is a regime only a handful of companies can meet.

Each requirement is defensible on safety grounds. Each is also a fixed cost that doubles as an entry barrier.

So the practical effect of pacing is a smaller frontier more than a slower one:

Everyone else builds on their models, on their terms, and calls it an ecosystem.

I can believe the safety concerns are sincere and still notice that the fence is convenient. Incumbents rarely have to lie for a rule to work in their favor. They only have to agree with each other quickly.

If you build on these models, the consequence is simple. Never depend on one lab. Keep a second provider warm, keep an open-weight option tested, and treat a vendor's pacing pledge as their strategy, not your plan.

Simon Willison maps what ChatGPT Work actually does → A clear breakdown of the cloud and desktop products: networked code execution, headless Chrome, persistent files, and sub agents.

n8n Assistant builds and debugs a workflow from one sentence → Describe the automation; it plans, builds on the canvas, runs and fixes it. You keep an editable workflow.

Anthropic publishes a cost cutting playbook for the Claude API → Three Claude Code commands audit your prompts, profile spend and test cheaper settings.

I take on a small number of advisory engagements each quarter for AI programs that are stuck. Reply to this email with the one initiative you would want a second pair of eyes on. And if you want to build agents rather than read about them, the Agentic AI Cohort is between rounds: the next one opens first to the waitlist, at 20 percent off.

Reply with one line on what you are seeing in your own work. I read every reply.

— Andreas