There is a particular quiet that settles over a fishing ground once the fish are gone. Not the working silence of a calm morning, but an absence — fewer boats each season, the ones that remain ranging farther for less, the talk on the radio turning from where they are biting to whether it is still worth the fuel. I have crossed waters that the charts still mark as rich and found them swept clean, and the thing that always stayed with me was not the emptiness itself. It was how few had truly seen it coming. Most of us suffer what I have come to call horizon blindness: we cannot register a loss that arrives slowly, or out past the edge of what we can see, until it is already complete. A handful could read the water well enough to know — and knowing, it turned out, was never the thing that could have saved the sea.

That is the shape of a commons. A shared thing that belongs to all and therefore to none, drawn down by people who are each behaving perfectly sensibly. The last skipper to keep fishing a dying ground is not a villain. He is simply doing the only rational thing available to him, because the fish he leaves in the water will not be there tomorrow — they will be in someone else's hold. Restraint, practised alone, is not virtue. It is just surrender with extra steps.

I have spent recent months reporting on how the sea was emptied, and most of a life before that on the water that was emptying. So when one of the largest artificial-intelligence companies on earth published a quiet, careful document this month asking the entire industry to consider slowing down, I did not read it as a technology story. I read it as a commons story. And I already know how those end when the coordination fails.

II

On the fourth of June, Anthropic — the company behind the Claude assistant, now valued near a trillion dollars and preparing to sell shares to the public — posted an essay by two of its senior figures arguing that the world should have the option to slow or temporarily pause the development of the most advanced AI systems. The reasoning rests on a phrase that deserves translating out of the jargon, because the jargon is doing a lot of hiding.

The phrase is recursive self-improvement. Stripped down, it means the moment at which an AI system becomes capable enough to design and build the next, better AI system largely on its own — and that one builds the one after, faster each time, with the humans shifted from the helm to the rail, watching. Anthropic is careful to say this has not happened and is not inevitable. But it offers an unsettling piece of its own evidence: more than four-fifths of the new code written into Anthropic's own software is now produced by Claude rather than by people, and a single engineer there now ships several times the work he did two years ago. One of the authors has suggested some models could reach the threshold within roughly two years.

The proposal is not "everyone stop." The authors know better. Their actual ask is for a verifiable mechanism — a way for rival companies in rival countries, the United States and China above all, to throttle back at the same time and prove to one another that they have. They want, in other words, a fisheries agreement for the mind.

They want a fisheries agreement for the mind.

III

Here is why the request is honest, and here is why it will be so hard to grant. Anthropic states the problem against its own interest, plainly: if one company stops alone, the least cautious of its rivals simply absorbs the lead. The careful lab falls behind. The reckless one takes the market. This is the trawler's exact logic, transposed into silicon. No single boat can afford to be the one that pulls its net first, because the catch it forgoes does not stay in the water as a gift to the future. It goes to whoever did not flinch.

There is a bitter symmetry in who is sounding the alarm. The people inside these companies are among the very few on earth not blind to this particular horizon — experts in a field barely a decade old, reading instruments the rest of us cannot see. They are not guessing. They know precisely what they are building, because they are the ones building it, and they are racing one another to build it faster. The warning and the race issue from the same hands.

So a pause is not really a question about technology at all. It is a question about whether a dozen fiercely competitive organisations, several of them answerable to governments locked in a contest for economic and military advantage, can be made to trust that the others have genuinely slowed — and to keep trusting it month after month, with billions of dollars and national standing riding on the answer. We have a word for arrangements like that. We call them treaties, and the graveyard of failed ones is large.

This is the part the daily coverage tends to skip, because it is easier to file the story as either a warning or a stunt. It is neither. It is an admission that the incentives all point the wrong way, written by a company that profits from the very acceleration it is asking the world to interrupt.

IV

Which is exactly where an honest reckoning has to hold two facts in one hand at once, and refuse to drop either for the comfort of a cleaner story.

The first fact cuts in Anthropic's favour. Earlier this year the company did something that its own commercial interest plainly argued against. It refused to let its models be used for autonomous weapons or for the mass surveillance of American citizens, and it walked away from a large defence contract rather than remove those limits. The government's response was severe: the Pentagon branded Anthropic a national-security supply-chain risk — a label normally reserved for foreign adversaries — and the administration ordered federal agencies to stop using its tools altogether. Anthropic sued. A federal district judge in San Francisco granted it a preliminary injunction, calling the blacklisting “classic illegal First Amendment retaliation” and finding the government's stated security rationale likely pretextual — though a federal appeals court soon afterward declined to block the designation while the case proceeds, so the fight remains unresolved. A rival lab struck its own Pentagon deal within hours of Anthropic's refusal. Whatever else this was, it was a company choosing principle at real and measurable cost, which is precisely the move a pure race-for-dominance model says should never happen.

The second fact cuts the other way, and it must be said in the same breath. This appeal to slow down arrives from a firm that is, at this very moment, preparing to raise tens of billions of dollars in order to speed up — to build the data centres and buy the computers that make models like Claude more powerful, faster. A memo urging caution, published on the eve of an enormous public offering, is a fact worth sitting with rather than resolving. Both things are true. The principled refusal was real. The commercial timing is also real. Anyone who tells you the second cancels the first, or that the first excuses the second, is selling you the easy version.

The race is undecided. It is not lost. But undecided is not the same as safe.

V

And the clock is not theoretical. The reason this debate stopped being academic can be dated to the second of June, two days before Anthropic's essay, when researchers at the University of Toronto and the Vector Institute, with colleagues at Cambridge and an enterprise software firm, published a proof of concept that should sober everyone in the argument.

They built a computer worm — self-spreading malware — driven not by a frontier system locked behind corporate walls, but by a small, free, openly available AI model running on the machines it had already infected. Inside a sealed laboratory network, with no human steering it, the worm reasoned its way from device to device, working out how to break into each one as it went, and within a week it had compromised roughly three-quarters of the machines. It did not need to invent exotic new weaknesses; it simply exploited the known, unpatched flaws that already riddle ordinary systems — which is what most real attacks rely on anyway.

The lesson is not that the sky is falling. The worm was clumsy in places and failed about as often as it succeeded on any given attempt. The lesson is the floor. You no longer need a billion-dollar model or a state's resources to point an autonomous, adapting attacker at a network. A free download and a laptop will do. That capability is already loose in the world, and the rules meant to govern it are, by any honest measure, nowhere close to catching up.

VI

So I come back to the water, because the water taught me the only thing about this I am sure of. The fisheries did not collapse because the people working them were stupid or wicked. They collapsed because no one could afford to be the first to stop, and because by the time anyone proposed a binding way for everyone to stop together, there was little left worth protecting. The tragedy was never the greed. It was the lateness.

What is genuinely different this time — and I do not say this to reassure, because false reassurance is its own kind of lie — is that someone has put the mechanism on the table before the grounds are bare. Imperfect, self-interested, suspiciously timed, take every caveat you like. But a verifiable way to slow down together, proposed while there is still something to save, is more than the fishermen of any emptied sea ever got.

The open question is not whether the warning is correct. It is whether anyone will sign. A treaty no one ratifies is just a well-worded epitaph, and the history of the commons is mostly a history of clear warnings issued slightly too late to the people least able to act on them alone.

The race is undecided. That is the honest report from the rail. Not won, not lost — undecided. Which means the net is still in the water, and we are all still standing on deck, watching to see who reaches for theirs first, and whether the rest of us will believe them when they say they have let go.

Credits & Sources

Further Reading