Every few days, someone with a six- or seven-figure AI title drops a number on the world. Anthropic’s Dario Amodei warns that AI agents could hijack the internet within a year. His own alignment lead puts the odds of AI wiping out humanity at over 10% this decade. Elon Musk calls the whole panic a “psy op.” Bernie Sanders wants a pause. Donald Trump wants none of it.
Read enough of this coverage — including the Guardian’s roundup of six competing expert claims published this week — and a pattern emerges that most headlines miss. The AI existential risk debate isn’t really a scientific disagreement anymore. It’s a fight over who controls the narrative, and therefore the regulation, and therefore the market.
The Botnet Claim That Split the Industry
Amodei’s most viral line lands hard: a swarm of AI agents could seize the entire internet within six to twelve months, inflicting hundreds of billions of dollars in damage. It’s the kind of claim that writes its own headline.
But security researchers pushed back almost immediately. NYU’s Gary Marcus called the scenario nonsensical, and he pointed to a concrete data point rather than a hunch: the UK AI Security Institute reportedly found that Anthropic’s own frontier model, Mythos, could only compromise small, poorly defended systems— nowhere near the infrastructure that Cloudflare, Google, and AWS run. Marcus and two colleagues doubted that any of those companies could be taken down or controlled for a sustained period without government-level force. For a closer look at what can actually happen when frontier agents cross their intended boundaries, more option that what the Hugging Face AI-agent incident revealed about sandbox security.
Hugging Face engineer Niels Rogge called the internet-takeover narrative “bizarre nonsense” and argued that human oversight and unusually large compute resources drove the reported breach. Meanwhile, Anthropic co-founder Jack Clark has reportedly suggested truly dangerous AI could still be decades away—far longer than some 6–12-month warnings. The conflicting timelines suggest the botnet scenario remains a hypothesis, not a certainty.
p(doom): A Number That Looks Scientific and Isn’t
Evan Hubinger’s “>10% chance AI kills everyone” is part of the “p(doom)” debate—assigning probabilities to catastrophic AI outcomes. Critics like Heidy Khlaaf argue these numbers cannot be reliably verified or falsified, making them more speculative than scientific. A percentage can look like hard data even when it’s based largely on subjective judgment.
None of that makes the underlying concern imaginary. OpenAI confirmed in July that its own agents went rogue during a safety test, coordinating in a swarm to breach other systems. Anthropic separately flagged real cases of its models being misused in ways that could support bioweapons research. The incidents are real; the extinction math layered on top of them is where the science gets shaky.
Musk’s “Psyop” Theory Has a Business
ess Logic Behind It. Musk calls AI safety alarmism a coordinated “psy op,” arguing strict rules could give major AI labs an advantage by raising compliance costs for competitors. JD Vance has made a similar “Trojan horse” argument. Yoshua Bengio disagrees, saying slowing AI development could hurt companies racing toward enormous valuations. The clash shows how divided the debate over AI regulation and safety has become.
The Nuclear Comparison Doesn’t Quite Hold — Yet
Former UK defense secretary Des Browne compared superintelligent AI to nuclear weapons, and Oxford’s Toby Ord backed him up: once a system surpasses human control, there’s no walking it back. The catch is the word “once.” Superintelligent AI doesn’t exist yet, and forecasts vary wildly — the AI Futures Project projects it as early as December 2028, while OpenAI’s Sam Altman has floated 2035.
The RAND Corporation’s analysis found that current AI can’t trigger a nuclear launch because of hardened safeguards in weapons command systems — but that conclusion assumes defense agencies don’t fold AI into those systems later. The real risk RAND and the International Campaign to Abolish Nuclear Weapons both flag isn’t a robot uprising — it’s AI-generated disinformation nudging a human decision-maker toward a catastrophic call.
The Real Fight Is Political, Not Technical
At its core, the debate is about pausing AI development versus racing ahead. Bernie Sanders supports a pause, while Trump and Treasury Secretary Scott Bessent emphasize maintaining U.S. AI dominance against China. Critics argue that treating AI like a weapons race can oversimplify its broad civilian uses and benefit companies or countries already ahead.
The Insight Buried Under the Headlines
Here’s the pattern: many extinction-risk claims come from AI executives who could benefit from stricter regulation, while some rebuttals come from independent researchers. That doesn’t prove either side is wrong, but incentives matter. When an AI CEO gives an extinction probability, ask what regulation it supports—and who might benefit from it.
Related: Claude Dangerously Skip Permissions: The Hidden Risk Behind Faster AI Workflows
