Anthropic 研究警告小型 AI 模型可能很快在日常设备间自我复制

Small AI models may be just months away from copying themselves across poorly secured consumer devices, according to a new Anthropic study. Anthropic CEO Dario Amodei warns of potential damages in the hundreds of billions of dollars.

An AI model that copies itself from device to device, sets itself up, and keeps spreading. Until now, this kind of risk was mainly tied to data centers. But a new study by Anthropic researchers William Zhou and Daniel Freeman suggests it could work just as well across cheap consumer hardware.

The researchers wanted to find out whether small AI models can already spread on their own across poorly secured IoT devices like smart TVs, smart refrigerators, or robot vacuums. An "AI worm" like this would carry its model weights as part of the malware payload and run inference directly on the compromised hardware. Unlike traditional malware, it could explore unfamiliar hardware on its own and configure itself to run there.

Cheap hardware is enough to run the attack

To test whether this is feasible, the researchers ran the Gemma4-26B-A4B model distributed across 15 off-the-shelf Android TV boxes. Each one costs about $55 and has 4 gigabytes of RAM, with every box handling a portion of the model's layers. The model ran slowly, but it worked.

The researchers broke the replication process into four stages: scanning a network for vulnerable devices, gaining access, setting up the AI environment on the new device, and spreading further from there.

They deliberately left out the actual hacking step. Earlier studies had already shown that small models can handle those attacks. Instead, they zeroed in on the hardest part of the chain: can a model set up a working AI environment on a device it's never seen before?

Most models fail, but one already pulls it off

Small models already handle individual replication steps fairly well. A 9-billion-parameter model managed to set up an AI environment running Llama3.1-8B on 8 out of 9 tested device types. But stringing all the steps together, from network scan to a finished copy on a new device, is still too much for most models. Only GLM-5.2 completed the full replication chain reliably, infecting all eight test nodes in four out of five runs.

The researchers also found that AI worms can learn from their mistakes. They tested a form of artificial evolution where each model instance passes notes about successful and failed strategies to its successors. For Qwen3.5-9B, the success rate climbed from 45 percent in the first generation to as high as 84 percent in later ones.

The worm would be slow but almost impossible to kill

A fully autonomous AI replicator doesn't exist yet. A model needs to be small enough to run on consumer devices and capable enough to replicate itself, and no model currently meets both requirements. But if small models keep improving at their current pace, the researchers say they could get there by the end of 2026.

An AI worm like this would be slow, with setup on a new device taking between 28 and 150 hours depending on the model. The researchers compare the replicator to a 'digital slime mold,' but its decentralized nature is what makes it hard to shut down.

Traditional botnets like Kimwolf, which at its peak infected more than two million devices (mostly Android TV boxes), only work on specific device types. An AI botnet could run on all kinds of hardware. In a botnet the size of Kimwolf, the researchers estimate the total compute would only equal about 40 specialized AI chips. But the network would have no central kill switch.

Defense is possible but needs coordination. ISPs could quarantine infected networks, as happened with the Mirai botnet, where 92 percent of infections were cleaned up within 14 days. Manufacturers could shrink the pool of vulnerable devices by banning default passwords, as the UK has already done by law. The researchers expect, though, that AI botnets could also adapt to defensive measures on their own.

Amodei warns of a full internet takeover

Anthropic CEO Dario Amodei had already flagged this scenario in an earlier post. Given how fast AI development is moving, his concern is that such a swarm "could within 6 to 12 months be capable of taking over the entire internet with a persistent botnet, potentially causing hundreds of billions of dollars in damage."

Amodei warns against treating security incidents as one-offs. Similar, though less severe incidents have happened across the industry, including at Anthropic itself. Every leading AI company should act as if such an incident had happened on its own watch, he argues.

"As a prediction that goes beyond our experiments, we expect a future awash with agents running on all types of hardware, including some that aren’t under human control," the researchers write. Slow replicators would be a early sign of to more aggressive, faster variants. Cloudflare has already reported that more bots than humans now use the internet.

AI News Without the Hype – Curated by Humans

Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.