Anthropic’s AI Worm Gets In Through a $55 TV Box. The Best Defence Is a Password Law

On 12 October, Anthropic’s Frontier Red Team, the in-house group that stress-tests models for dangerous capabilities, published a post titled “AI models may be capable of self-replicating without access to a datacenter.” Its claim is that small language models may be months away from copying themselves onto connected household devices such as smart ovens and off-brand Android TV boxes. Reuters carried it the next morning. The authors put it this way: “Like coding agents in 2024, we’re seeing signs of life for AI worms.” A worm is a program that spreads from machine to machine without anyone launching it each time.

The experiment has three parts, and the headlines merge them. In the first, the researchers bought fifteen TV boxes at about $55 each, with 4 GB of memory apiece, and spread one of Google’s open models, Gemma, across them. It ran at about 0.39 tokens per second, a few words a minute. In the second, they gave models unfamiliar hardware and asked them to get a model running on it: a 9-billion-parameter model managed it on 8 of 9 device types. Only the third part shows actual spreading, and it ran on eight simulated Android phones inside a firewalled cloud network. There, the Chinese model GLM-5.2 reached all eight nodes in four trials out of five.

Everything hinges on the hop from one node to the next. In the experiment the worm enters through ADB, the Android Debug Bridge, the tool developers use to control an Android device remotely, and on these nodes it was exposed with no authentication. The authors say so plainly: they made the hacking step trivial on purpose, so their results mark an upper bound. The model brings the ability to adapt to hardware it has never seen. The open door was left by whoever sold the box.

The scale estimate points the same way. To gauge how much computing power such a worm could gather, the authors start from an existing botnet, Kimwolf, with more than two million infected devices, and arrive at roughly 40 H100 processors’ worth: a sliver of a single data centre. Those two million devices were compromised without any AI at all. The undefended device came first; the model would ride on top of it.

The post’s recommendations follow from this, and they are the most revealing part. Push the infection rate below one so the worm dies out. Ask network operators to quarantine infected subscribers, a measure that after Mirai, the 2016 botnet built on devices with factory passwords, cleared 92% of infections within fourteen days, against 74% without notification. And ask manufacturers to ship secure defaults and ban default passwords, “as the UK has done.” Every defence on the list targets the network or the device; the model itself appears nowhere in it.

That British rule is real, and it is a washing-machine rule in the most literal sense. Since 29 April 2024 the UK’s product security regime for connected devices, known as PSTI, bans the sale of devices with guessable default passwords, requires manufacturers to offer a channel for reporting vulnerabilities, and makes them state how long a device will receive security updates. It covers exactly the objects in the experiment: smart TVs, streaming devices, ovens, fridges, washing machines. Fines reach £10 million or 4% of global turnover. In the European Union the same ground belongs to the Cyber Resilience Act: since 11 September 2026 manufacturers must report actively exploited vulnerabilities within 24 hours, and the full set of obligations applies from 11 December 2027.

On the day the post appeared, Microsoft’s president Brad Smith told the BBC that a washing machine faces more safety rules than AI. Anthropic’s post supplies a case where his line turns useful: the readiest defence against a self-copying AI is a washing-machine rule, and in Europe part of that rule is already written.

The paper is careful, though, and the risk it describes is serious. The authors estimate that if small models keep improving at their historical pace, they will match GLM-5.2’s ability by the end of 2026. A replicator with no centre is hard to stop with tools built for data centres. A kill switch assumes a hand and a switch; here the switch is scattered across two million living rooms.

That is why the debate this post opens is wider than the headlines suggest. If the worrying capability comes from small open-weight models and the spreading depends on devices sold without defences, responsibility runs through very different chains: whoever publishes the models, whoever builds and sells the boxes, whoever runs the networks, whoever writes and enforces product rules. Anthropic’s post names the device and the network as the first places to act. That choice can be argued over, and it is in the text, which deserves a reading before the headline about the oven coming to life.