We Are Losing Control?

Yoshua Bengio told AFP on Tuesday that we are losing control of AI and that the world needs guardrails like the ones on nuclear arms. Who says it matters. In 2003 Bengio published the neural language model every LLM descends from, and in 2014, with Bahdanau and Cho, the paper that introduced the attention mechanism: the part that decides which portions of the input count. The Transformer was built on top of it three years later. He shared the 2018 Turing Award with Hinton and LeCun.

Most people in this field met him as a textbook before they met him as a public voice. Deep Learning, written in 2016 with Goodfellow and Courville, is the book a generation trained on, and it read as engineering rather than prophecy. Then in 2023 he signed the letter calling for a six-month pause and said he felt lost about his life's work. The man who formalised how a machine decides what counts now spends his time asking who decides. He chairs the International AI Safety Report, which synthesises more than a hundred independent experts on behalf of thirty nations and by design issues no policy recommendations. When he reaches the extreme he names it and then withdraws it: "one extreme could be the destruction of humanity", and at once, "that's an extreme", and the case he actually cares about is smaller and nearer, a banking system switched off. He is measuring.

The nuclear analogy is an analogy about counting, and the answer already has a name that rarely leaves the policy papers: compute governance. IAEA safeguards are nuclear material accountancy: inspectors audit a facility's accounting and operating records against the state's declarations, tamper-proof seals and cameras hold the containment, and inspections at the strategic points where material flows take environmental samples and radiation measurements to confirm that what is present matches what was declared. That machinery exists for a physical reason. Uranium has mass, it decays, it leaves traces in soil, and it cannot be in two places at once. Model weights have none of those properties. A copy leaves the original intact and the ledger balanced, and no soil sample will ever reveal it. A treaty needs an inventory, and for weights there is none.

The inventory does exist, and it is not where anyone is looking for it. Data centres have an address, a building permit, a power contract and an electricity bill; chips carry serial numbers and clear customs. Compute is countable, with the same dull instruments that have counted physical things for centuries. The difficulty that makes a treaty on models impossible points, in the same gesture, at the object on which a treaty is possible. A treaty needs an inventory, and this one has a postcode.

Counting has a limit of its own, and the IAEA learned it in the field. Safeguards verify what a state declares, and the agency's experience in Iraq and North Korea in the early 1990s showed it was, in its own words, "not well-equipped to detect undeclared nuclear material and activities". The Model Additional Protocol, approved in May 1997, exists to cover that gap, and it took two undeclared programmes to write it. An inventory is necessary and it is not sufficient. Whatever the equivalent turns out to be for a data centre nobody declared, it has not been drafted.

Which makes Wednesday's announcement more interesting than the coverage made it. In Montreal, the day after the interview, LawZero, the non-profit Bengio founded in June 2025 when he stepped down as Mila's scientific director, received a commitment of up to CAD 300 million: CAD 150 million from Canada and EUR 100 million from Germany, the German share subject to notification to the European Commission. Three hundred and sixty jobs, a Berlin office, and sovereign compute built with Hypertec and 5C. The answer to losing control is a contract for megawatts in Quebec, and even the people asking for treaties, once words turn into spending, buy the thing that can be counted.

Two details, because precision is the whole value here. The partners are two names in one orbit: 5C Group was formed in April 2025 out of 5C Data Centers and Hypertec Cloud, and The Logic describes it as Hypertec's data centre arm. Since the autumn of 2025 Mila's technical staff have been working on the infrastructure at Hypertec's campus in LaSalle, which is adjacency and not a link: that arrangement is Hypertec's, no site has been announced for LawZero's compute, and Bengio left Mila's scientific direction last year. On the sequence of days, AFP writes on its own account that Bengio spoke the day before the announcement. Nothing indicates coordination, the money is public, and for now it is a ceiling. An argument can be right and carry a stake, and both belong on the record.

What they intend to build is the finest idea in the story. Scientist AI is designed to be non-agentic, to "reason transparently and provide reliable, evidence-based outputs that are not biased by goals of its own". A machine built to explain rather than to want, with no memory and no state, and one operational task: to say whether the action an agent proposes is likely to cause harm. Judging that reliably means modelling the domain the action happens in, which is close to the work of the system it is watching, and no evaluation has been published. It is in development, not in service, and it may fail. Cynical it is not.

The question that project cannot dodge is who writes the definition. The guardrail is itself a model, it runs on hardware with an owner, and somebody has to set what counts as harm. That definition will be the most powerful thing produced in Quebec, more than any of the hardware it runs on. Written in public, with outside review, it is a common good. Written in a private repository, it is the pen.

The institutions answered this week. On Wednesday in Strasbourg, von der Leyen told the European Parliament: "CEOs of the most advanced companies tell us that it is time to slow down on the self-recursive models. To pace the frontier. If the people developing the technology are clear, then we should be too." And: "I will invite the main frontier labs for a discussion on how we can support ongoing industry efforts to pace the frontier." The stated ground of the European position is that the CEOs have been clear, and the table she convenes exists to support an effort the industry is already making. In the same speech Europe commits to work with Canada and the United Kingdom on model evaluation, verification and AI security. That line, not the other one, is the one to watch.

For the next two years you will be told that governing these systems is technically impossible. It is worth knowing exactly where the impossible ends: nobody will ever count the copies of a weights file, and anybody can count a data centre that somebody declared. A treaty needs an inventory, and for the first time the inventory is being poured in concrete, with an address and a building permit. It is not much. It is the first time the thing that needs watching can be seen from the road.