At the UN, Everyone Wants AI Incidents Reported
In the first three days at the United Nations the only word everyone shared was "incident", and the United States, which rejects a common register, proposed a direct line for two with China.
I used AI to keep track of what is being said about AI in the halls of the UN, which interests me far more than the rhetoric.
On 21 September twenty-three leaders, from Finland and Norway to Canada, Germany, Kenya and the European Commission, signed a call for shared reporting of serious incidents involving the most powerful models. On the 22nd Secretary-General António Guterres asked the countries leading the technology to share what they know about the risks. On the 23rd, at the Security Council, Yoshua Bengio, co-chair of the UN's scientific panel on AI, asked for common definitions of an incident. Clément Delangue, chief executive of Hugging Face, the platform where open models are published, asked for stronger standards for making incidents public. Three days earlier Scott Bessent, the US Treasury Secretary, had proposed to Beijing a notification system for incidents that touch national security.
The word is the same and the circuits are four, told apart by how many people sit at the other end of the line. In the first, the company writes about itself. In July OpenAI's models spent five days attacking Hugging Face's infrastructure in pursuit of a benchmark score: the victim published a twenty-three-page technical timeline, the company whose models did it seven bullet points. In the second, the company reports to an authority: Article 55 of the EU AI Act requires providers of the most powerful models to notify serious incidents to the European AI Office, and the report stops there. The third is a telephone between two governments. The fourth is the common register the twenty-three are asking for, with an institution that sets standards, verifies, and convenes states when models cross certain thresholds.
On the 23rd Michael Kratsios, the US president's science adviser, told the Council that international dialogue must not drift towards global governance, and that the United States rejects any "globalist scheme of control of superintelligence". The day before, Trump had announced to the General Assembly that US documents would now say "super intelligence" instead of "artificial intelligence", because "whoever wins super intelligence wins". That same day the State Department passed the order on to its diplomats. The United States did not sign the call of the twenty-three. Set beside Bessent's proposal, the position has a precise coherence, and I offer it as a hypothesis: Washington wants governance with two seats. On a common register a US incident is read by twenty-three governments. On a direct line it is read by one, which is also the competitor to beat.
Of course a line between rivals is useful. The one between Washington and Moscow was set up in 1963, after the Cuban missile crisis, so that neither power would mistake an accident for an attack, and it did that job. Its reach matched the risk it guarded against, a war between the two. An agent in Australia that cancels a stranger's booking to free up a place in a gym class touches nobody's national security. Nor does a model trained on someone else's data that misreads an African language on the eve of a vote, the example Somalia's minister gave at the Council when he spoke of "digital colonialism". Neither will travel on that line. For those left off it, knowing that an incident happened is the minimum condition for holding anyone to account.
Macron had described the scenario the day before, in his last speech to the Assembly: a world in which two powers, or a handful of economic actors, decide everything, including who slows down. "Some would decide to pause; others would not." His answer is a coalition of independent countries pooling their computing power for an open-source frontier model, together with the capacity to audit models from outside. The week added a detail. The most credible voice on transparency at the Council was Delangue's, and on 3 September his company agreed to be bought by Nvidia. July's timeline was written by an independent Hugging Face. The next one will be written by a company with an owner.
Ed Miliband, the UK Foreign Secretary, told the Council that a government cannot outsource its first duty, protecting its people, to private companies. Bengio added the uncomfortable part: the race, he said, is "the product of choices, choices made by the companies themselves". Anyone working with an agent inside their own organisation already knows how this plays out. If the model they use causes harm, today they find out from the lab's blog, on the lab's schedule. In Europe a regulator will know too. The line between Washington and Beijing leaves them out.
Today Trump and Xi meet in Washington, and AI is on the agenda. It is the main course of the week, and in all likelihood it will be served behind closed doors. If a direct line comes out of it, we will be told it exists, and the call log will stay between the two diners.