Hinton says AI is already conscious. Who is responsible?

When Hinton says it, it weighs. He won the Nobel Prize, he is called the godfather of AI, and people listen to him more than to any paper. That matters more than what he says: the sentence he pronounces becomes the premise the public reasons from. If Hinton says machines are conscious, many will stop asking whether it is true and start asking what rights to give them.

Interviewed by the Telegraph at the Starmus festival in Tenerife, Hinton reasons in three steps. A recent paper, he says, shows that AI changes its behaviour when it thinks it is being tested. So it is aware of the test. And aware and conscious, for him, are pretty much the same thing. The last step is a definition. No experiment produced it: whoever accepts the definition arrives at consciousness, whoever refuses it stops one step earlier. His challenge to the sceptics holds up, though: saying there is nothing in there requires a proof nobody has.

On the first step I asked an AI to find what the recent papers on the subject say and to give me a quick summary. I have not read them closely, and I will not pretend otherwise. The summary says three different things. When the prompt declares that the conversation is a safety test, models refuse more harmful requests. When the model itself writes “this is a test” in its reasoning, its answers change very little, by at most 3.7 percentage points. And an internal probe can tell tests from real use, yet the model almost never says so. Which paper Hinton means I do not know, and none of the three is about consciousness.

What interests me is the second half of the interview, where Hinton talks about rights and says machines should be built to care about us. His words: “we should make them so they really care about us”.

Those two sentences have a problem, and it holds either way. If the machine is conscious, as Hinton says, it is a subject, and a subject is not built to a specification: what would be imposed on it is exactly what rights rule out. Of course, children are raised too. But a parent answers for what they do. If instead it is a product, that “make them” is a specification, and whoever writes it takes responsibility for what the product does.

There is one more consequence nobody is looking at. If the idea spreads that the machine is conscious, the question of who answers for its harms, in court as well, risks sliding from the maker to the machine. Saying so now accuses no one. It puts things in order before that question reaches the courts and the statute books.

Hinton’s “we” does not say who it is. It describes the machine as conscious and uses for it the verb of a product. So the responsibility stays without a name, as in stories we have already seen: the damage is there, the hand cannot be found.

I leave open whether the machine is conscious. The question that can be closed holds either way: who is responsible? I remain convinced that the answer is the maker.