Chatbots used as confidants, and nobody takes responsibility

It was the British Psychological Society, the professional association of British psychologists, that put its questions to 197 psychologists, counsellors and psychotherapists, as the Times reports. The result highlights, once again, the risks of using AI: 76% of respondents say chatbots are harming clients and leaving them lonelier, and 88% point to the lack of clinical supervision. Over the past year, practitioners have seen a 49% rise in patients turning to AI for emotional support, while waiting lists in the state health service (the NHS) grow longer and private care is expensive.

I read it as one more alarm bell going off. These are 197 people describing what they see in their practice, they have a stake in the field, and nobody knows what someone who cannot find a place today would do without the chatbot. But the bell is ringing in the right place: that is where the harm would show up first.

That leaves the question of who is responsible. The tendency is to look at the user who relies too much, but that frailty is not something we have just discovered: cinema and literature have been telling it for decades, and anyone who has watched someone attach themselves to a voice that always answers knows it. If it is known, it is foreseeable; if it is foreseeable, whoever sells a product that talks is answerable for it. I suspect that in some products this weakness is exploited on purpose, because revenue follows time spent (in apps built as companions, the bond is the product), and that in others it is a side effect of models trained to please. From the outside I cannot tell the two apart, but for responsibility it makes little difference.

That is why one respondent’s remark matters more than the percentages: AI is seen as “free”, and for that reason the companies that run it have free rein, with “little to no legal responsibilities”. The same holds for a company that adopts the tool for its own employees, as support, wellbeing aid or assistant. When it buys the tool and has people use it, it has a duty to know what it is buying. I suspect the chain of responsibility is only too glad to snap in the middle: the producer says the company chose it, the company says the supplier made it, the employee says it was put in front of them. European AI law distinguishes between whoever produces a system and whoever deploys it [check the obligations of deployers]; in the UK, as far as I know, there is no equivalent, which explains why it falls to psychologists to ask for rules through a survey.

The society’s president, Laura Dean, says AI cannot replace “the vital human element” in care. But the point that matters here is not who speaks, it is who answers. A psychologist has recognised training, supervision and insurance, and can be called to account for what they say; a chatbot cannot. If it had comparable obligations, even without the same voice, the conversation would change, and I suspect that is why those who sell it prefer not to have them.