Since this article was first published, Koko founder Rob Morris clarified some details of the experiement. We have updated the article to reflect this.
A mental health service that allows people to receive encouraging words of support and advice from others has received criticism after announcing it tested AI-generated responses.
Rob Morris, founder of the free mental health service Koko, outlined in aĀ Ā how the firm tested using a chatbot to help provide mental health support to about 4000 people. The chatbots were powered by GPT-3, a publicly available AI built by San Francisco-based company OpenAI.
Advertisement
The test enabled users of Kokoās online peer support network to enlist a chatbotās help in composing ākind wordsā as responses to other peopleās posts.
Morris described Koko users as rating AI-composed messages āsignificantly higher than those written by humans on their ownā, but also said that āonce people learned the messages were co-created by a machine, it didnāt work. Simulated empathy feels weird, empty.ā
One element of the experiment that has drawn criticism was the process by which recipients found out messages had been composed with the help of the chatbot. Initially, it seemed that there was a period where people were completely unaware, though Morris has since said that wasnāt the case and that those messages included a note saying āwritten in collaboration with Koko Botā.
The experiment āraises significant ethical and moral concernsā, saysĀ Ā at the AI Now Institute, a research centre in New York City.
Multiple researchers, tech developers and journalists responded on Twitter by describing the demonstration as unethical, citing issues aroundĀ Ā and the failure to first run the experiment by anĀ Ā (IRB) ā a group specifically tasked with protecting the welfare of research subjects. Morris says the experiment Ā from informed consent.
On its website, Koko says over 2 million people ā most of them adolescents ā have used its mental health support services.
There are many examples of people knowingly consulting chatbots for online advice and support, including the early example of computer scientist Joseph Weizenbaumās ELIZA that was developed in 1964. But this particular experiment āis deserving of every bit of the close scrutiny itās currently gettingā, says West.
