AI as a Mental Prosthesis

In the US it has become common for people in therapy to have intense relationships with robots and bots. The big outlets are paying attention:

https://time.com/7379564/ai-emotional-intelligence-support-bots/

The dangers of the mental prosthesis, much bigger than deepfakes: https://venturebeat.com/technology/what-if-the-real-risk-of-ai-isnt-deepfakes-but-daily-whispers

And this looks to me like the same problem as social networks getting us hooked on short reward cycles, taking over our day: https://neurosciencenews.com/effort-recalibration-digital-media-cognition-30985/

TIME

Millions of people now trust AI with their feelings. Can they trust the companies creating it to prioritize their welfare?

At least once a month, two-thirds of people who regularly use AI turn to their bots for advice on sensitive personal issues and emotional support.

Many people now report trusting their chatbots more than their elected representatives, civil servants, faith leaders—and the companies building AI. That’s according to data from 70 countries, gathered by the Collective Intelligence Project (CIP). As CIP’s research director, neuroscientist Zarinah Agnew, puts it, AI is becoming “emotional infrastructure at scale.” And it’s being built by companies whose economic incentives may not align with our wellbeing.

Already, we’ve seen instances of AI companies optimizing their models to keep people engaged, even when this goes against their best interests. Last April, OpenAI had to roll back an update to one of its ChatGPT models after it was widely-criticized for being overly-flattering to users. When the company stopped offering the model to people, the day before Valentine’s Day, some were distraught.

Humans finding comfort in machines is not new. In the late 1990s, MIT Professor Rosalind Picard—who founded the field of affective computing—found that people responded positively to computers performing empathy. But two key things have changed since then: thanks to technical advances, AI systems today are new entities, capable of sophisticated conversation and surprising behavior; and thanks to the billions of dollars investors have poured into AI companies, these entities are accessible to virtually anyone with an internet connection. ChatGPT alone currently has more than 800 million weekly active users—and the number is growing.

But with millions of people forming different kinds of human-machine relationships, we don’t yet know whether AI is helping more people than it harms. And meanwhile, AI companies are investing in making their models not just smarter, but also more emotionally savvy—better at detecting emotion in a person’s voice, and at responding appropriately.