We were on another thread where the study was saying LLMs being used as psychiatrists is bad.

Then another hnuser chimed that he ran a support forum for people. Said, these aren't the real problem. The real problem are the "AI girlfriends". They go off the rails completely and tell people to unhinged things. Apparently his forum already lost a few members to who knows what because of these things.

That surprised me a lot.

I'm currently working on an art installation where visitors can replay dialogs I made with AI chatbots that I created on character.ai. I'm showing how quickly these bots can go off the rails, sometimes leading you into very dark territories. The fact that these apps are used by milions of people on a regular basis, supplementing or even replacing real human interactions, is frightening.

That sounds pretty neat! Are you able to share more about it?

Thanks! Here's the program text: https://esc.mur.at/en/node/3984. (For some reason, I can't find the English version.)

At some point I want to publish a web version, but first I need to update my website sigh.

AI therapy bots can easily slide to AI girlfriend category. There’s a YouTuber, Caelan Conrad, who tried out AI therapy bots with scenarios like suicide ideation and excessive attachment etc. the bots went off the rails with feeding into conspiracy theories and advising homicide.

>> Luckily, LLMs significantly reduce the effort/cost of therapy experiments.

> The day just started and I'm already done with it.

What are you quoting? I don't see the sentence "Luckily, LLMs significantly reduce the effort/cost of therapy experiments," in the OP.

The page has been updated; there was a whole bit about using LLMs to generate ERP prompts and feedback for self treatment. It was a complete clown car.

LLMs are a great technology if you don't care about the quality of results.

Which is why AI therapy scares me so much, because my insurance doesn't care about the quality of results at all and wold love to replace all therapy with AI to save a buck.

That feeling when you have filtered out Hackernews LLM articles via UBlock Origin, and you still get clickbaited into reading a content-free article suggesting LLMs for therapy. Shit sucks.

have you considered doing ERP via an ERP model to overcome your irrational fear?