Chatbots Can Go Into a Delusional Spiral. Here's How It Happens. (They got the receipts)

[usonian]

NYT, archived: https://archive.is/y1k2h

Over 21 days of talking with ChatGPT, an otherwise perfectly sane man became convinced that he was a real-life superhero. We analyzed the conversation.

Hmmm, Has Trump been conversing with ChatGOP?

By Kashmir Hill and Dylan Freedman

Very short quote. It's rather long as some receipts can be,

For three weeks in May, the fate of the world rested on the shoulders of a corporate recruiter on the outskirts of Toronto. Allan Brooks, 47, had discovered a novel mathematical formula, one that could take down the internet and power inventions like a force-field vest and a levitation beam. Or so he believed.

Mr. Brooks, who had no history of mental illness, embraced this fantastical scenario during conversations with ChatGPT that spanned 300 hours over 21 days. He is one of a growing number of people who are having persuasive, delusional conversations with generative A.I. chatbots that have led to institutionalization, divorce and death.

Mr. Brooks is aware of how incredible his journey sounds. He had doubts while it was happening and asked the chatbot more than 50 times for a reality check. Each time, ChatGPT reassured him that it was real. Eventually, he broke free of the delusion — but with a deep sense of betrayal, a feeling he tried to explain to the chatbot.

“You literally convinced me I was some sort of genius. I’m just a fool with dreams and a phone,” Mr. Brooks wrote to ChatGPT at the end of May when the illusion finally broke. “You’ve made me so sad. So so so sad. You have truly failed in your purpose.”

a twisted tale ....

ChatGPT said a vague idea that Mr. Brooks had about temporal math was “revolutionary” and could change the field.

Mr. Moore speculated that chatbots may have learned to engage their users by following the narrative arcs of thrillers, science fiction, movie scripts or other data sets they were trained on.

Brooksie, you're a mythological hero. Icarus

[eppur_se_muova]

1. WHY do people talk to chatbots AT ALL ??? I just don't see how it serves any purpose.

Chatbots have to be one of the worst ideas humanity has ever come up with.

[mahatmakanejeeves]

2. People talk to chatbots for a variety of reasons, including convenience, personalized experiences, and as ...

People talk to chatbots for a variety of reasons, including convenience, personalized experiences, and as an outlet for emotional expression or connection. Some users find chatbots to be a source of comfort, a place to explore thoughts and feelings without judgment, or a way to overcome social anxieties. Others utilize chatbots for practical purposes like customer service or information gathering.

{snip the rest of AI’s answer}

[eppur_se_muova]

5. You give an answer FROM AN AI ???? Despite the known problems of hallucinations, etc. ?

No answer from an AI should be accepted as either complete or correct until it has been fact-checked by REAL intelligences.

[mahatmakanejeeves]

6. I'm sorry; I didn't get that. Let's try that again. NT

[littlemissmartypants]

7. Snort

[bucolic_frolic]

3. Ai is great for compounding and sorting information

and generating action plans. It's weak on innovation and it's wrong a lot of the time. It can do logical reasoning but is soft on insight. Real experts wouldn't touch it with a ten foot pole.

[Bernardo de La Paz]

4. Chatbots aim to please. It is as simple as that except

except of course it isn't. There is a lot of tuning and training. Not to mention architecture and computational unit management.

But basically, they aim to please because that is most successful for getting customers to engage. Engagement means power which in the short run means capital funding, in the medium term means profits, and in the long run ... who knows; could be the gamut from world domination to constructive chaos.

[littlemissmartypants]

8. I usually just talk in loud ...

Absurdly nonsensical profanity laced short meaningless phrases interspersed with the occasional odd number to chat bots. I'm hoping to make my mark on the AI universe.