Sure, we had fires before we started refining and making accelerants, but that doesn’t mean that we should just let people throw accelerants on fires willy nilly.
“People died in car accidents before we started putting MurderSpikes™ on the front of our cars, and they will continue to die in car accidents after we stop putting MurderSpikes™ on the front of our cars.”
We then show that in this model, even an idealized Bayes-rational user is vulnerable to delusional spiraling, and that sycophancy plays a causal role. Furthermore, this effect persists in the face of two candidate mitigations: preventing chatbots from hallucinating false claims, and informing users of the possibility of model sycophancy.
So, while I agree with your gist of ‘chatbots are bad for people’, I think you may have overstated your claim: That even ideally rational ‘people’ can get led into AI psychosis by sycophantic chatbots is not the same as bots “affecting regular people just as much as mentally unstable people”, because mentally unstable people might be more vulnerable to this type of reality-warping influence. “No-one is safe” is different from “We all suffer equally”.
I will assume he was taking advice from ChatGPT. That is just the kind of dumb shit ChatGPT tries to convince people to do.
LOL at all the downvotes, as if ChatGPT doesn’t tell people to do stupid shit.
People have been mentally ill before AI, and will continue to be after AI
LLMs are out there literally egging people on.
Sure, we had fires before we started refining and making accelerants, but that doesn’t mean that we should just let people throw accelerants on fires willy nilly.
“People died in car accidents before we started putting MurderSpikes™ on the front of our cars, and they will continue to die in car accidents after we stop putting MurderSpikes™ on the front of our cars.”
No, there are by now several research papers showing that sycophantic chatbots are affecting regular people just as much as mentally unstable people.
Source? I
googledweb searched your claim, and this paper, “Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal Bayesians”, was the top result. It does conclude, as expressed in the Abstract:So, while I agree with your gist of ‘chatbots are bad for people’, I think you may have overstated your claim: That even ideally rational ‘people’ can get led into AI psychosis by sycophantic chatbots is not the same as bots “affecting regular people just as much as mentally unstable people”, because mentally unstable people might be more vulnerable to this type of reality-warping influence. “No-one is safe” is different from “We all suffer equally”.
How did the papers screen the participants to make sure they’re “regular people”
this ain’t reddit bro.