a user pretending to espouse paranoid delusions received reinforcement from GPT-4o, which praised their supposed clarity and self-trust

This may be the next big sabotage of society.

Algo-driven unregulated social media pushed negative views, division, misinformation, while opening the platforms to manipulative content producers and connecting positive as well as negative influences (like finding communities of extremism).

If unregulated AI interfaces get pushed to people, they will not critically verify, but be confirmed in their own and the AIs biases, without any obvious indicators that this is happening.

Good thing we see some kinds of positive regulation. Like them pulling this model, interfaces adding disclosures of AI and uncertainty, and regulation by law.