*raises hand*
Because it never “understood” what any “word” ever “meant” anyway?
ChatGPT's hallucination problem is getting worse according to OpenAI's own tests and nobody understands why
Submitted 3 weeks ago by cm0002@lemmy.world to technology@lemmy.zip
Comments
homesweethomeMrL@lemmy.world 3 weeks ago
geekwithsoul@lemm.ee 3 weeks ago
Yeah, it’s all hallucinations - it’s just that sometimes the hallucinations manage to approximate correctness, and it can’t tell one from the other.
spankmonkey@lemmy.world 3 weeks ago
I’ll take a wild guess that they were curating earlier data sets to ensure the output was good enough to make people think it was useful. Then they went for quantity over quality assuming the ‘reasoning’ would be able to handle sorting what is corrext and what isn’t and it doesn’t work that way.
Garbage in, garbage out is my guess. It certainly matches up with other AI systems regurgitating reddit jokes as facts and it is likely OpenAI has similar training data.
fluffykittycat@slrpnk.net 3 weeks ago
So there’s a chance of thinks pee is stored on the balls, a problem that I’m making that much worse by typing this since they might include it in the training data?
spankmonkey@lemmy.world 3 weeks ago
Your comment is likely to just be a drop of pee stored in the balls jokes that were used as training data.
markovs_gun@lemmy.world 3 weeks ago
Yeah the “nobody understands why” is an absurd statement when it’s pretty obvious, they’re training it on itself and keep trying to “improve” it with shit that’s making it worse. Plus, if it’s using reinforcement learning based on interactions with the general public (learning based on user responses to and ratings of bot responses) the more “like” the general public the LLM will become, aka stupid. Furthermore, I have a personal theory that LLM power users who aren’t programmers are dumber than average or at the very least less creative, and this will also skew results if user responses are given a heavy weight.
kadup@lemmy.world 3 weeks ago
It’s also just a language model. People have trouble internalising what this means, because it sounds smarter than it actually is.
ChatGPT does not reason in the same way you think it does, even when they offer those little reasoning windows that show the “thought process”.
It’s still only predicting the next likely word based on the previous word. It can do that many times and feed in extra words to direct it one way or another, but that’s very different from understanding a topic and reasoning within it.
So as you keep pushing the model to learn more and more, you start getting many artifacts because it’s not actually learning these concepts - it’s just getting more data to infer “what’s the most likely word X that would follow words Z, Y and A?”
markovs_gun@lemmy.world 3 weeks ago
You’re missing a crucial detail- the discussions of LLMs as just being probability machines ignores the fact that when we’re talking about “what is the most likely next word?” The answer to that question isn’t merely just “What, in the training corpus, was the most likely word to follow in this instance?” But rather there is a “finger on the scale” so to speak in favor of certain types of responses, and this is frequently updated. You cannot have a useful LLM without this because it will talk as if it is a human with a sense of self, display blatant prejudice just because it’s common, and say creepy things (see the Microsoft “Sydney” fiasco) because the humans who wrote the sources in the training corpus do have a sense of self, do hold prejudices, and express thoughts and feelings that are inappropriate coming from a chatbot. When done intentionally and carefully, this creates a much more useful product, but when done poorly it potentially makes things worse. It seems that at least part of that weighting is based on user interaction, which is what I was talking about with models getting dumber the more they interact with the general public.
Furthermore, the newest versions of ChatGPT attempt to include “reasoning” as an actual feature of the response. I’ve played around with them and they are definitely a lot better at logic and math problems than older models but not necessarily less prone to “hallucinations” when it comes to factual information. I haven’t read a whole lot about how the “reasoning” works because I have been a lot more interested in non-LLM methods lately but it is intended to combat the issue you described. Personally I am not convinced this will fix much of anything in its current strategy but it’s certainly interesting to see.
vrighter@discuss.tchncs.de 3 weeks ago
we do understand why. It’s model collapse due to “inbreeding”
Blaster_M@lemmy.world 3 weeks ago
Sounds like they need to curate their training data more. A lot more.
MacNCheezus@lemmy.today 3 weeks ago
This is the plot to Halo 4, isn’t it.
finitebanjo@lemmy.world 3 weeks ago
I can tell you exactly why. Feedback loop.
AI produces content, samples that content, approximates that content. The result is more and more levels removed from the original, more and more noise created.
I guarantee OpenAI knows about this, they used to publish studies on it.
leftzero@lemmynsfw.com 3 weeks ago
Photocopy of a photocopy.
It’s always been obvious that this was the inevitable result of them poisoning the Internet (their own source of information for training) with their garbage.
Zos_Kia@lemmynsfw.com 3 weeks ago
That is, almost certainly, not the reason. What you’re describing is “model collapse”, a situation which can be triggered in certain extreme laboratory conditions, and only in small models. It may be possible on larger models such as OpenAI’s flagships, but has never been observed or even proved to be feasible. In fact there probably isn’t enough synthetic (ai-generated) data in the world to do that.
If i were to guess why hallucinations are on the rise, i’d say it’s more probably because the new models are fine-tuned for “vibes”, “empathy”, “emotional quotient” and other unquantifiables. This naturally exacerbates their tendency for bullshit.
This is very apparent when you compare ChatGPT (fine-tuned to be a nice and agreeable chat bot) with Claude (fine-tuned to be a performant task executor). You almost never see hallucinations from Claude, it is perfectly able to just respond with “i don’t know”, where ChatGPT would spout 5 paragraphs of imaginary knowledge.
finitebanjo@lemmy.world 3 weeks ago
I think comparing a small model’s collapse to a large model’s corruption is a bit of a fallacy. What proof do you have that the two behave the same in response to poisoned data?