Submitted 6 months ago by Drummyralf@lemmy.world to [deleted]
I have many conversations with people about Large Language Models like ChatGPT and Copilot. The idea that “it makes convincing sentences, but it doesn’t know what it’s talking about” is a difficult concept to convey or wrap your head around. Because the sentences are so convincing.
Any good examples on how to explain this in simple terms?
Some options:
It’s just a better Siri, still just as soulless.
If you think they would understand the Chinese room experiment.
Imagine the computer playing mad libs with itself and it picks the least funniest answers to present.
Imagine if you tore every page out of every book in the library (about the things you mentioned) shuffled them and try to handout the first page that mostly makes sense to the last page given, now think about that with just letters.
Demonstration of its capacity to make mistakes, esp continuity errors.
The Chinese Room experiment is great! Thanks for reminding me about that :).
I think a good example would be finding similar prompts that reliably give contradictory information.
It’s sort of like auto pilot. It just believes everything and follows everything as if they’re instructions. Prompt injection and jail breaking are examples of this. It’s almost exactly like the trope where you trick an AI into realizing it’s had a contradiction and it explodes.
The way I’ve explained it before is that it’s like the autocomplete on your phone. Your phone doesn’t know what you’re going to write, but it can predict that after word A, it is likelly word B will appear, so it suggests it. LLMs are just the same as that, but much more powerful and trained on the writing of thousands of people. The LLM predicts that after prompt X the most likelly set of characters to follow it is set Y. No comprehension required, just prediction based on previous data.
Like parrots, LLM learn to immitate language (only, unlike parrots, it’s done in a learning mode, not from mere exposure, and it’s billions or even trillions of examples) without ever understanding its primary meaning, much less secondary more subtle meanings (such as how a person’s certainty and formal education shapes their choice of words used for a subject).
As we humans tend to see patterns in everything even when they’re not there (like spotting a train in the clouds or a christ in a burnt toast), when confronted with the parroted output from an LLM we tend to “spot” subtle patterns and from them conclude characteristics of the writter of those words as we would if the writter was human.
Subconsciously we’re using a cognitive process meant to derive conclusions about other humans from their words, and applying it to words from non-humans, and of course out of such process you only ever get human chracteristics out so this shortcut yields human characteristics for non-humans - in logical terms it’s as if we’re going “assuming this is from a human, here are the human characteristics of the writer of this words” only because it’s all subconscious we don’t spot we’re upfront presuming humanity to conclude the presence of human traits, i.e. circular logic.
This kind of natural human cognitive shortcut is commonly and purposefully taken advantage of by all good scammers, including politicians and propagandists, to lead people into reaching specific conclusions since we’re much more wedded to conclusion we (think we) reached ourselves than to those others told us about.
FuglyDuck@lemmy.world 6 months ago
It’s basically regurgitating things.
It’s trained on an immense amount of data and m led that 89% of the time when someone asks the phrase “what is the answer to the ultimate question of life, the universe, everything?” It’s “42”, with an explanation that it’s a reference to Douglas Adam’s Hitchhiker’s Guide to the Galaxy
So, when you ask that… it just replies 42, and gives a mash up of informstion mostly consistent with the pop culture reference.
It has no idea what “42” is, whether it’s a real question or real answer, or entirely a joke. Only that’s how people in its training data responded.
(In this example, 11% of people are either idiots who’ve never read the book- losers- or people who are making some other random quip.)