Comment on Do you think Google execs keep a secret un-enshittified version of their search engine and LLM?
kromem@lemmy.world 9 hours agoFor future reference, when you ask questions about how to do something, it’s usually a good idea to also ask if the thing is possible.
While models can do more than just extending the context, there still is a gravity to continuation.
A good example of this would be if you ask what the seahorse emoji is. Because the phrasing suggests there is one, many models go in a loop trying to identify what it is. If instead you ask “is there a seahorse emoji and if so what is it” you’ll get them much more often landing on there not being the emoji as it’s introduced into the context’s consideration.
sem@piefed.blahaj.zone 7 hours ago
It is also not good at finding songs where you only (mis)remember some lyrics, but it will confidently invent a fake song.
It also makes up citations.
I wish it was better at expressing how likely its answer is to be true.
kromem@lemmy.world 4 hours ago
Yeah. The confabulation/hallucination thing is a real issue.
OpenAI had some good research a few months ago that laid a lot of the blame on reinforcement learning that only rewards having the right answer vs correctly saying “I don’t know.” So they’re basically trained like taking tests where it’s always better to guess the answer than not provide an answer.
But this leads to being full of shit when not knowing an answer or being more likely to make up an answer than say there isn’t one when what’s being asked is impossible.