Comment on ChatGPT's o3 Model Found Remote Zeroday in Linux Kernel Code
shnizmuffin@lemmy.inbutts.lol 3 days agoIf I were to ask my Magic 8 Ball “Is the word ‘difinitely’ misspelled?” 100 times, it’s going to reply in the affirmative over 16% of the time. Literally double. This would also be “the very first experiment in this use case, done by a single person on a model that wasn’t specifically designed for this.”
It’s not impressive.
The issue with hallucinations…
This is the real problem: working under the false assumption that there are two kinds of output. It’s all the same output. An LLM cannot hallucinate in the same way that it cannot think or reason. It’s fancy autofill. Predictive text.
You can use it to brainstorm creative solutions, but you need to treat its output for what it is: complicated dice rolls from the tables in the back of the Dungeon Masters Guide. A fun distraction. Implausible fantasy 9 times out of 10.
knokelmaat@beehaw.org 3 days ago
This comparison makes no sense. Your example has a binary question. In that case, any system that replies correctly at even a rate of around 50% would be useless. However, the problem space in this scenario is way larger than 2 options and still way larger than 100 options. Being correct in even a small number of 100 attempts is still statistically significant.
The fact that an LLM is unable to reason and that it is based on statistics doesn’t change anything about this behavior. At the end of the day you get a tool that is able to point you to actual new information that you by yourself did not arrive at.
Imagine that you put a lot of effort in a better model specifically for vulnerability research and you get it up to a correctness rate of a mere 10%. I would gladly hire some programmers to sift through these reports and possibly find overlooked vulnerabilities.