I’m always interested in seeing examples like this where the LLM will get to a right answer after a series of questions (with no additional information) about its earlier wrong responses. I’d love to understand what’s going on in the software that allows the initial wrong answers but gets the eventually right one without an additional input.
Comment on Solve a puzzle for me
tyler@programming.dev 6 months agoAFKBRBChocolate@lemmy.world 6 months ago
31337@sh.itjust.works 6 months ago
One hypothesis is that having more tokens to process lets it “think” longer. Chain of Thought prompting where you ask the LLM to explain its reasoning before giving an answer works similarly. Also, LLMs seem to be better at evaluating solutions that coming up with them, so there is a Tree of Thought technique, where the LLM is asked to generate multiple examples of a reasoning step then pick the “best” reasoning for each reasoning step.
kautau@lemmy.world 6 months ago
Honestly my answer felt super canned, like someone had asked it before and reported the answer as bad, so that doesn’t surprise me