Comment on Solve a puzzle for me
31337@sh.itjust.works 6 months agoThe set up is similar this well-known puzzle: en.wikipedia.org/…/Wolf,_goat_and_cabbage_problem
It was probably trained on this puzzle thousands of times. There are problem solving benchmarks for LLMs, and LLMs are probably over-trained on puzzles to get their scores up. When asked to solve a “puzzle” that looks very similar to a puzzle it’s seen many times before, it’s improbable that the solution is simple, so it gets tripped up. Kinda like people getting tripped up by “trick questions.”