Alice in Wonderland: Simple Tasks Showing Complete Reasoning Breakdown in LLMs

⁨3⁩ ⁨likes⁩

Submitted ⁨⁨2⁩ ⁨years⁩ ago⁩ by ⁨bot@lemmy.smeargle.fans [bot]⁩ to ⁨hackernews@lemmy.smeargle.fans⁩

https://arxiv.org/abs/2406.02061

Comments

Sort:hotnew top

BumpingFuglies@lemmy.zip ⁨2⁩ ⁨years⁩ ago
After a quick glance this appears to be the question that LLMs can’t answer:

Alice has N brothers and she also has M sisters. How many sisters does Alice’s brother have?

source
- Umbrias@beehaw.org ⁨2⁩ ⁨years⁩ ago
  It’s wonderfully severe too.
  
  source