Tiny models only get stupid like that because you’re compressing a general purpose model that knows everything in the world farther than it can handle. If you start with a model that only knows how to speak basic english and info about a few hundred things in the game world, it can be much smaller.