Disorientation, panic and difficult terrain can all be factors. Getting injured in the process of trying to escape too.

Picture a dense forest. You know there is a fire, you vaguely know what direction it is probably in but that is where the path goes. So you go off the path away from it. Probably. Now you hit a river and try to follow the river away from the fire instead. Now hit some impassable terrain and the only option is to go back, but now that route is on fire.