I understand that it bothers you when he asks but maybe it’s because you feel obligated to get up and go?

Have you tried not doing that? Seriously, tell that dude to tell you what it is or ignore him until he comes to you. You’re the master of your brain, big dawg. Don’t stress 🙏