Sounds like you should take this up to him, and ask him yourself, having communication is really important, and remember, you could be doing something that also annoys him and he doesn’t say anything