If I may rephrase what I’m reading: You don’t want to tell him to leave you alone because you would be upset if someone told you that.

Here’s the thing: you don’t know that will upset him. TL;DR of the rest of my post: he probably won’t take it the way you would, and I highly recommend being straightforward with him.

I suggest reading about the difference in Ask Culture vs Guess Culture. Those of us who grew up in a guess culture manage our own actions based on what we think will be acceptable to those around us and won’t even initiate something if it would be deemed inappropriate, so it’s rare we have to be told “no”. Those in ask culture will just ask and be totally fine if told no, because they haven’t already done the pre-work to figure out if their request will be approved.

One of the best lessons I’ve had in the past few years is that other people don’t respond like me. I mean, that should be obvious. But it came up in the context of being a manager at work with an underperformer. I would be devastated if my boss told me I was not doing well at my job, and so I was terrified of telling my direct report that. I communicated the gaps in her specific actions for months, but we finally got to a point where I needed to have the conversation that I didn’t think the role was the right fit for her. It was one of the hardest days in my career. And she thanked me for it!

I was so scared because I was imagining how I’d feel hearing what I was going to say. But she’s not me! And instead of being upset, she felt relief to hear someone else say it.

You’re afraid of being rude, and that shows you have compassion and care for others. But I bet you that this coworker of yours just needs to be told, and not communicating with him is actually less kind.

A quote from a favorite book series of mine is a take on our “golden rule” through an alien culture: “The Iron Rule: Treat others less powerful than you however you like. The Silver Rule: Treat others as you’d like to be treated. The Golden Rule: Treat others as they’d like to be treated.”