it feels like colleagues talk bad about me.

What makes you say that? It’s pretty likely that’s just your anxiety talking.

I’m not exactly the most social—or even socially competent—person, but I know friends are important to have, especially if you don’t have family you can lean on. Friends can balance you out and provide alternate perspectives when you need them, and we all need them sometimes.

Also, I’m pretty sure having (good) friends is beneficial for your mental health. Having no friends can be detrimental to it.

Regardless of what you decide, I suggest putting yourself out there. Start slowly, and ask people about themselves. It can very difficult and stressful at first, but it does get easier over time.

It might be a good idea to make a pros/cons list for staying/getting a job; I’m sure there are other important factors to consider aside from the social climate.