You absolutely can refuse to hire someone for something they have no control of, assuming it’s not one of the few protected classes. I could refuse to hire you over height, inability to grow facial hair, etc with zero repercussions.