I think comparing gender or sexual orientation to a disability is also not the way to go. Its a very different experience with very different challenges in our current society.

If what you say is true about not being anti queer, I think you should try to refelct the way you view and talk about queer people. The reactiona to ypur comments are so big/aggressive because the rhetoric you use is very similar to the rhetoric many bigots use when talking about queer folks. So even if you mean it differently, from our perspective it really looks the same / similar.