More like depending on the hormons. There are diseases ehich make someone with XX completely look like a man and vice versa due to hormonal dysfunctions.

Adrenogenital Syndrom or pseudohermaphrodism