Humans have sexual dimorphism, but it’s a cultural thing that women wear skirts and men drink themselves to death instead of talking about their problems.

Also, genetics is tricky, there a plenty of examples of people who do not fall into one category or another for these sexually dimorphic traits. There are people who have genetics from both sex, as well and differences in hormones distributions will causes these traits to appear or not appear.

Is a huge grey area.