Desired profile and fit are different.

I “passed” before HRT often just by wearing men’s clothes. The way that pants shape your butt is very different. I occasionally thrift women’s blouses and think I pull them off well, but these are ones from the 80’s and 90’s with the more “triangular” frame.

Women’s sizing is an incoherent mess. Men’s sizing at least in theory should be based on measurements. Historically, you would have been expected to have your clothes adjusted by a tailor anyway - this world of fast fashion, “ready to wear” means most of us are walking around with clothes that fit us poorly. Mass produced clothes are trying to fit some sort of “average” person, and none of us have a perfectly average body.

As far as shoes - there’s differences in shape and men’s feet on the whole tend to be larger. I think toebox proportions are different.

(The real danger of a “mixed gender” shopping section would be people realizing how shit women’s clothing is. Cheap flimsy material, lacking pockets, constructed to fall apart with a stiff breeze - designed to be disposed of. Shoes being designed with no thought for comfort or long term health…)