Lets say you go to a public building and in that public building there is a room marked women only, lets say in that room are some toilets, would go in that room? Since it’s a public space in the same building as all the other public space, the only difference is that portion of the space is understood to be only for women, or those that identify as women.

You may stumble in accidentally, and you will be gently corrected, but if you keep stumbling in, it’s gonna start to seem weird.