You will be understood, it will just give people a small pause.

Sometimes it may cause confusion, like “the phone (he) went through the washing machine (she) and now <she/he> is broken” changes meaning if you get the pronoun wrong. But then if you are used to disambiguate that as you have to in english, it shouldn’t happen too often