Nah, reach is a huge advantage. I’m not sure how rapier fencing differs from regulation sabre/epée/foil, but here’s my 2 cents from that perspective:

Smaller people are not, as a rule, substantially quicker than larger. If you see any difference in your experience, it’s likely a selection bias (shorter people have to be quicker to compete at the same level). The shorter person must enter the strike range of the taller person before the taller person comes within theirs and must be significantly quicker or more skilled to overcome that dead space. If the taller person can maintain a proper distance, gg. Taller people can also lunge farther, giving a wider active range.

Targeting is a smaller issue than you make it out to be; footwork and maintaining balance, which reposition the core, are at least as important as leaning to dodge, and advantage the taller person (longer legs = more movement range). If the taller person is coming from above as you say, they can just continue their slash (sabre) downward toward that less mobile core, or squat a bit deeper if the arc won’t reach. If instead you were referring to a poke, they’re either already targeting the torso anyway (foil) or whatever body part is most easily reachable (epée; still often torso, but cheeky wrist/arm strikes can be something of an equalizer here), and anyway they are already striking at a range that the shorter person cannot, making a successful counterattack more difficult.

Besides reach, a height difference is brutal when it comes to sabre fencing; the shorter person is restricted to targeting arms and torso (can’t reach the head easily), so the taller person can anticipate strikes from fewer angles. The taller person can come from any direction and has gravity on their side for own overhead strikes. Those suck to defend against.