The lower bound on what age a person has to be before being sexually attracted to them goes up as you age, well past 18-20 by the time you’re in your 30s. However, this lower bound does eventually slow or stop, and finding a 40 year old attractive by the time you’re 60 doesn’t feel nearly as weird.

So I’d imagine it’s because basically anyone of any age can find a 40+ year old woman attractive without feeling like a creep, whereas most people start aging out of finding 18-20 year olds attractive by the time they’re in their 30s.