1. They’re traveling in a medium, so they move slower than in space
2. Due to the random walk caused by multiple scattering, it can take millions of years for a photon to escape the sun after bring produced in the core.

You are right that they don’t gather energy, but they do multiply. What would be a single high energy x ray in the core will eventually downscatter into an army of optical photons.