The reason you need to slow down is because you’re starting on Earth, which means you’re moving fast enough parallel to the sun’s surface that for every foot you fall downwards toward the sun, the sun’s surface curves away by 1 foot. This results in the nearly circular orbit around the sun we exist in.

If you start speeding up, the orbit becomes more elliptical, except your aphelion starts raising away from the sun because now you’re moving fast enough that you’ve moved more than 1 foot sideways in the time you’ve fell 1 foot downwards.

Slowing down has the opposite effect. If you get your speed down to 0, you’ll fall straight down toward the sun as normal with gravity. But you don’t need to go all the way down to 0 velocity to enter the sun, you just need to slow down until your elliptical orbits brushes up against the sun’s surface.