You’re almost there. It has to do with the angle of the sun and the water drops relating to the view point. Rainbows only show when the sun is behind you, and if you imagine a cone going out from the viewpoint outwards you get the possible paths of the rainbow (different radius different wavelength and therefore color)

A similar concept happens in certain reflective surfaces (metal pots and pans, car hoods and much more). You always see the tiny scratches in circles, but if you alter the angle in any way you keep seeing different scratch circles. This is because the circle you see in any given angle is the exactly the scratches that are turned just perfectly to reflect the light in the perfect way. It does not mean that the scratches you see at any given moment are the only ones. It means there are plenty, and only a few more visible at a time.

To me, playing around with the second concept (much easier to manipulate yourself and see) made me understand rainbows much better.