What codec are you downloading? My internet is a joke, its slow and vids are always compressed in real-time to adapt to the connection speed. Downloading the video (audio) gives you the option to get the original or uncompressed audio stream, you can set yt-dlp to download at the best available quality, so that could be the reason.