Technically it's a *FAT filesystem thing, and it still carries over into default Windows behaviour today. If I remember correctly, you have to explicitly tell Windows to distinguish between case in filenames, even on NTFS, which otherwise supports it... and then hope that something in the system wasn't relying on case-insensitivity, which is now broken.

I have a few exFAT-formatted USB thumb drives and I find that Linux (an OS that has been case sensitive, or at least, its filesystems have, since the outset) complains that the filename is already in use when renaming files to lower case versions of themselves on those drives. I generally have to rename to something different (maybe add a letter) and then rename back. The case can be changed at either stage, but the intermediate name has to be different.