I’m pretty sure it’s a joke to refer to some form of mysterious, unknowable power conducted by secretive practitioners or something. However, if you’re gonna get upset about it and accuse people of being bigots then you should probably be more specific about which religion you’re referring to and what’s wrong with their statement. There are a number of religions that get called “voodoo” like Louisiana Voodoo, Haitian Vodou, Hoodoo, and Juju. Some of these religions encourage secrecy, others don’t. Some incorporate magic, others don’t. Some use talismans, others feature spiritual possession, and so on.