“No matter who” means that your vote has no influence. If nothing will dissuade you from voting, nevermind make you switch sides, you don’t influence party politics as all. Your vote is a freebie, the part can just sweeten the deal with anyone else.