Yes, that’s normally a good rule to follow, since “whom” is for a sentence’s object, but this is a special case. The clause in question is either a salutation that has no subject or object, and so either “whoever” or “whomever” is correct, or it’s a subject clause (a noun phrase, really) with an unnecessary, stylized “to” for the sake of comedic impact, in which case “whoever” would be correct.