If I’m honest, I think your dad’s logic is sound:
Whether or not your brother has COVID is immaterial to the fact that he has an illness that is likely contagious and is still planning to visit your compromised aunt.
If she’s ill enough that any illness could tip the scales for her, then it doesn’t matter which illness. The real question is why the hell is your brother considering visiting at all.

Not that you’re wrong for asking that, it’s just that your question is superseded by a question that it seems your dad forgot to think about while contradicting you.