You should have bit your lip and said the appropriate goodbyes, but leaving early was probably the best choice.

It was your day, not theirs. It was completely inappropriate for what happened. But it happened all the same, so using decorum to your advantage would have allowed you to escape without having to explain yourself as much as you had to.