Sounds legal overall for America. “Brandishing” is generally illegal, i.e pulling out your gun and pointing it at or actively threatening someone with it, regardless if you fire it.

Pulling out your legal gun in your home and setting it flat on your table? Hard to argue it was a threat without a verbal threat or some other action. It may have been one, but id be suprised if anyone would arrest someone over that.

OP pulling his own gun does make this a hilarious and potentially deadly situation.