Sure, but it was not like they were doing anything important that needed their attention. It is fine to reply immediately, your conversation partners will thank you for it.