It is going to be harder having a discussion with people who have a significantly different body of knowledge than you because it takes more effort in communicating. For some, it means explaining a lot more than you would otherwise need to. For others, you aren’t going to be able to explain it enough.

One thing that I’d ask is what they are talking about. I don’t think it is exactly fair to pull laypeople into the topics that you are educated in and expect an enriching conversation.