Is there truly an audience for "I don't want any proof, just answer my question"?

More people than I think we'd like to admit. Most people don't spend time verifying whether or not what they've seen is true, they just believe what they see first, especially if it conforms to their existing beliefs.

After all, these models are quite literally plausibility machines. Their entire goal is to generate text that sounds plausibly accurate, because that's how manual content reviewers fine-tune them. Their sole purpose is to generate whatever sounds plausible, not what's necessarily correct, so if there's one thing that will convince the masses that what it says is correct, it will be these "AI" models.