who would waste good caviar on you? And because taste degrades so steeply with price

I’m not sure caviar is the best example here. A lot of people just don’t like fish eggs, which is fine. But tons of other people eat fish eggs that aren’t caviar, like salmon eggs, which are a common sushi ingredient and relatively cheap. Fish eggs definitely don’t need to be sturgeon caviar to taste good, even if most people don’t like them in general.

But unless you’re in Japan, you’re probably a somewhat rich westerner if you’re eating any kind of fish eggs regularly, so maybe rich people just tend to have more access to things that would expand their palates.