I’d give it to alcohol, not caffeine personally. I wouldn’t say most people “abuse” caffeine, they just drink it.
Abuse to me implies having a negative impact, and I can think of more people who have been negatively impacted by weed than by caffeine, but way more from alcohol than either, and with a significantly more negative impact.

I know people who smoke too much and it’s definitely made them stagnate in life and gain a lot of weight.
I know people who drink way too much caffeine and get insomnia, leading to a cycle of discomfort and heartburn from all the coffee.
I know people who drank too much alcohol and died, or developed terrible health complications.

Most people are totally fine with all of them, but alcohol is easily the worst and most common.