Running LLMs do take a lot of power. Trying to run a very small model on my old laptop makes the fans go crazy and power draw must be high given that it gets very hot (granted, the cooling in it isn’t all that great, but power draw is high). Then when you get models with billions upon billions of parameters, you can see how it all adds up. Then you multiply these humongous models with millions of users, as well as consider all the energy required to cool all those servers (fans, pumps, etc).

The biggest power draw in the AI industry is probably training the models, given that it’s quite literally going through the entire internet for every crumb of data and mixing it all together into coherent text and images. Then you see how power grids can be taken down by these power hungry data centers.