LLMs are actually spectacular for indexing large amounts of text data and pulling out the answer to a query. Combine that with natural language processing and it is literally what we all thought Ask Jeeves was back in the day. If you ever spent time sifting through stack overflow pages or parsing discussion threads, that is what it is good at. And many models actually provide ways to get a readout of the “thought process” and links to pages that support the answer which drastically reduces the impact of hallucinations.

And many of those don’t necessarily require significant power usage… relative to what is already running in data centers.

The problem is that people use it and decide it is “like magic” and then insist on using it for EVERYTHING. And you go from “Write me a simple function to interface with this specific API” to “Write me an application to do my taxes”

Of course, there is also the issue of where training data comes from. Which is why so much of the “generative AI” stuff is so disgusting because it is just stealing copyrighted data left and right. Rather than the search engine style LLMs that mostly just ignore the proverbial README_FBI.txt file.

And the “this is magic” is on both sides. The evangelists are demonstrably morons. But the rabid anti-AI/“AI” crowd are just as bad with “it gave you a wrong answer, it is worthless”. Think of it less like a magic box and more like asking a question on a message board. You are gonna get a LOT of FUD and it is on you to do additional searches to corroborate when it actually matters.