I think there may be some confusion about how much energy it takes to respond to a single query or generate boilerplate code. I can run Llama 3 on my computer and it can do those things no problem. My computer would use about 6kWh if I ran it for 24 hours, a person in comparison takes about half of that. If my computer spends 4 hours answering queries and making code then it would take 1kWh, and that would be a whole lot of code and answers. The whole thing about powering a small town is a one-time process when the model is made, so to determine if that it worth it or not it needs to be distributed over everyone who ends up using the model that is produced. The math for that would be a bit trickier.
When compared to the amount of energy it would take to produce a group of people that can do question answering and code writing, I’m very certain that the ai model method is considerably less. Hopefully, we don’t start making our decision about which one to produce based on energy efficiency. We may, though, if the people that choose the fate of the masses sees us like livestock, then we may end up having our numbers reduced in the name of efficiency. When cars were invented, horses didn’t end up all living in paradise. There were just a whole lot less of them around.
d3Xt3r@beehaw.org 3 months ago
That’s going to change in the future with NPUs (neural processing units) being bundled with both regular CPUs (such as the Ryzen 8000 series) and mobile SoCs (such as the Snapdragon 8 Gen 3). The NPU included with the the SD8Gen3 for instance can run models like Llama 2 - something an desktop would normally struggle with. Now this is only the 7B model mind you, so it’s a far cry from more powerful models like the 70B, but this will only improve in the future. Over the next few years, NPUs - and applications that take advantage of them - will be a completely normal thing, and it won’t require a household’s worth of energy. I mean, we’re already seeing various applications of it, eg in smartphone cameras, photo editing app, digital assistants etc. The next would be I guess autocorrect and word prediction, and I for one can’t wait to ditch our current, crappy markov keyboards.