How does AI use so much power?

Comments

• ImplyingImplications@lemmy.ca ⁨7⁩ ⁨hours⁩ ago

  It’s mostly the training/machine learning that is power hungry.

  AI is essentially a giant equation that is generated via machine learning. You give it a prompt with an expected answer, it gets run through the equation, and you get an output. That output gets an error score based on how far it is from the expected answer. The variables of the equation are then modified so that the prompt will lead to a better output (one with a lower error).

  The issue is that current AI models have billions of variables and will be trained on billions of prompts. Each variable will be tuned based on each prompt. That’s billions to the power of billions of calculations. It takes a while. AI researchers are of course looking for ways to speed up this process, but so far it’s mostly come down to dividing up these billions of calculations over millions of computers. Powering millions of computers is where the energy costs come from.

  Unless AI models can be trained in a way that doesn’t require running a billion squared calculations, they’re only going to get more power hungry.

  source
• InvisibleShoe@lemmy.world ⁨7⁩ ⁨hours⁩ ago

  My understanding is that traditional AI essentially takes a bruteforce approach to learning and because it is hardwired, its ability to learn and make logical connections is impaired.

  Newer technologies like organic computers using neurons can change and adapt as it learns, forming new pathways for information to travel along, which reduces processing requirements and in turn, reduces power requirements.

  techradar.com/…/a-breakthrough-in-computing-corti…

  corticallabs.com/cl1.html

  source

  • TranquilTurbulence@lemmy.zip ⁨5⁩ ⁨hours⁩ ago

    Machine learning always felt like a very wasteful way to utilize data. Even with ridiculous quantities of it, and the results are still kinda meh. So just dump in even more data, and you get something that can work.

    source
  • snausagesinablanket@lemmy.world ⁨4⁩ ⁨hours⁩ ago

    Lower voltage chip advancement along with better cooling options may come along some day.

    They should consider building their super centers underwater in places like Iceland.

    source

    • InvisibleShoe@lemmy.world ⁨4⁩ ⁨hours⁩ ago

      Thats only a short term solution, global warming will negate those benefits.

      Southern ocean currents just reversed and will likely cause rapid warming of water temps.

      Southern Ocean circulation reversed

      Southern Ocean current reverses for first time, signalling risk of climate system collapse

      France and Switzerland just had to shutdown their nuclear reactors due to the water sources they use for cooling being too warm.

      France and Switzerland shut down nuclear power plants amid scorching heatwave

      When heat halts power: Europe’s nuclear dilemma

      source

      • -> View More Comments
• adespoton@lemmy.ca ⁨8⁩ ⁨hours⁩ ago

  Supercomputers once required large power plants to operate, and now we carry around computing devices in out pockets that are more powerful than those supercomputers.

  There’s plenty of room to further shrink the computers, simplify the training sets, formalize and optimize the training algorithms, and add optimized layers to the AI compute systems and the I/O systems.

  But at the end of the day, you can either simplify or throw lots of energy at a system when training.

  Just look at how much time and energy goes into training a child… and it’s using a training system that’s been optimized over hundreds of thousands of years (and is still being tweaked).

  AI as we see it today (as far as generative AI goes) is much simpler, just setting up and executing probability sieves with a fancy instruction parser to feed it its inputs. But it is using hardware that’s barely optimized at all for the task, and the task is far from the least optimal way to process data to determine an output.

  source

  • null_dot@lemmy.dbzer0.com ⁨38⁩ ⁨minutes⁩ ago

    Supercomputers once required large power plants to operate, and now we carry around computing devices in out pockets that are more powerful than those supercomputers.

    This is false. Supercomputers never required large [dedicated] power plants to operate.

    Yes they used a lot of power, yes that has reduced significantly, but it’s not at the same magnitude as AI

    source
  • BussyCat@lemmy.world ⁨7⁩ ⁨hours⁩ ago

    It is also a very large data set it has to go through the average English speaker knows 40kish words and it has to pull from a large data set and attempt to predict what’s the most likely word to come next and do that a hundred or so times per response. Then most people want the result in a very short period of time and with very high accuracy (smaller tolerances on the convergence and divergence criteria) so sure there is some hardware optimization that can be done but it will always be at least somewhat taxing.

    source
  • semisimian@startrek.website ⁨7⁩ ⁨hours⁩ ago

    This is an astute answer. Bravo.

    source
• calamityjanitor@lemmy.world ⁨6⁩ ⁨hours⁩ ago

  OpenAI noticed that Generative Pre-trained Transformers get better when you make them bigger. GPT-1 had 120 million parameters. GPT-2 bumped it up to 1.5 billion. GPT-3 grew to 175 billion. Now we have models with over 300 billion.

  To run, every generated word requires doing math with every parameter, which nowadays is a massive amount of work, running on the most power hungry top of the line chips.

  There are efforts to make smaller models that are still effective, but we are still in the range of 7-30 billion to get anything useful out of them.

  source
• vrighter@discuss.tchncs.de ⁨6⁩ ⁨hours⁩ ago

  imagine that to type one letter, you need to manually read all unicode code points several thousand times. When you’re done, you select one letter to type.

  Then you start rereading all unicode code points again for thousands of times again, for the next letter.

  That’s how llms work. When they say 175 billion parameters, it means at least that many calculations per token it generates

  source