I recently read that asking a single question in Chat GPT uses energy/electricity equal to powering a lightbulb for something like 20 minutes. How does Chat GPT work and is it an energy drain?
In: Technology
In a simple terms, ChatGPT is generating words by looking at the prompt and words that come before.
To do that is has huge tables of all the words it seen and how likely they are connected, with additional tables to handle context and memory of the text.
So what it does it makes lookups in those tables via some math to check what the next word should be, and then it repeats that for the each word it generates.
The math there is mostly matrix multiplications, so a lot of number multiplications and additions. That is done by the GPU on the server (because GPUs are good at massive math stuff). So powering that GPU requires power.
Lightbulb, on the other hand, is now a quite efficient thing and consumes very little energy.
ChatGPT uses a lot of energy because it’s powered by big, powerful computers that need to work really hard to answer your questions. Imagine if every time you asked a question, a giant machine had to solve a huge puzzle. This machine needs electricity to keep working, just like how your phone or computer needs a battery.
When you ask a question, it’s kind of like turning on a light bulb for about 20 minutes. The computers use this energy to figure out how to answer your question, and they also need to stay cool because they get hot while working. People are trying to make these machines use less energy so they don’t waste as much and are better for the planet.
“ChatGPT consumes over half a million kilowatts of electricity each day, an amount staggering enough to service about two hundred million requests. ChatGPT’s daily power usage is nearly equal to 180,000 U.S. households, each using about twenty-nine kilowatts.”
One of main concerns around ChatGPT and similar LLM models becoming ubiquitous is that there is not enough electricity in the world to run those massive GPU farms.
I had attended one of the trainings provided in my company some months ago regarding this. IIRC it consumes 10-15 times the energy of a traditional CPU as it primarily uses GPU for computation and approx 500ml of fresh water for 20-50 simple questions for keeping the system cool.
Latest Answers