I recently read that asking a single question in Chat GPT uses energy/electricity equal to powering a lightbulb for something like 20 minutes. How does Chat GPT work and is it an energy drain?
In: Technology
In a simple terms, ChatGPT is generating words by looking at the prompt and words that come before.
To do that is has huge tables of all the words it seen and how likely they are connected, with additional tables to handle context and memory of the text.
So what it does it makes lookups in those tables via some math to check what the next word should be, and then it repeats that for the each word it generates.
The math there is mostly matrix multiplications, so a lot of number multiplications and additions. That is done by the GPU on the server (because GPUs are good at massive math stuff). So powering that GPU requires power.
Lightbulb, on the other hand, is now a quite efficient thing and consumes very little energy.
Latest Answers