What about GPU Architecture makes them superior for training neural networks over CPUs?

Question

In ML/AI, GPUs are used to train neural networks of various sizes. They are vastly superior to training on CPUs. Why is this?

In: 679

Answer 1

CPUs are generalists and can do a lot of things. Most of the “stuff” in a CPU is not for doing math but is there to perform complex tasks. For example, the reason you can interact with your computer in real time (when you press your mouse button to open a web browser while using a text editor in the background) is because the CPU can pause a task anytime and resume it later when needed.

GPUs cannot do most things that CPUs can, but everything in a GPU is dedicated to perform math operations. Because neural networks need a lot of math, using a GPU is much more efficient than a CPU.

Answer 2

CPUs are generalists and can do a lot of things. Most of the “stuff” in a CPU is not for doing math but is there to perform complex tasks. For example, the reason you can interact with your computer in real time (when you press your mouse button to open a web browser while using a text editor in the background) is because the CPU can pause a task anytime and resume it later when needed.

GPUs cannot do most things that CPUs can, but everything in a GPU is dedicated to perform math operations. Because neural networks need a lot of math, using a GPU is much more efficient than a CPU.