What about GPU Architecture makes them superior for training neural networks over CPUs?

Question

In ML/AI, GPUs are used to train neural networks of various sizes. They are vastly superior to training on CPUs. Why is this?

In: 679

Answer 1

TLDR; A CPU can perform many different tasks but a GPU can perform one task very, very efficiently and that task is computation.

Your CPU is designed to run different kind of tasks. It is general-purpose and flexible: you can play games, listen to music, watch a movie, access websites, all at once. But because of that, it is not the most efficient in doing any of those things.

OTOH GPUs are designed for computational efficiency. (That’s it.) And neural networks are made of repetitve calculations: multiply two numbers and then add another number. You do this for every neuron in the network, for thousands of cycles. For repetitive calculations such as these, GPUs can perform them in parallel on a scale vastly larger than a CPU.

Answer 2

TLDR; A CPU can perform many different tasks but a GPU can perform one task very, very efficiently and that task is computation.

Your CPU is designed to run different kind of tasks. It is general-purpose and flexible: you can play games, listen to music, watch a movie, access websites, all at once. But because of that, it is not the most efficient in doing any of those things.

OTOH GPUs are designed for computational efficiency. (That’s it.) And neural networks are made of repetitve calculations: multiply two numbers and then add another number. You do this for every neuron in the network, for thousands of cycles. For repetitive calculations such as these, GPUs can perform them in parallel on a scale vastly larger than a CPU.