What about GPU Architecture makes them superior for training neural networks over CPUs?

Question

In ML/AI, GPUs are used to train neural networks of various sizes. They are vastly superior to training on CPUs. Why is this?

In: 679

Answer 1

GPUs have dedicated circuitry for graphics math, and now recently they’re being included with circuitry dedicated for AI math. CPUs do this math using general purpose circuitry which makes them slower at it.

In addition, GPUs have higher total computing power than CPUs. But most tasks are very difficult or impossible to program to run on a GPU or fully utilize it because of the design of GPUs compared to CPUs. Other comments have explained those differences.

AI training and execution happens to take advantage of GPUs well.

Answer 2

GPUs have dedicated circuitry for graphics math, and now recently they’re being included with circuitry dedicated for AI math. CPUs do this math using general purpose circuitry which makes them slower at it.

In addition, GPUs have higher total computing power than CPUs. But most tasks are very difficult or impossible to program to run on a GPU or fully utilize it because of the design of GPUs compared to CPUs. Other comments have explained those differences.

AI training and execution happens to take advantage of GPUs well.