What about GPU Architecture makes them superior for training neural networks over CPUs?

Question

In ML/AI, GPUs are used to train neural networks of various sizes. They are vastly superior to training on CPUs. Why is this?

In: 679

Answer 1

AI & ML build out neural networks and train then on data.

A neural network is like your brain, each cell is connected to other cells, so when you get an input a bunch of cells fire off and the eventually decide if something is a traffic light or not.

The math involved in this is very simple, you blast inputs at the NN, see the result, then if it’s right you increase the strength of the links that fired & if it’s wrong you decrease their strength.

The hard part for AI/ML is that you need to do these simple operations many times (once for every node’s connection to other nodes, every time you show it training data (which itself requires a lot of training data).

Graphics cards do this simple math many times to decide what exact color pixels should be.

CPUs are setup to do more complex processing these days, so instead of having a “dual core, or even 32 core machine of CPUs” with a GPU you’re getting far more parallelism.

Answer 2

AI & ML build out neural networks and train then on data.

A neural network is like your brain, each cell is connected to other cells, so when you get an input a bunch of cells fire off and the eventually decide if something is a traffic light or not.

The math involved in this is very simple, you blast inputs at the NN, see the result, then if it’s right you increase the strength of the links that fired & if it’s wrong you decrease their strength.

The hard part for AI/ML is that you need to do these simple operations many times (once for every node’s connection to other nodes, every time you show it training data (which itself requires a lot of training data).

Graphics cards do this simple math many times to decide what exact color pixels should be.

CPUs are setup to do more complex processing these days, so instead of having a “dual core, or even 32 core machine of CPUs” with a GPU you’re getting far more parallelism.