What about GPU Architecture makes them superior for training neural networks over CPUs?

Question

In ML/AI, GPUs are used to train neural networks of various sizes. They are vastly superior to training on CPUs. Why is this?

In: 679

Answer 1

GPUs are very good at doing the same thing over and over again on a huge pile of data.

Each pixel in an image (and there may be millions) will have an equation relating it to a texture and then a series of vector or matrix calculations to give a final pixel colour. The same equation is used for every pixel in an object, its just that each pixel has slightly different data (different coordinate).

CPUs are very good at switching from one task to another and hopping about doing different things one after another.

Training neural networks is all about doing the same calculation over and over on a ton of data
In particular it’s mainly matrix operations (or tensor operations, but these can be broken down into matrix operations) which is exactly what GPUs are good at.

Answer 2

GPUs are very good at doing the same thing over and over again on a huge pile of data.

Each pixel in an image (and there may be millions) will have an equation relating it to a texture and then a series of vector or matrix calculations to give a final pixel colour. The same equation is used for every pixel in an object, its just that each pixel has slightly different data (different coordinate).

CPUs are very good at switching from one task to another and hopping about doing different things one after another.

Training neural networks is all about doing the same calculation over and over on a ton of data
In particular it’s mainly matrix operations (or tensor operations, but these can be broken down into matrix operations) which is exactly what GPUs are good at.