What about GPU Architecture makes them superior for training neural networks over CPUs?

Question

In ML/AI, GPUs are used to train neural networks of various sizes. They are vastly superior to training on CPUs. Why is this?

In: 679

Answer 1

Imagine CPU as one person who is really good at doing all the math you can throw at it. However they can only do one task at a time. GPU is a whole high school full of kids doing simple math tasks. A CPU might have few cores, each of them a person who can do maths. GPU has thousands of smaller cores that do simpler math tasks.

The math done in machine learning is actually rather simple. It is just simple vector calculations in an matrix. They are just multiplication and division. However the issue is that there is A LOT of it. Just absurd amount of it. ML/AI neural networks are just complex n-dimensional arrays with multiplie layers. Now this is exactly what computer graphics are also. They are just calculating translation of triangles in 2-3D space (2 or 3-dimensional array). Simple calculations; just a lot of them.

So you can imagine AI/ML calculations to just be graphics without graphics. Intead of calculating path of a light being reflect off the armor of a game character, you calculate the path of information within AI model’s “mind”. But as the white light turning red through shader or reflection, you change the path of the information depending on what path has the most desired value, these are done with basic matrix calculations..

Answer 2

Imagine CPU as one person who is really good at doing all the math you can throw at it. However they can only do one task at a time. GPU is a whole high school full of kids doing simple math tasks. A CPU might have few cores, each of them a person who can do maths. GPU has thousands of smaller cores that do simpler math tasks.

The math done in machine learning is actually rather simple. It is just simple vector calculations in an matrix. They are just multiplication and division. However the issue is that there is A LOT of it. Just absurd amount of it. ML/AI neural networks are just complex n-dimensional arrays with multiplie layers. Now this is exactly what computer graphics are also. They are just calculating translation of triangles in 2-3D space (2 or 3-dimensional array). Simple calculations; just a lot of them.

So you can imagine AI/ML calculations to just be graphics without graphics. Intead of calculating path of a light being reflect off the armor of a game character, you calculate the path of information within AI model’s “mind”. But as the white light turning red through shader or reflection, you change the path of the information depending on what path has the most desired value, these are done with basic matrix calculations..