What about GPU Architecture makes them superior for training neural networks over CPUs?

Question

In ML/AI, GPUs are used to train neural networks of various sizes. They are vastly superior to training on CPUs. Why is this?

In: 679

Answer 1

Each CPU core tends to have 1 floating point unit, maybe a very small number of arithmetic units, etc. While each CPU core has many operating modes, lots of features, the amount of calculation it can do is more limited as a result. A lot of the CPU’s actual circuitry is dedicated to things other than actual computation, like instruction processing and event ordering.

A GPU’s equivalent of a CPU core has dozens, maybe hundreds, of floating point units available to it. Basically a single instruction can order all floating point units it controls to simultaneously perform the operation `x += y` or such. However each such core is more limited, and anything that can’t make good use of that bulk of FPUs will seriously hurt performance. Furthermore it has generally fewer features available.

GPUs tend to do best when the job involves more calculation and less decision making along the process.

Answer 2

Each CPU core tends to have 1 floating point unit, maybe a very small number of arithmetic units, etc. While each CPU core has many operating modes, lots of features, the amount of calculation it can do is more limited as a result. A lot of the CPU’s actual circuitry is dedicated to things other than actual computation, like instruction processing and event ordering.

A GPU’s equivalent of a CPU core has dozens, maybe hundreds, of floating point units available to it. Basically a single instruction can order all floating point units it controls to simultaneously perform the operation `x += y` or such. However each such core is more limited, and anything that can’t make good use of that bulk of FPUs will seriously hurt performance. Furthermore it has generally fewer features available.

GPUs tend to do best when the job involves more calculation and less decision making along the process.