What about GPU Architecture makes them superior for training neural networks over CPUs?

Question

In ML/AI, GPUs are used to train neural networks of various sizes. They are vastly superior to training on CPUs. Why is this?

In: 679

Answer 1

I’ve read through all of this but here’s a real simple example from my actual work experience years ago.

I started out on Wang 2200s, which were fast little things that engineering people especially loved to use because they did math fast. The reason was they had specialized chips for matrix arithmetic.

Before these chips, if I had to init an array of 10 X 10 cells, I’d have to loop through and set each one to zero and then get started on what I wanted to do. When the first machine with these chips came in, all I had to do was say “Mat Y = Zer” where Y was the 10 X 10 array I was looking to init. It was instantaneous. It meant I could spit out reports at multiples of the speed I could before.

That’s the difference between a CPU and a GPU for math stuff.

Answer 2

I’ve read through all of this but here’s a real simple example from my actual work experience years ago.

I started out on Wang 2200s, which were fast little things that engineering people especially loved to use because they did math fast. The reason was they had specialized chips for matrix arithmetic.

Before these chips, if I had to init an array of 10 X 10 cells, I’d have to loop through and set each one to zero and then get started on what I wanted to do. When the first machine with these chips came in, all I had to do was say “Mat Y = Zer” where Y was the 10 X 10 array I was looking to init. It was instantaneous. It meant I could spit out reports at multiples of the speed I could before.

That’s the difference between a CPU and a GPU for math stuff.