It’s a balance between area/power requirements and actual boost in performance.
And in a way that’s what a GPU is except they call them SIMD units not ALUs but it’s the same idea.
The difference between a CPU and a GPU though is what they are meant to run. A CPU is meant to execute programs that are largely serial in nature (e.g. compile a program, etc…) whereas a GPU is meant to execute programs that are highly parallel like compute shader expressions over millions of pixels.
Latest Answers