ChatGPT just announced they need an absurd amount of video cards. Why are video cards used and not dedicated CPUs? Why not make a motherboard that has the right processors?
In: 2
Compared to CPUs, GPUs are much better at doing a lot of calculations in parallel. CPUs are designed to have the full range of instructions and calculations that a computer might need, and execute them all quickly in order. GPUs, by contrast, are designed primarily for rendering 3D models, which doesn’t require a ton of different types of calculations, but it does require a lot of calculations to be done all at the same time. So GPUs are designed for that kind of parallelization: not the best individual core speed and flexibility, but a lot of cores that can all work at the same time.
AI “deep learning” is basically repeating the same computations for the AI model over and over again, so it makes sense that parallelization would have a lot of benefits there. They could in theory of course design bespoke hardware that did this even better, but it’s far more likely that strapping a couple thousand GPUs to a couple hundred CPUs is far more cost effective.
Latest Answers