ChatGPT just announced they need an absurd amount of video cards. Why are video cards used and not dedicated CPUs? Why not make a motherboard that has the right processors?
In: 2
Video cards are designed for graphics processing and come equipped with specialized processors that are designed highly optimized for 3D graphics, which would be necessary if the task is related to graphics processing. Dedicated CPUs are not as efficient as the specialized processors found in video cards. Similarly, a motherboard with the right processors, while theoretically possible, is not nearly as cost efficient as using video cards.
Video cards are designed for graphics processing and come equipped with specialized processors that are designed highly optimized for 3D graphics, which would be necessary if the task is related to graphics processing. Dedicated CPUs are not as efficient as the specialized processors found in video cards. Similarly, a motherboard with the right processors, while theoretically possible, is not nearly as cost efficient as using video cards.
GPUs are specifically designed to perform huge numbers of calculations extremely fast. The trade-off is that they are specifically designed such that they preform the same operation (eg A+B*C) with differing input values over and over*. They are also generally optimized specifically towards floating point operations at the cost of making integer operations slower. GPUs also generally have dedicated memory that is specifically designed to be extremely fast.
CPUs, on the other hand, are optimized towards performing differing operations constantly, and generally also towards integer operations. This makes them ideal for performing business logic that often follows the flow of “if X do Y else do Z”, which a GPU is pretty bad at.
While GPUs were designed for performing graphics operations, hence the `G`, the same pattern has proven extremely useful for many other types of calculations, such physics, encryption, and artificial intelligence. While you could make a dedicated processor just for artificial intelligence, physics, or encryption, work, and it has been done, the benefit is not especially high compared to using a GPU for the same work. Additionally GPUs are much easier to find and are a very well known factor for software to develop against, making it cheaper and easier to write the software you need.
Due to all of this, you can actually find specialized motherboards with lots of PCIe slots for lots of graphics cards. When performing specialized types of operations, this is generally the best way to go: software can still mostly pretend its running on readily available hardware, making it easy to program against, while being able to take advantage of insane computing power.
* When rendering computer graphics, every pixel on your screen that has the same object on it needs to perform the same set of calculations with slightly different input values. There are other layers involved as well with similar patterns, but this simple example should make it fairly clear why this is useful.
GPUs are specifically designed to perform huge numbers of calculations extremely fast. The trade-off is that they are specifically designed such that they preform the same operation (eg A+B*C) with differing input values over and over*. They are also generally optimized specifically towards floating point operations at the cost of making integer operations slower. GPUs also generally have dedicated memory that is specifically designed to be extremely fast.
CPUs, on the other hand, are optimized towards performing differing operations constantly, and generally also towards integer operations. This makes them ideal for performing business logic that often follows the flow of “if X do Y else do Z”, which a GPU is pretty bad at.
While GPUs were designed for performing graphics operations, hence the `G`, the same pattern has proven extremely useful for many other types of calculations, such physics, encryption, and artificial intelligence. While you could make a dedicated processor just for artificial intelligence, physics, or encryption, work, and it has been done, the benefit is not especially high compared to using a GPU for the same work. Additionally GPUs are much easier to find and are a very well known factor for software to develop against, making it cheaper and easier to write the software you need.
Due to all of this, you can actually find specialized motherboards with lots of PCIe slots for lots of graphics cards. When performing specialized types of operations, this is generally the best way to go: software can still mostly pretend its running on readily available hardware, making it easy to program against, while being able to take advantage of insane computing power.
* When rendering computer graphics, every pixel on your screen that has the same object on it needs to perform the same set of calculations with slightly different input values. There are other layers involved as well with similar patterns, but this simple example should make it fairly clear why this is useful.
> Why not make a motherboard that has the right processors?
I don’t think any of the other comments have answered this bit.
We _can_ make very specific processors for machine learning that are much faster than GPUs. Potentially orders of magnitudes faster. These are called ASICs (Application Specific Integrated Circuits).
However, there are big downsides to this:
* People: Doing this requires a big team of experience engineers.
* Time: Making these products from scratch will take at least a year or two for the first generation.
* Cost: Manufacturing custom silicon is _ridiculously_ expensive, especially at a smaller scale.
* Flexibility: The faster a computer chip is at doing one task, the slower it will be at anything other than that task (if it can do it at all). This means you can’t improve an algorithm once it is made into hardware, you need a whole new chip.
All these factors mean that doing custom silicon just isn’t viable for most people*. A GPU offers a lot of performance, but is also very cheap for what it is. So it’s the best option in many cases.
*There are exceptions, it’s an increasing trend for finance companies to produce small batches of custom chips for their high frequency trading. They’re in a very unique situation though.
> Why not make a motherboard that has the right processors?
I don’t think any of the other comments have answered this bit.
We _can_ make very specific processors for machine learning that are much faster than GPUs. Potentially orders of magnitudes faster. These are called ASICs (Application Specific Integrated Circuits).
However, there are big downsides to this:
* People: Doing this requires a big team of experience engineers.
* Time: Making these products from scratch will take at least a year or two for the first generation.
* Cost: Manufacturing custom silicon is _ridiculously_ expensive, especially at a smaller scale.
* Flexibility: The faster a computer chip is at doing one task, the slower it will be at anything other than that task (if it can do it at all). This means you can’t improve an algorithm once it is made into hardware, you need a whole new chip.
All these factors mean that doing custom silicon just isn’t viable for most people*. A GPU offers a lot of performance, but is also very cheap for what it is. So it’s the best option in many cases.
*There are exceptions, it’s an increasing trend for finance companies to produce small batches of custom chips for their high frequency trading. They’re in a very unique situation though.
> Why not make a motherboard that has the right processors?
I don’t think any of the other comments have answered this bit.
We _can_ make very specific processors for machine learning that are much faster than GPUs. Potentially orders of magnitudes faster. These are called ASICs (Application Specific Integrated Circuits).
However, there are big downsides to this:
* People: Doing this requires a big team of experience engineers.
* Time: Making these products from scratch will take at least a year or two for the first generation.
* Cost: Manufacturing custom silicon is _ridiculously_ expensive, especially at a smaller scale.
* Flexibility: The faster a computer chip is at doing one task, the slower it will be at anything other than that task (if it can do it at all). This means you can’t improve an algorithm once it is made into hardware, you need a whole new chip.
All these factors mean that doing custom silicon just isn’t viable for most people*. A GPU offers a lot of performance, but is also very cheap for what it is. So it’s the best option in many cases.
*There are exceptions, it’s an increasing trend for finance companies to produce small batches of custom chips for their high frequency trading. They’re in a very unique situation though.
Latest Answers