Why does GPU not have expandable RAM like CPU?

Question

Would having an expandable RAM help in running non training workloads like inference for LLMs / Stable Diffusion models?

In: 6

Answer 1

One of the problems not mentioned yet is that graphic cards use more modern chips which are not being sold as RAM modules. So this would require creating these modules which generally only few users would buy.

If you would add normal RAM modules it would mean slowing down the whole VRAM which is a pitfall nobody wants to have. (for more reasons than just slowing)

e.g. 4090 uses GDDR6x while the top memory moduels for sale are DDR5.