Why does GPU not have expandable RAM like CPU?

Question

Would having an expandable RAM help in running non training workloads like inference for LLMs / Stable Diffusion models?

In: 6

Answer 1

https://www.quora.com/Why-dont-GPUs-have-expandable-RAM

Tl;Dr they aren’t built with extra memory lines (wires) to empty slots because this would be overhead on the GPU side, and the signal timing requirements are extremely tight for GPUs so there is no margin for uncertainty due to the slot etc. The VRAM needs to be physically close to the GPU and tightly connected.