You can think of voltage like the water pressure in a pipe, and a fuse like a valve.
When the fuse is intact it allows current to flow, so it’s like an open valve. And if you measure the water pressure right in front of an open valve, and right behind it, it’s basically the exact same pressure, so the difference in pressure, or the potential, will read zero. The same when you measure across an intakt fuse, the difference in electrical potential between the two ends, which is what your multimeter measures, is pretty much zero.
Now you close the valve. On the side with the water you still have that same high pressure, but on the other side there’s no longer any water pressure at all, so now you have a pressure difference.
Same with a blown fuse, on the PSU attached side the voltage is still the same, but on the disconnected side there is no longer any voltage, so there’s now a difference between the two sides.
Latest Answers