It started off as a mathematical hypothesis using Einstein’s Theory of General Relativity. Basically, Einstein showed that mass warps space time. They plugged some ridiculously large masses into the equations and came out with an escape velocity that was greater than the speed of light.
After that, they started looking for things where we could see the gravitational effects of things but couldn’t see what that massive thing was.
For instance, we can see these stars orbiting around the supermassive black hole at the center of our galaxy, Sagittarius A*… but we can’t see SagA* directly: https://youtu.be/VP3uMtP4kIw
By plotting the orbits of those stars, we can get a good idea of the location and mass of whatever is holding them in orbit.
Latest Answers