You would be limited by what we call “The speed of sound” – but that’s because the speed of sound is just the speed at which molecules (basically) vibrate against one another
Your initial movement (or impulse, or vibration) would travel at the speed of sound down the length of the rod, so the other end of your several light-years long rod would take a long time before noticing your initial movement.
Latest Answers