The speakers are shaking with a greater amplitude, meaning going in and out thurther. This displaces more air creating a larger sound wave. When this wave passes through the air, it vibrates your ear drum greater so you perceive it louder.
You are exactly right that you need more power to drive the speaker to move bigger distances.
Latest Answers