What you’re talking about is called “frisson.” It can include goosebumps, tightness in your throat, and shivers.
First of all, it doesn’t affect everyone: somewhere between half or one in six never experience frisson. Pity them if you want.
Second, it’s because music stimulates emotions. People who experience frisson tend to experience stronger emotions in general, and especially to have stronger connections in their brain between hearing and emotions.
Thirdly, strong emotions, especially fear, anxiety, love, desire, inspiration, and nostalgia which can come from music, tells your body something strange is going on and makes you more alert and excited (not always in a positive way if it’s a horror soundtrack). Goosebumps are one of many things your body does when it becomes more actively alert to danger or desire.
Latest Answers