You can’t just take the average over time of an alternating current, because it keeps changing direction, so over time it will average to zero, and that doesn’t tell you how much voltage there is.
What you can do is square the voltage, so it all becomes positive, then take the average, then square root it again so it’s in units of voltage. This quantity will be representative of the amount of voltage you have, and it’s related to both the peak voltage and the waveform.
If the voltage was constant DC, then the rms would just be the average. Because it alternates, the times it goes through zero will bring the average down, and the rms will come out less than the peak voltage.
For sine waves, which are the easiest waveform to generate and the one you get out of your wall, it’s a relatively simple calculation to show that the rms is the peak divided by the square root of 2. So that’s where those specific numbers come from.