I’ve noticed most U.S. polls query around 1,000 people and sometimes even less. Somehow that qualifies for headlines like “Americans say…” or “Most Americans…” How is it acceptable that 0.0002% of the population is accepted as representative?
In: Other
Suppose there is an issue on which people are split 70:30. If you ask a randomly chosen person, with probability 70% they will answer “yes” and with probability 30% they will answer “no”.
If you poll 1000 people, chosen at random and mutually independently, the most likely outcome of the poll is that you get 700x yes and 300x no. So far, this should be pretty obvious. It should also be obvious that usually you’ll get something approximate and not the exact split, such as 684 times yes and 316 times no.
The main question now is: how likely is it that we get something *substantially* different as the outcome of our poll? Can we realistically get, for example, 300x yes and 700x no?
And the answer is that this is *very very* unlikely.
We can do the math and calculate that in our example scenario:
* Already getting the 500-500 split or anything worse is almost impossible. Roughly on par with having a day on which you take part in four separate lotteries, win the jackpot in all of them and then get hit by lightning. (And yes, I did the math here.)
* Almost all polls will give you a result somewhere between 650-350 and 750-250. Maybe once per 1000 such polls will you see something that falls slightly outside these bounds, but almost always you’ll get a very good estimate.
That’s plenty accurate if all we need is a rough estimate.
Latest Answers