By: Troy Nottingham
September 23, 2014
What’s The Right Sample Size?
An ever-present question in the world of research. The answer I often give – because it is true – but one most people don’t want to hear is … it depends. However, the answer I love to give is … stop me when I get to the number that feels good. People don’t really like that answer very much either.
It depends. It depends on a host of things. For instance, what are you trying to do with the data? If it is a brand or ad tracking study and you want to demonstrate change over time, the larger the sample size the better. Because the larger the sample size, the smaller the margin of error. And, the smaller the margin of error, the smaller the change can be from one wave to the next and still be a significant difference – meaning real change occurred.
If you are trying to profile your audience and want to run a segmentation analysis, then the minimum sample size you want to consider is 1,000. Why 1,000? Good question. For the segmentation to be predictive, the statistics you run require at least 1,000 respondents. There is no need to run a segmentation if it can’t accurately predict each segment of the market – how big each segment is, how to message to each segment and how best to reach them. AKA…segmentation needs to be predictive and being predictive requires a lot of interviews.
If you’re running a nationwide poll that you want to submit to USA Today for publishing, then it seems the accepted sample size is somewhere between 200 – 300 interviews. Not sure why really… maybe because the first study ever published in USA Today was somewhere between 200 – 300 interviews.
On the low end, we advise our clients not to go below 100 for any cell or group of people they want to survey. The margin of error can get pretty high on a sample size of 100. For instance, if 50% of America says they are Libertarians – the margin of error on that finding means the real answer could be anywhere between 40.2% and 59.8%. And we recommend extreme caution when trying to interpret findings on subsets of surveys that are less than 50 people.
The rule of thumb when it comes to the margin of error is that you have to quadruple the sample size to cut the market of error in half. That’s a lot (of money) for a little (drop in margin of error) in some cases – especially once you get north of 200 – 300 interviews.
So, you can see my dilemma when someone is looking for a quick answer to what seems like a very simple question. Many clients glaze over long before I get through all you just read. And most respond with the following: so, what’s the right sample size?