Marketing Research is Simple…
Asking the “Right” People
In my last article, as part of my series on conducting “good” Marketing Research, I discussed the importance of asking the “Right” Questions, and some of the perils of not doing so.
In this article, I’m going to delve more deeply into the importance of choosing the “Right” people to include in your survey.
Since it is usually impossible (or impractical) to survey ALL of the people in a population, the critical issue in selecting the “Right” people is the sample design. When designing a sample for research, it is critical to watch for and avoid 5 types of errors.
- Sampling Error – Sampling error is unavoidable. Whenever you take a sample of observations from a population to estimate that population, you will grapple with sampling error. The sample is never exactly the same as the entire population. However, the good news is that statistical theory provides a method to estimate and minimize the degree of sampling error.
Sampling error is affected primarily by the size of the sample drawn from the population. The larger the sample, the lower the sampling error. Designing an effective sample for a study is balancing the size of the sample with the budget you have for the study.
For example, a random sample of 400 from a large population will yield an estimated sampling error of +/- 4.8% at the 5% level of confidence. This means that you can be 95% sure that the data you generate from your survey will be within +/-4.8% of the population parameter. Of course, a sample of 500 would reduce the sampling error to +/-4.3%. Conversely, a sample of 300 would be less costly but would increase the sampling error to +/-5.6%. Considering sample error only, the sample size decision is based on your budget and your tolerance for error. Does a 0.5% reduction in sampling error justify the additional cost for a sample of 500? Can you accept a 0.8% increase in sampling error with a less costly sample of 300?
You also need to account for any sub-groups that you wish to analyze in your sample. A total sample size of 400 may be adequate for total sample analysis, but if you want to compare results by particular market segments, you may need to increase the total sample size to provide enough observations by segment.
The other four errors are referred to as Non-Sampling Errors. Unfortunately, these errors cannot be measured statistically, but they can be mitigated through careful sample design and selection.
- Population Specification Error occurs when the population from which the sample is to be drawn does not match the objectives of the study. For example, I once managed a project to identify the key factors driving the purchase decision of a type of industrial equipment. We interviewed a sample of Purchasing Managers from our target industries who were thought to be the decision-makers for this category. But when we completed the study, the results were inconclusive. Price emerged as the only attribute that was perceived to be important. Further investigation revealed that while the final purchase decision was indeed made by a Purchasing Manager, it was the Plant Engineer who determined the specifications and vendors. The Purchasing Managers only negotiated prices and contract terms and executed the transaction. Repeating the study among Plant Engineers, the more relevant population, identified the key technical specifications that were driving the purchase decision.
- Sample Frame Error is similar to Population Specification Error. However, instead of choosing the wrong population, you choose the wrong subgroup or groups from within the population. This error is commonly encountered when a survey is conducted without any quota controls. For example, very often women are more willing to answer consumer surveys than men. Without any controls, your data may be improperly skewed toward women. Setting a minimum quota for men in you sample plan can limit this error. Likewise, if a key constituency of your research is the Latino segment and your survey is programmed only in English, you will likely under-represent the Latino segment.
- Self-Selection Error occurs because you can’t force people to answer your surveys; people have the option to respond or not. The results may become biased if those who do select to respond differ substantively from those who do not. This happens a lot in customer satisfaction surveys. People who tend to be dissatisfied are more likely to respond to such a survey to voice their complaints about poor service, introducing a negative bias to your results.
- Non-Response Error occurs when there is a practical difference between people who respond and those who fail to respond to your survey. For example, if you are conducting a political poll and the members of one party generally refuse to participate in the survey, your results will be skewed to the opinions of only one party.
Self-selection and Non-response errors are extremely common in almost every type of marketing research. You can’t measure these errors and therefore don’t know the impact on the data you collect. There are ways that you can reduce the impact of these errors by encouraging a higher, more random participation rate by:
-
- Offering incentives (cash, coupons, prize drawings, information) for completed interviews
- Utilization of respondent panels made up of people who opt-in to surveys
- Short, simple, neat, and clean survey design that encourages participation
- A distinct and credible promise of confidentiality and anonymity
- A clear description of the purpose of the survey and assurance that it is not a sales pitch
- Follow-up with reminder invitations to non-responders
In summary, to interview the “Right” people:
-
- Keep focused on the objectives of the research!
- Make sure that you clearly know the identity of your target respondent
- Optimize the size of your sample within your budget to minimize sampling error
- Clearly identify the Population and Subgroups that define your target respondent
- Mitigate self-selection and non-response biases by providing incentives, using opt-in panels, good survey planning, and survey design, assuring confidentiality and anonymity, clearly describing your purpose, and sending reminders
Look for the final installment in this series, Asking at the “Right” Time, next week.
For more on this and other Marketing Research topics, follow me on LinkedIn or reach out to me at carl_fusco@yahoo.com if I can help you in any way.
Carl Fusco is an accomplished Marketing Research Consultant who helps businesses more effectively solve problems by applying research techniques and data-based insights.
Thank you for visiting our Blog!
Jim Weber – Managing Partner, ITB Partners
I hope you enjoyed our point of view and would like to receive regular posts directly to your email inbox. Toward this end, put your contact information on my mailing list.
Your feedback helps me continue to publish articles that you want to read. Your input is very important to me so; please leave a comment.