Survey bias is a general term for a range of cognitive biases that knowingly or unknowingly influence survey participants to respond inaccurately. Unethical researchers may intentionally construct a survey instrument with survey bias in an effort to skew survey results in one direction or another. Ethical researchers, like you and us, would never do such a thing. We want reliable survey results that we can stand behind, regardless of the outcome.

The following are the four most-common types of survey bias, along with advice for avoiding each one:

#1: Sampling bias: For any research survey, your objective is to gain insights from the people who represent the audience that you’re targeting. In statistical terms, we call this our “sample.” Sampling bias occurs when you over-sample a certain segment of your audience.

How to avoid: Ensure that no demographic or firmographic (i.e., company attribute) is overly represented. At CyberEdge, we ensure that no single industry represents more than 15% of survey respondents, unless the survey is targeted at an individual industry.

#2: Acquiescence bias: People tend to want to say “yes” more often than “no.” They also tend to agree with whatever it is that you’re asking or stating. Experience has shown that when you ask respondents to agree or disagree with a given statement, they’re more likely to agree – especially if the question is worded in a biased way.

How to avoid: Let’s look at two variations of a customer satisfaction question. Option #1 is biased because it presumes that the respondent is satisfied. Option #2 is far more neutral and unbiased. Be sure to never pose questions in a leading way.

Option 1: I am satisfied with the quality of customer support at Acme Corporation.

1. I completely agree
2. I mostly agree
3. I neither agree nor disagree
4. I mostly disagree
5. I completely disagree

Option 2: How satisfied or dissatisfied are you with the quality of customer support at Acme Corporation?

1. Extremely satisfied
2. Somewhat satisfied
3. Neither satisfied nor dissatisfied
4. Somewhat unsatisfied
5. Extremely unsatisfied

#3: Answer option order bias: There are two types of answer option bias: primacy bias and recency bias. Primacy bias is when people choose from the first few options of a multiple-choice question. Recency bias is when people choose the last response of a multiple-choice question because it’s the “most recent” and therefore the most memorable.

How to avoid: Overcoming answer option order bias is simple – randomize your multiple-choice responses. If you provide a “Don’t know” option and/or an “Other” option where the respondent can type in their response, most survey platforms enable you to keep these choices at the bottom while randomizing all other responses.

#4: Social desirability/conformity bias: Sometimes survey respondents are not completely honest for fear of being judged by the survey sponsor or interviewer. Perhaps the survey pertains to lifestyle choices that are bad or unhealthy, such as drinking, smoking, consuming illegal drugs, or practicing unsafe sex.

How to avoid: Another easy fix – ensure that all respondents are fully aware that their responses are completely anonymous, confidential, and only reported in the aggregate. Also, if you have survey questions that you fear could be subject to social desirability/conformity bias, be sure to ask questions in a neutral way.

CyberEdge consistently applies best practices for avoiding both survey bias and survey fatigue in all of its single- and multi-sponsored survey reports. To learn more about CyberEdge’s survey reports, click here: https://cyber-edge.com/services/survey-reports/