What is response bias?

Bias is a byproduct of human nature—it's everywhere. Our brains sort through 11 million bits of information every second, so we use bias as a cognitive shortcut to make super fast decisions about our next move.

Although entirely natural, these gut reactions are a pounding headache for market research teams. Accurate survey data is key to understanding your audience and making sound business decisions.

But response bias can throw a real spanner in the works by shaping those results. What if some of your yays should have been nays? Or customers you needed to hear from didn't even bother to fill out your form?

To maximise the success of your surveys, you need to be aware of the different response biases, how they affect your data and know how to avoid them, so you have accurate numbers to work with.

What is response bias?

Response bias is a term that describes peoples’ tendency to answer questions inaccurately. Sometimes they do this deliberately, perhaps because they want to give an answer that makes them look better.

For example, if your dentist has ever asked how much sugar you eat every day, you'll understand how easily white lies can show up in your response.

How about when you're asked how much alcohol you consume in a week, and you've thought to yourself, "hmm, I'd better round that down?"

But often, response bias is unintentional and could come down to how the question is phrased. Heck, the respondent could just be having a bad day. Either way, the result is a data set filled with holes.

Response bias typically involves two groups of people:

• Researchers: bias creeps into survey questions depending on how you frame them. This happens when you lead the respondent towards response options they wouldn't necessarily have chosen.
• Respondents: people select an incorrect response to a question, either deliberately or accidentally. Perhaps they don't have time for your survey and don't consider their answers carefully. Or maybe they want to conceal their true opinion from you.

What is non-response bias?

Non-response bias happens when people who should have been included in your survey sample don't participate.

This is super frustrating and just as detrimental to your market research as response bias. But why does this happen?

• Poor communication: Did you forget to invite someone?
• Lack of interest: Your recipients aren't jazzed about completing your survey. Perhaps they aren't comfortable with the questions, think your survey is clunky and unattractive, or don't believe their response would make a difference.
• Delivery problems: Your recipients never received your survey—did you send it by snail mail, or perhaps it ended up in their spam folder?
• Form construction: Was there a technical hitch, and it timed out, or perhaps your recipients were trying to respond on the move, but the survey didn't load properly on their mobile devices?
• Missed deadline: Your recipients were late to the party and returned the survey after the deadline (also known as late response bias). The data collected outside the timeframe might not be helpful to your results.

Why does response bias matter?

Response bias leads to inaccurate data, ultimately impacting your ability to make sound business decisions.

Let's say you want to find out how often people use your gym equipment, but you've worded your survey in a way that leads respondents to give you the answer you want to hear.

You might ask, "Do you use the Treadmill Xtron Buster 5000 daily or weekly?" but your survey respondent hasn't used it in over a month, so to avoid appearing lazy, they'll bend the truth and choose "weekly" rather than admit they don't use it at all.

Voilá—you've got yourself one skewed data set.

This leads researchers towards inaccurate conclusions, which can have far-reaching implications. After all, research is expensive to conduct, so you want to get it right the first time.

Types of response bias

Here are seven different forms of response bias to get your head around, along with tips to avoid each.

1. Demand characteristics

Unless this is their first rodeo, most survey respondents come to the table with preconceived ideas about their role in the survey and the questions you’ll ask them.

These demand characteristics happen when people second-guess the study's purpose, which can subconsciously nudge them into changing their responses.

Psychologists Thomas Cook and Stephen Weber explain that other participants would actively lend a hand and provide the answers they think you want to hear, especially if they believe they’ll receive a reward for positive answers. Not great for your response validity.

How to avoid demand characteristics:

• Avoid communication with the survey participant. If you speak to them face-to-face or personally email the survey to them, they might feel under pressure to answer a certain way.
• Keep the purpose of your study under wraps, so the rumour mill doesn't go into overdrive.
• Avoid using a survey title so your respondents don’t try to guess your goals.
• Get around your respondents' good intentions by using a double-blind study design where neither the researchers nor respondents know who will complete a control survey (a dummy survey) or a treatment survey (the one you're interested in.)

2. Social desirability bias

Social desirability stems from our deep-rooted need to “fit in” and be accepted by others. (Many of us are people pleasers at heart.)

This form of bias crops up when you ask sensitive questions, particularly if one viewpoint is more socially acceptable than another.

There are two main types of social desirability bias.

1. Self-deception enhancement is a type of social desirability bias where people genuinely believe statements they make about themselves, even if they're a little deluded. For example, you might check a box to confirm you care about the environment and are committed to recycling—even though you regularly leave half-eaten takeaway containers on the bus.
2. Impression management happens when people think you’re judging them, and this influences their answers. If you've ever paused before participating in a social media poll when you noticed "the author can see how you vote," you'll understand the pressure to make the right impression with your response.

How to avoid social desirability bias:

• Use an online survey tool, like Paperform, that offers anonymity. Respondents feel more comfortable answering honestly when they know you can't track their answers back to them.
• Keep your questions neutral with non-biased vocabulary, and avoid using judgmental words. Worried about unconscious bias in your surveys? Take part in Harvard’s Implicit Bias Project to learn if this is a weakness in your business.

3. Extreme responses

Extreme responses are common on questions that use Likert scales—those with answers ranging from the most negative to the most positive.

Here's an example of a five-point Likert scale.

Other examples would include choosing a number between 1 and 10 or a star rating scale like when you're rating your Uber driver.

The survey participant must decide where their opinion sits on the scale. If they're feeling extreme, they'll choose either the 'Strongly Disagree' or 'Strongly Agree 'options (or the equivalent), snubbing the neutral ground entirely.

Why? Well, extreme responding can happen if your respondents think your survey is a snoozefest and don't want to take the time to consider if their opinion sits in the middle.

If they're mildly dissatisfied, they'll choose the strongest possible answer to quickly make their point known. Or if they're reasonably happy, they'll elevate their answer to the most positive response as a swift way to move on and get some of their day back.

How to avoid extreme response bias:

• Avoid using emotive language in your questions. If you're trying to gauge if a customer enjoyed their interaction with your sales team, don't ask, "How much did you love your experience?" as this could prompt a more emotional response. Instead, ask, "How would you rate your experience?"
• Consider the demographic variables of your sample group. If you're surveying people who feel passionately about your topic, you're more likely to attract those extreme results. For a more balanced curve, try inviting a broader range of people to take your survey.

4. Neutral responses

Neutral survey response bias happens when you’ve lost the interest of the survey participant. Using the same Likert scale example, you’ll find their answers sit firmly in the middle for every answer.

How do you spot it? If your survey respondent has selected 'Neutral' for every single response, you probably have yourself a fence-sitter.

How to avoid neutral responses:

• Keep your survey questions short and sweet, so your participants aren't turned off.
• Make sure your questions are clear. Break down complex ideas into digestible statements your participants can easily understand and give an opinion about.

5. Acquiescence bias

Acquiescence bias is the nodding dog of survey responses—when someone tends to agree with every statement, even if it's not a true reflection of their opinion.

A study from the American Psychological Association found that it often happens when it's the respondent's best guess on a topic they're not clued up on.

In a Yes/No style question, acquiescence bias sometimes happens because we subconsciously like to be polite. It would look like this:

It also happens in the courtroom when lawyers ask leading questions to witnesses. They might say, "was the defendant wearing a green jacket?" The witness is more likely to confirm this even if they don't know or can't remember.

How to avoid acquiescence bias:

• Ask two contradictory questions. Present one statement, follow up with a flipped version, and see if the answers change. Logically, they should.

6. Dissent bias

If there's a rebel in your survey ranks, it'll show up as dissent bias, which is the opposite of acquiescence bias.

It happens when your respondents choose a negative response to all your questions. Think of it like a toddler who's just learned how to say no, and then says it repeatedly.

It's more common in surveys with close-ended questions, where people can only choose from a set response. The respondent doesn't necessarily believe what they're saying but goes against the grain for the sake of it.

How to avoid dissent bias:

• Avoid close-ended questions and give your respondents the chance to explain their answers. You could even ask them to elaborate on why they disagreed with a particular statement.
• Diversify your questions with different formats and interactive elements to keep the participant engaged and avoid that speed-run of negativity. Try using Paperform’s Guided Mode to play around with the theme and create a truly immersive experience.

7. Voluntary response bias

Market research teams love it when people volunteer to participate in their surveys. Your respondents are motivated to read the questions carefully and provide considered answers.

Sounds great so far, right? But when people self-select to take part, this is often because they have strong opinions about the topic, which may introduce bias into your survey results.

For example, those passionate about your brand are more likely to respond if you're surveying people about your new product launch.

How to avoid voluntary response bias:

• Use a representative sample. This is a group of people that accurately reflects the population you're trying to reach rather than the most vocal participants.

How to minimise and avoid response bias

It’s a big ask for researchers to structure a survey that provides the best chance of honest answers, but it can be done.

Here are some best practices to reduce the odds of response bias from seeping into your surveys.

1. Know your audience

The better you understand your respondents, the easier it will be to identify which type of response bias could affect them.

For example, if you're surveying parents about their kids' school lunches, social desirability bias might come into play.

Once you've identified what biases your audience demographic may slip into, you can craft your survey in a way that addresses those biases up front.

2. Use anonymous responses

Consider making responses anonymous if your survey questions focus on a sensitive topic like reproductive rights or unisex bathrooms—this will encourage people to give honest answers without fear of judgement.

Anonymity also comes in handy for internal surveys, like exit interviews or employee reviews. Employees are much more willing to give feedback that's honest when they aren't worried about getting in trouble (or fired!)

3. Learn how to phrase your questions

Be mindful of your tone, and avoid using loaded language that could influence a person's response.

"Why do you feel that our service is the best in the industry?" makes assumptions and doesn't allow to interpret if that's how your survey participant feels.

Intentionally or not, this type of language forces a viewpoint on your users. Instead, aim for a more neutral approach: "How does our service compare to others in the industry?" is much better.

4. Don't forget about question order bias

People tend to provide answers consistent with their prior responses, so your previous questions unwittingly influence them.

Wondering what this looks like? If you ask your audience what their favourite shade of blue is, then follow up and ask what their favourite color is, they're more likely to say blue.

Avoid this type of "priming" by randomising the order of your questions, so long as it makes sense to do so.

5. Use conditional logic

Survey participants can feel alienated if a question doesn't apply to them. For example, asking someone how long they have been married, assumes a lot about their relationship status.

Single participants will either exit the survey, causing non-response bias, or enter a random number to give you false data.

Get around this by using conditional logic. With logic you can create a personalised survey experience based on your respondents' previous answers.

6. Vary your survey question types

We've seen how tempting it is for some respondents to zip through your questions, answering extremely, or choosing neutral answers to get the job done as soon as possible.

Get around this by mixing up your survey questions. Use a blend of Yes/No, scale and multiple-choice questions to keep your participants engaged and on their toes.

Reduce response bias with Paperform

There you have it—you might not be able to completely eradicate bias from your surveys, but you've now got a loaded toolbox to help reduce the chances of it occurring.

The next step? Start building a survey with Paperform. We have 650+ pre-built templates and a rich range of question types, from multiple-choice and dropdowns to sliding scales and conditional logic questions.

Sure, we're a bit biased, but Paperform has everything you need to create engaging surveys that respondents will enjoy filling out, enabling you to find the next great opportunity for your business.

Create your response bias-free form today with our 14-day free trial—no credit card required.