Explore all the solutions you can create with Paperform: surveys, quizzes, tests, payment forms, scheduling forms, and a whole lot more.See all solutions
Connect with over 2,000 popular apps and software to improve productivity and automate workflowsSee all integrations
Bias is a byproduct of human nature—it's everywhere. Our brains sort through 11 million bits of information every second, so we use bias as a cognitive shortcut to make super fast decisions about our next move.
Although entirely natural, these gut reactions are a pounding headache for market research teams. Accurate survey data is key to understanding your audience and making sound business decisions.
But response bias can throw a real spanner in the works by shaping those results. What if some of your yays should have been nays? Or customers you needed to hear from didn't even bother to fill out your form?
To maximise the success of your surveys, you need to be aware of the different response biases, how they affect your data and know how to avoid them, so you have accurate numbers to work with.
Response bias is a term that describes peoples’ tendency to answer questions inaccurately. Sometimes they do this deliberately, perhaps because they want to give an answer that makes them look better.
For example, if your dentist has ever asked how much sugar you eat every day, you'll understand how easily white lies can show up in your response.
How about when you're asked how much alcohol you consume in a week, and you've thought to yourself, "hmm, I'd better round that down?"
But often, response bias is unintentional and could come down to how the question is phrased. Heck, the respondent could just be having a bad day. Either way, the result is a data set filled with holes.
Response bias typically involves two groups of people:
Non-response bias happens when people who should have been included in your survey sample don't participate.
This is super frustrating and just as detrimental to your market research as response bias. But why does this happen?
Response bias leads to inaccurate data, ultimately impacting your ability to make sound business decisions.
Let's say you want to find out how often people use your gym equipment, but you've worded your survey in a way that leads respondents to give you the answer you want to hear.
You might ask, "Do you use the Treadmill Xtron Buster 5000 daily or weekly?" but your survey respondent hasn't used it in over a month, so to avoid appearing lazy, they'll bend the truth and choose "weekly" rather than admit they don't use it at all.
Voilá—you've got yourself one skewed data set.
This leads researchers towards inaccurate conclusions, which can have far-reaching implications. After all, research is expensive to conduct, so you want to get it right the first time.
Here are seven different forms of response bias to get your head around, along with tips to avoid each.
Unless this is their first rodeo, most survey respondents come to the table with preconceived ideas about their role in the survey and the questions you’ll ask them.
These demand characteristics happen when people second-guess the study's purpose, which can subconsciously nudge them into changing their responses.
Psychologists Thomas Cook and Stephen Weber explain that other participants would actively lend a hand and provide the answers they think you want to hear, especially if they believe they’ll receive a reward for positive answers. Not great for your response validity.
How to avoid demand characteristics:
Social desirability stems from our deep-rooted need to “fit in” and be accepted by others. (Many of us are people pleasers at heart.)
This form of bias crops up when you ask sensitive questions, particularly if one viewpoint is more socially acceptable than another.
There are two main types of social desirability bias.
How to avoid social desirability bias:
Extreme responses are common on questions that use Likert scales—those with answers ranging from the most negative to the most positive.
Here's an example of a five-point Likert scale.
Other examples would include choosing a number between 1 and 10 or a star rating scale like when you're rating your Uber driver.
The survey participant must decide where their opinion sits on the scale. If they're feeling extreme, they'll choose either the 'Strongly Disagree' or 'Strongly Agree 'options (or the equivalent), snubbing the neutral ground entirely.
Why? Well, extreme responding can happen if your respondents think your survey is a snoozefest and don't want to take the time to consider if their opinion sits in the middle.
If they're mildly dissatisfied, they'll choose the strongest possible answer to quickly make their point known. Or if they're reasonably happy, they'll elevate their answer to the most positive response as a swift way to move on and get some of their day back.
How to avoid extreme response bias:
Neutral survey response bias happens when you’ve lost the interest of the survey participant. Using the same Likert scale example, you’ll find their answers sit firmly in the middle for every answer.
How do you spot it? If your survey respondent has selected 'Neutral' for every single response, you probably have yourself a fence-sitter.
How to avoid neutral responses:
Acquiescence bias is the nodding dog of survey responses—when someone tends to agree with every statement, even if it's not a true reflection of their opinion.
A study from the American Psychological Association found that it often happens when it's the respondent's best guess on a topic they're not clued up on.
In a Yes/No style question, acquiescence bias sometimes happens because we subconsciously like to be polite. It would look like this:
It also happens in the courtroom when lawyers ask leading questions to witnesses. They might say, "was the defendant wearing a green jacket?" The witness is more likely to confirm this even if they don't know or can't remember.
How to avoid acquiescence bias:
If there's a rebel in your survey ranks, it'll show up as dissent bias, which is the opposite of acquiescence bias.
It happens when your respondents choose a negative response to all your questions. Think of it like a toddler who's just learned how to say no, and then says it repeatedly.
It's more common in surveys with close-ended questions, where people can only choose from a set response. The respondent doesn't necessarily believe what they're saying but goes against the grain for the sake of it.
How to avoid dissent bias:
Market research teams love it when people volunteer to participate in their surveys. Your respondents are motivated to read the questions carefully and provide considered answers.
Sounds great so far, right? But when people self-select to take part, this is often because they have strong opinions about the topic, which may introduce bias into your survey results.
For example, those passionate about your brand are more likely to respond if you're surveying people about your new product launch.
How to avoid voluntary response bias:
It’s a big ask for researchers to structure a survey that provides the best chance of honest answers, but it can be done.
Here are some best practices to reduce the odds of response bias from seeping into your surveys.
The better you understand your respondents, the easier it will be to identify which type of response bias could affect them.
For example, if you're surveying parents about their kids' school lunches, social desirability bias might come into play.
Once you've identified what biases your audience demographic may slip into, you can craft your survey in a way that addresses those biases up front.
Consider making responses anonymous if your survey questions focus on a sensitive topic like reproductive rights or unisex bathrooms—this will encourage people to give honest answers without fear of judgement.
Anonymity also comes in handy for internal surveys, like exit interviews or employee reviews. Employees are much more willing to give feedback that's honest when they aren't worried about getting in trouble (or fired!)
Be mindful of your tone, and avoid using loaded language that could influence a person's response.
"Why do you feel that our service is the best in the industry?" makes assumptions and doesn't allow to interpret if that's how your survey participant feels.
Intentionally or not, this type of language forces a viewpoint on your users. Instead, aim for a more neutral approach: "How does our service compare to others in the industry?" is much better.
People tend to provide answers consistent with their prior responses, so your previous questions unwittingly influence them.
Wondering what this looks like? If you ask your audience what their favourite shade of blue is, then follow up and ask what their favourite color is, they're more likely to say blue.
Avoid this type of "priming" by randomising the order of your questions, so long as it makes sense to do so.
Survey participants can feel alienated if a question doesn't apply to them. For example, asking someone how long they have been married, assumes a lot about their relationship status.
Single participants will either exit the survey, causing non-response bias, or enter a random number to give you false data.
Get around this by using conditional logic. With logic you can create a personalised survey experience based on your respondents' previous answers.
We've seen how tempting it is for some respondents to zip through your questions, answering extremely, or choosing neutral answers to get the job done as soon as possible.
Get around this by mixing up your survey questions. Use a blend of Yes/No, scale and multiple-choice questions to keep your participants engaged and on their toes.
There you have it—you might not be able to completely eradicate bias from your surveys, but you've now got a loaded toolbox to help reduce the chances of it occurring.
The next step? Start building a survey with Paperform. We have 650+ pre-built templates and a rich range of question types, from multiple-choice and dropdowns to sliding scales and conditional logic questions.
Sure, we're a bit biased, but Paperform has everything you need to create engaging surveys that respondents will enjoy filling out, enabling you to find the next great opportunity for your business.
Create your response bias-free form today with our 14-day free trial—no credit card required.
Explore easy tips for better employee reviews and see how Paperform can make the whole process a bre...
Explore the top UX design tools of 2024— from Figma to Webflow and everything in between.
The Well-Paid Expert helps small business owners grow through interactive quizzes.