Bias in Sampling
Even with the best math, bad data leads to bad conclusions. Learn how to identify and avoid the traps of bias.
Sources of Bias: The Incomplete Frame
Sampling Bias (Undercoverage)
Occurs when the technique used to obtain the sample's individuals tends to favor one part of the population over another.
The Prediction: Alf Landon would crush FDR in a landslide (57% to 43%).
The Reality: FDR won in one of the biggest landslides in history (62%).
They surveyed people using Telephone Durectories and Car Registrations.
The Bias: In 1936 (Great Depression), only the wealthy had phones and cars. The wealthy favored Landon. The poor (FDR's base) were completely assumed out of existence (Undercoverage).
Nonresponse Bias
Nonresponse Bias exists when individuals selected to be in the sample who do not respond to the survey have different opinions from those who do.
- Refusal: "I assume to take this survey."
- Nonrespondent: Just wasn't home / didn't pick up.
How to fix it:
- Callbacks:Don't give up. Contact them again at a different time.
- Incentives:"Complete this survey for a $10 gift card." (Increases participation).
Response Bias
When the Answer is a Lie
Response bias exists when the answers on a survey do not reflect the true feeling of the respondent.
Interviewer Effect
A respondent might give the "safe" answer to please the interviewer. (e.g., Telling a dentist you floss daily).
Wording of Questions
Leading questions influence the answer.
"Do you support the destruction of our parks?" vs "Do you support cutting costs?"
Ordering of Questions
Previous questions prime the mind. Asking about "Love of Country" before "Flag Burning" yields different results than vice-versa.
Nonsampling vs. Sampling Error
Nonsampling Error
Errors that result from undercoverage, nonresponse bias, response bias, or data-entry error. These are mistakes that can be fixed.
Sampling Error
The error that results from using a sample to estimate information about a population.
Even a perfect random sample will likely not match the population exactly. This is natural variation.
LogicLens Practice Suite
Log in to Access Adaptive Practice
Our AI engine generates unique practice problems based on your progress.