Bias in Research
A board-focused tour of research bias for STEP 1: how to name each bias from a vignette, which study design is most vulnerable, and the design or analytic fix — including the confounding vs effect-modification discriminator and the lead-time vs length-time screening traps.
What bias is (and isn't)
Bias is a systematic error that pushes a study's result in one direction, wrecking internal validity. It is NOT random error (imprecision, fixed by a bigger sample) — no amount of extra subjects removes bias; it must be prevented by design. The boards test three moves: (1) name the bias from the vignette, (2) know which study design is most vulnerable, (3) pick the fix.
Keep three ideas separate:
- Bias — flawed methods systematically skew the estimate.
- Confounding — a third variable distorts (or manufactures) an apparent exposure–outcome association (often listed on its own).
- Effect modification — a real difference in effect across subgroups; a finding to report, not an error to eliminate.
Two families dominate STEP 1: selection bias (who enters or stays in the study) and information / measurement bias (how the data are gathered).
- Selection bias — nonrandom enrollment/retention → unrepresentative sample. Fix: randomization, consistent enrollment.
- Berkson bias — hospital-based controls are sicker than the general population → distorted (classically spuriously low) case-control OR. Fix: community/population controls.
- Attrition (loss to follow-up) — differential dropout in a cohort. Fix: minimize and track losses.
- Recall bias — cases remember past exposures more than controls; classic in retrospective case-control. Fix: records, prospective design.
- Measurement / observer-expectancy bias — data recorded inaccurately or per expectation. Fix: blinding, objective/standardized instruments.
- Hawthorne effect — subjects change behavior because they are watched. Fix: control group, unobtrusive measurement.
- Procedure bias — groups handled differently apart from the intervention. Fix: blinding, placebo.
- Lead-time & length-time bias — screening traps (see below).
Core biases at a glance
| Bias | Definition | Classic vignette | Reduce by |
|---|---|---|---|
| Selection (Berkson) | Sample not representative | Hospitalized controls sicker | Randomize; population controls |
| Recall | Differential memory of exposure | Mothers of sick infants "recall" more | Records; prospective design |
| Measurement | Systematically wrong data | Faulty BP cuff; leading questions | Calibrate; standardize |
| Observer-expectancy | Researcher sways the result | Unblinded investigator scores outcome | Double-blinding, placebo |
| Hawthorne | Behavior changes when watched | Hand-washing jumps during an audit | Control group |
Vignette 1 — A case-control study links maternal caffeine to birth defects. Mothers of affected infants report far more caffeine than mothers of healthy infants, yet medical records show similar intake. → Recall bias (cases over-recall exposures); most common in retrospective case-control designs. Next best step / fix: use prospective data or objective records.
Vignette 2 — A hospital case-control study uses inpatients on other wards as controls; the odds ratio comes out implausibly low. → Berkson (selection) bias — hospitalized controls are sicker than the general population, spuriously deflating the OR. Fix: community/population-based controls.
Vignette 3 — ICU hand-hygiene compliance rises from 40% to 90% while observers are on the unit. → Hawthorne effect.
- Confounding — a variable tied to BOTH exposure and outcome that is NOT on the causal pathway; it distorts the association. Classic: smoking confounds the apparent alcohol–lung cancer association.
- Reduce confounding — by design: randomization (best), restriction, matching, crossover.
- Reduce confounding — by analysis: stratification, standardization, multivariate regression.
- Effect modification (interaction) — the exposure's true effect genuinely DIFFERS across strata (e.g., a drug helps one genotype, not another). Report it; do not "correct" it — it is not an error.
- Key discriminator on stratified analysis: confounding → stratum estimates are similar to each other but differ from the crude estimate; effect modification → stratum estimates differ from each other.
Vignette A — A new screening test detects a cancer 3 years earlier than symptom-based diagnosis. Screened patients appear to "survive" longer measured from diagnosis, but age at death is unchanged. → Lead-time bias — earlier detection lengthens apparent survival without delaying death. Fix: compare disease-specific mortality, not survival time from diagnosis.
Vignette B — A screening program reports excellent survival, but it mainly catches slow-growing, indolent tumors while aggressive cancers surface between screens (interval cancers). → Length-time bias (extreme form = overdiagnosis): slowly progressive disease is over-represented among screen-detected cases. Fix: randomized screening trial with mortality endpoints.
Lead-time vs length-time bias
| Feature | Lead-time bias | Length-time bias |
|---|---|---|
| Mechanism | Earlier detection, same date of death | Preferential detection of slow/indolent disease |
| Apparent effect | ↑ survival measured from diagnosis | ↑ survival credited to screening |
| Real benefit | None (unless earlier Tx truly helps) | Overstated |
| Correction | Disease-specific mortality | RCT with mortality endpoints |
| Extreme form | — | Overdiagnosis |
Practice Research Methods now
Board-style questions, spaced-repetition flashcards, and a Socratic AI tutor — free to start.