Part 14 · Module 3
Information Bias
Identify information bias from inaccurate exposure, outcome or covariate measurement, and design health studies to reduce misclassification and observer-related error.
Learning outcomes
After completing this lecture, you should be able to:
- Define information bias and distinguish it from random measurement variation.
- Differentiate differential and non-differential misclassification using the measurement mechanism.
- Recognise recall, interviewer, detection, verification and record-based information biases.
- Explain why non-differential misclassification does not universally bias estimates towards the null.
- Plan measurement, blinding, validation and sensitivity strategies that reduce information bias.
Overview
Information bias occurs when information used to classify participants, exposures, outcomes or covariates is systematically inaccurate in a way that distorts the study estimate. It can arise from self-report, interviewer behaviour, diagnostic work-up, medical-record completeness, device calibration, coding rules or differential intensity of observation.[1]
Measurement error and information bias overlap, but the terms are not identical. Random fluctuations can reduce precision without creating a systematic directional distortion. Information bias focuses on measurement processes that systematically alter comparison between groups. The same variable can be measured with both random and systematic error.
Why this matters
Health data are rarely direct observations of the underlying construct. “Medication adherence” may come from self-report, pharmacy refill, electronic caps or serum levels. “Disease” may depend on a reference standard applied differently across participants. “Exposure” may be reconstructed from memory years later.
If the measurement process differs according to exposure, outcome or group membership, statistical adjustment may not recover the intended information. The best defence is usually design-stage standardisation, masking where feasible, validated measurement and explicit source hierarchies.[1,2]
Key concepts
For categorical variables, misclassification means assigning an observation to the wrong category. Misclassification is called differential when its sensitivity or specificity differs across levels of another variable relevant to the comparison; otherwise it is often described as non-differential. For continuous measures, systematic calibration differences, heaping, digit preference and instrument drift can create analogous problems.
A common teaching shortcut says that non-differential misclassification always biases an association towards the null. That statement is not universally true. The direction depends on the number of categories, error structure, dependence between errors, covariate adjustment and the scale of the effect.[2,3]
Recall and self-report bias
Recall bias is particularly concerning when memory of a past exposure differs according to current outcome status. In a case-control study, people with a new diagnosis may search their memories more intensively than controls. Social desirability can similarly affect reporting of smoking, sexual behaviour, alcohol use or adherence.
Strategies include shorter recall periods, neutral wording, validated instruments, objective records when appropriate and asking interviewers to follow the same prompts regardless of group.
Interviewer and observer effects
An interviewer who knows the hypothesis may probe exposed participants more intensively or classify ambiguous responses differently. Clinicians who know treatment allocation may assess subjective outcomes differently. Masking outcome assessors can reduce this risk, but masking is not always feasible. Standardised training, competency checks, scripts and objective adjudication rules are alternatives.
Inter-rater reliability describes consistency, but a group of raters can agree perfectly while all applying a biased rule. Reliability should therefore not be used as a substitute for validity.
Detection and surveillance bias
Groups receiving more healthcare contact have more opportunities for diagnoses, laboratory tests and adverse events to be recorded. An apparent association may therefore reflect observation intensity rather than biology. Protocols can standardise scheduled assessment, define outcome windows and collect healthcare-use variables when surveillance intensity is unavoidable.
Record-based misclassification
Electronic health records are created for care, not for the research question. Absence of a diagnosis code does not necessarily mean absence of disease. Site migration, external care, coding incentives and changing software can alter completeness over time. Researchers should validate critical algorithms when possible, state source hierarchies and avoid assuming that routinely collected data are intrinsically objective.
Diagnostic verification and incorporation
Diagnostic studies face specific information-bias mechanisms. Verification bias occurs when the reference standard is applied selectively, often according to the index-test result. Incorporation bias arises when the index test forms part of the reference standard. These mechanisms are addressed in Part 10 and in QUADAS-2, but the general lesson is the same: the measurement pathway can depend on the very result being evaluated.[4]
Prevention through measurement design
Define variables operationally before data collection; use appropriate instruments; calibrate devices; standardise timing; train observers; mask hypothesis or group where feasible; use the same source hierarchy across groups; record reasons a reference measurement was unavailable; and monitor site-specific distributions for drift.
For newly developed instruments, content validity, response-process evidence and structural evaluation should precede substantive causal interpretation.
Validation studies and correction
A validation substudy can compare the routine measure with a more accurate reference method in a subset. If sensitivity, specificity or calibration parameters are estimated credibly, quantitative bias analysis or measurement-error models may assess how much the main estimate could change. Such correction introduces its own assumptions and should not be presented as exact recovery of truth.
Investigate a suspected information bias
- Identify the variable that may be measured incorrectly.
- Specify the operational rule or instrument that produced the measurement.
- Ask whether error probabilities differ by exposure, outcome, intervention group, site or time.
- Determine whether the error affects classification, scale, timing or completeness.
- Choose prevention, validation or sensitivity analysis based on the mechanism rather than the bias label.
Illustrative outcome-source hierarchy
illustrativeTable 1. Standardised hierarchy for postpartum glucose-testing status
| Priority | Source | Interpretation |
|---|---|---|
| 1 | Linked laboratory result with valid date | Test completed |
| 2 | External laboratory report verified by study staff | Test completed |
| 3 | Participant reports testing but no verifiable record | Status uncertain; retain source flag |
| 4 | No local result and external source not checked | Do not automatically code as test failure |
| 5 | Verified no testing after complete source search | Test not completed |
The hierarchy prevents sites from treating absence of a local record as equivalent to absence of testing.
Questions for appraising information bias
- Were exposure, outcome and key covariates defined before data collection?
- Were measurement methods comparable across study groups and sites?
- Could knowledge of exposure, treatment or outcome influence measurement or adjudication?
- Were critical algorithms or instruments validated for the intended population and purpose?
- Was the likely direction and consequence of important measurement error considered rather than merely listed?
Probabilistic quantitative bias analysisMMed extension
When plausible distributions are available for sensitivity and specificity or other error parameters, probabilistic bias analysis can repeatedly sample those parameters and recalculate corrected effect estimates. The resulting distribution reflects uncertainty in the bias parameters as well as the observed data under the specified model. This approach can be more informative than assuming one fixed correction, but its usefulness depends on credible validation information and transparent parameter choices.[3]
Correlated and dependent measurement errors
Measurement errors are often discussed as though each variable were measured independently, but real studies frequently use the same source, observer or clinical encounter to determine several variables. A participant who systematically under-reports socially undesirable behaviours may misreport both exposure and confounders. A clinician who knows the index-test result may classify the reference diagnosis differently. A research assistant who knows treatment allocation may probe symptoms more intensely in one group. These dependent errors can produce bias patterns that are difficult to predict from simple non-differential-versus-differential labels.[1,2]
The design implication is to identify shared measurement pathways. If exposure and outcome are abstracted from the same narrative clinical note, the abstraction manual should specify independent criteria for each. If feasible, separate assessors or masked adjudication can reduce the chance that knowledge of one variable influences classification of another. Where masking is impossible, objective thresholds, standard prompts, recorded reasons for classification, and periodic reliability checks become more important.
Sensitivity and specificity belong to a measurement process
For a binary classification, sensitivity and specificity are useful ways to characterise error, but they should not be treated as immutable properties of a questionnaire, algorithm or diagnostic code. They may change with threshold, disease spectrum, assessor training, setting, data quality and the reference procedure. A code list validated in a tertiary hospital may perform differently in primary care; a self-report item may behave differently across languages or modes of administration.[2,3]
Accordingly, investigators should describe how a variable was measured in the study population, not simply cite a validation coefficient from another context. When a validation substudy is possible, its participants should span the relevant exposure or outcome spectrum, and the reference method should be as independent and credible as practical. Estimates of measurement properties should be reported with uncertainty rather than as exact constants.
In quantitative bias analysis, assumed sensitivity and specificity values should therefore be justified by empirical evidence and explored over plausible ranges rather than inserted as unquestioned correction factors.[3] The purpose is to determine how conclusions would change under credible measurement-error scenarios.
Harmonisation across sites, observers and time
Multicentre and longitudinal studies add a further source of information bias: the measurement system itself can drift. Laboratories may change assays, imaging equipment can be upgraded, clinical definitions may be revised, interviewers may deviate from scripted probes, and electronic record fields may change meaning after a software update. If these changes occur differently by exposure group, site or calendar period, apparent biological or behavioural differences may partly reflect the measurement process.
A measurement-harmonisation plan should therefore specify instrument versions, calibration procedures, allowable equipment, reference ranges, coding rules, interviewer training, retraining triggers and version dates. When a method changes during follow-up, the team should document overlap periods or bridging studies where feasible. Dataset metadata should identify the version or source used for each observation so that analysts can examine method effects later. Measurement-property evidence should also be considered in the population and use context for which an instrument is intended.[5]
For subjective outcomes, periodic blinded duplicate assessment can reveal observer drift. For laboratory outcomes, internal and external quality-control material can identify changes in analytic performance. For routinely collected data, code lists and extraction algorithms should be version-controlled. These procedures address information quality at its source rather than expecting statistical modelling to rescue an undocumented measurement system.
Reasoning about the likely direction of bias
Statements such as “non-differential misclassification biases towards the null” are unsafe as universal rules. The direction and magnitude of bias depend on the type of variable, number of categories, dependence of errors, true association and other design features.[1-3] For binary exposure misclassification under restrictive conditions, attenuation is common, but exceptions occur. Outcome misclassification can also behave differently depending on whether error is related to exposure.
A better reporting approach is mechanism-based: state what was measured imperfectly, why the error may differ between comparison groups, what direction seems plausible under specified assumptions, and whether a sensitivity analysis was performed. When the direction cannot be established, say so. Methodological honesty is preferable to a reassuring but unjustified assertion that bias would necessarily make the observed association smaller.
Interface with statistical analysis
Information bias should be addressed separately from sampling error. Measurement-error models, regression calibration and quantitative bias analysis require assumptions about the error process. For binary variables, sensitivity and specificity of classification may be incorporated; for continuous variables, reliability and calibration information may be relevant. The Biostatistics section provides the detailed analytical methods.
Worked example
Harmonising postpartum outcome ascertainment
Before pooling data, investigators discover that Site B searched only its local laboratory system. They do not simply add “site” as a regression covariate and assume the problem is solved. Instead, they identify all participants whose outcome could be affected, extend the source search to external results where permitted, and retain a source variable that documents how each outcome was established.
The team then compares the proportion reclassified after the harmonised search by access-burden category. If reclassification is more common among women with high burden, they recognise differential misclassification. A sensitivity analysis evaluates plausible remaining false-negative outcome rates in each exposure group.
The final report describes the initial site discrepancy, the harmonisation procedure and the sensitivity analysis. This preserves a distinction between correcting preventable ascertainment errors and making assumptions about residual error.
Suggested protocol or reporting wording
Common mistakes
Knowledge check
Cases are interviewed after diagnosis and remember past medication use more intensely than controls. What mechanism is most likely?
Answer and reasoning
This is a classic recall-bias mechanism: exposure reporting depends on outcome status. The concern is differential exposure misclassification rather than random recall error alone.
If a binary exposure has the same sensitivity and specificity among cases and controls, must the odds ratio be biased towards one?
Answer and reasoning
No. Although attenuation is common under particular simple conditions, the direction is not universal. It depends on the misclassification structure, categories, dependencies and analytical model.
Why does adding “site” to a regression model not necessarily repair different outcome ascertainment rules across sites?
Answer and reasoning
Because site adjustment controls mean differences associated with site; it does not reconstruct outcomes that were systematically misclassified. The measurement process itself should be harmonised or modelled.
Can an instrument be reliable but invalid?
Answer and reasoning
Yes. It can produce consistent repeated values while systematically failing to represent the intended construct or measuring it with a biased rule.
Key take-home messages
Key take-home messages
- Information bias is created by systematic error in how study information is obtained or classified.
- Differential versus non-differential error should be defined from the measurement mechanism.
- Non-differential misclassification does not universally bias effects towards the null.
- Standardisation, masking, validation and explicit source hierarchies are central preventive strategies.
- Sensitivity or quantitative bias analysis can assess residual measurement uncertainty but depends on explicit assumptions.
References
- Althubaiti A. Information bias in health research: definition, pitfalls, and adjustment methods. J Multidiscip Healthc. 2016;9:211-217. https://doi.org/10.2147/JMDH.S104807
- Rothman KJ, Lash TL, VanderWeele TJ, Haneuse S. Modern epidemiology. 4th ed. Philadelphia: Wolters Kluwer; 2021.
- Lash TL, Fox MP, Fink AK. Applying quantitative bias analysis to epidemiologic data. New York: Springer; 2009. https://doi.org/10.1007/978-0-387-87959-8
- Whiting PF, Rutjes AWS, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155(8):529-536. https://doi.org/10.7326/0003-4819-155-8-201110180-00009
- Mokkink LB, Prinsen CAC, Patrick DL, Alonso J, Bouter LM, de Vet HCW, et al. COSMIN study design checklist for patient-reported outcome measurement instruments. Amsterdam: COSMIN; 2019. https://www.cosmin.nl/
These materials support learning and research planning. They require adaptation to the study context and do not replace methodological, statistical, ethical, or regulatory consultation.