Introduction to Bias, Confounding and Effect Modification

Overview

Medical studies can produce precise numerical results that nevertheless give a distorted answer. Bias may enter through participant selection or inaccurate measurement. Confounding may make an exposure appear beneficial or harmful because groups differed before the exposure occurred. In contrast, effect modification is not an error: it indicates that the effect or association differs across clinically meaningful subgroups.

These concepts answer different questions:

• Bias: Has the study process systematically distorted the estimate?
• Confounding: Is the exposure–outcome comparison mixed with the influence of a pre-exposure common cause?
• Effect modification: Does the exposure effect differ according to another characteristic, and on which effect scale?

The distinction matters because the responses are different. Bias should be prevented or assessed. Confounding should be controlled when estimating a causal effect. Effect modification should be described and interpreted rather than automatically removed.

Why this matters

A larger sample can narrow a confidence interval around a biased estimate. A sophisticated regression model can adjust for measured variables but cannot rescue an outcome that was recorded inaccurately or a population that entered the analysis through a distorting selection process.

Confounding is also not solved by adding every available variable to a model. Adjusting for a factor that occurs after the exposure may remove part of the effect being studied. Adjusting for a common consequence of exposure and outcome causes may introduce selection bias.

Effect modification creates a different challenge. An overall estimate may conceal a clinically important benefit in one subgroup and little benefit in another. Researchers must therefore know whether subgroup differences represent bias, confounding, chance variation or genuine variation in the effect measure.

Modern epidemiology treats these questions as problems of study design, measurement and causal structure rather than as matters that can be settled by statistical significance alone [1].

Bias: systematic distortion of an estimate

Bias is a systematic difference between the estimate produced by a study and the target quantity that the study intends to estimate.

Bias differs from random sampling variability:

• random variability causes estimates to differ across repeated samples;
• bias shifts estimates systematically because of how participants were selected, variables were measured or analyses were conducted;
• increasing sample size commonly reduces standard error but does not automatically reduce bias.

Bias can move an estimate towards or away from no association, exaggerate it, conceal it or reverse its direction.

Selection bias

Selection bias occurs when the process determining who enters, remains in or is included in an analysis creates a distorted exposure–outcome comparison. A structural view describes this as distortion caused by conditioning on selection when selection is influenced by the exposure and outcome, or by causes of those variables [2].

In the pneumonia cohort, selection bias could arise if the analysis includes only patients with a complete electronic severity score and completeness is more likely:

• among patients who received early specialist review; and
• among patients whose clinical deterioration prompted more intensive documentation.

The comparison among complete records could then differ from the comparison in the intended cohort.

Other examples include:

• differential loss to follow-up;
• excluding participants after exposure according to a post-exposure event;
• hospital-based control selection that does not represent the population producing the cases;
• analysing only treatment adherers when adherence is related to prognosis; and
• conditioning on survival to a later study visit.

Selection bias is not the same as limited generalisability. A trial may enrol a narrow, highly selected group yet still estimate an internally valid treatment effect for those participants. The narrow eligibility may limit applicability to other patients without necessarily biasing the internal randomised comparison.

Information bias

Information bias occurs when information used in a study is measured, recorded or classified inaccurately in a way that distorts the estimate [3].

In the scenario, the documented review time may not equal the true clinical review time. A specialist may assess the patient at 05:40 after admission but enter the note at 06:20. If exposure is defined from note-entry time, the patient may be incorrectly classified as not having early review.

Misclassification is the assignment of a categorical exposure or outcome to the wrong category. It may be:

• non-differential, when classification accuracy does not vary according to the comparison variable; or
• differential, when accuracy differs across exposure, outcome or relevant subgroup status.

Non-differential misclassification does not universally bias an estimate towards no association. The direction depends on the variable, number of categories, error mechanism and effect measure. Differential misclassification can move the estimate in either direction.

Examples of information bias include:

• recall of previous exposure differing between cases and controls;
• outcome assessors using knowledge of treatment when classifying a subjective outcome;
• laboratory instruments calibrated differently across study sites;
• self-reported adherence differing from actual medicine use; and
• a diagnostic code that incompletely identifies the clinical condition.

Information bias is best reduced through design and data collection:

• operational definitions;
• validated instruments;
• standardised procedures;
• training and quality checks;
• blinding of assessors where feasible;
• repeated or independent measurements; and
• validation studies in an appropriate subsample.

Confounding: mixing of causal effects

Suppose the aim is to estimate the causal effect of early specialist review on 30-day mortality.

A confounder is a pre-exposure common cause of the exposure and outcome, or a measured variable that adequately represents such a cause for the intended analysis. It creates a mixing of the exposure effect with differences in baseline prognosis.

Baseline pneumonia severity is a plausible confounder because:

1. severity is determined before specialist review;
2. severe illness increases the likelihood of early specialist review; and
3. severe illness increases 30-day mortality.

Defining confounding only as “a variable associated with both exposure and outcome” is inadequate. That rule can incorrectly identify:

• a mediator, which lies on the pathway through which exposure affects outcome; or
• a collider, which is a common consequence of two other variables.

Causal knowledge must guide confounder selection; statistical association alone is insufficient [4].

Confounding by indication

In clinical practice, patients with worse prognosis may be more likely to receive an intensive treatment or specialist service. This is confounding by indication.

A crude observational comparison can therefore make beneficial care appear harmful because the care was preferentially given to patients who were already at greater risk.

Preventing or controlling confounding

Confounding can be addressed through design or analysis.

Design approaches include:

• random allocation;
• restriction to a defined subgroup;
• matching on prespecified baseline factors; and
• careful measurement of important causes of exposure and outcome.

Analysis approaches include:

• stratification;
• direct standardisation;
• regression adjustment;
• inverse-probability weighting; and
• other causal-inference methods.

Each method requires assumptions. Adjustment cannot control a confounder that was not measured adequately. Residual confounding remains when relevant confounding is unmeasured, measured imprecisely, modelled inadequately or categorised too coarsely.

Randomisation reduces confounding through the treatment-assignment mechanism, but post-randomisation selection, non-adherence and missing outcomes can still introduce other biases.

Effect modification: clinically meaningful variation

Effect modification occurs when an effect measure differs across levels of another variable.

In the scenario, baseline severity modifies the effect on the risk-difference scale if the early-review versus later-review difference in 30-day mortality risk is larger among patients with severe pneumonia than among patients with non-severe pneumonia.

Effect modification is not a bias. It may reveal:

• a subgroup with greater potential benefit;
• a subgroup with possible harm;
• differences in baseline risk that change absolute benefit;
• biological or service-delivery mechanisms; or
• a need for targeted implementation.

Effect modification should be prespecified when possible, supported by clinical reasoning and reported using subgroup-specific effect estimates with confidence intervals.

Effect modification depends on the scale

Two common effect scales for a binary outcome are:

• risk difference: exposed risk minus comparator risk;
• risk ratio: exposed risk divided by comparator risk.

The effect may vary on one scale but not another. A constant risk ratio can produce different risk differences when baseline risks differ.

Researchers should therefore name the scale rather than saying only that “an interaction was present”.

Effect modification and statistical interaction

A regression interaction term is a model parameter used to represent departure from a specified no-interaction pattern. Its interpretation depends on the model and scale.

Effect modification refers to variation in an effect measure across strata. Statistical interaction is related but not identical, particularly in causal analyses involving two exposures [5].

The terms are often used interchangeably in applied reports, but clear reporting should state:

• the subgroup variable;
• the effect measure;
• the subgroup-specific estimates;
• the confidence intervals; and
• how the subgroup comparison was assessed [6].

Confounder, modifier or both?

A variable can play more than one role.

Baseline severity in the simulated cohort is:

• a confounder, because it affects who receives early review and is a cause of mortality; and
• a possible effect modifier, because the absolute association between early review and mortality may differ between severe and non-severe illness.

These roles require different responses:

• control severity when estimating an overall severity-adjusted association;
• present severity-specific estimates when assessing effect modification.

A variable should not be labelled a confounder or modifier without reference to:

• the exposure;
• the outcome;
• the target effect;
• the population; and
• the effect scale.

Table 1. Distinguishing the core concepts

Data structure

Simulated pneumonia-cohort variables

simulated data

Table 2. Variables needed to assess bias, confounding and effect modification

Variable	Example value	Role
patient_id	CAP001	Record identifier
severity_baseline	Severe	Pre-exposure confounder and possible effect modifier
review_time_documented	4.2 hours	Recorded exposure information
early_review	Yes	Exposure derived from documented review time
death_30d	No	Binary outcome
followup_complete	Yes	Indicator of inclusion in complete-outcome analysis
hospital_site	Site C	Design and contextual variable

Note: Severity must be assessed using information available before early specialist review. A post-review measure may lie on the causal pathway and should not automatically replace baseline severity.

Table 3. Simulated mortality data by baseline severity

Baseline severity	Review group	Deaths	Participants	Mortality risk
Severe	Early review	120	600	20.0%
Severe	Later or no early review	60	200	30.0%
Non-severe	Early review	4	200	2.0%
Non-severe	Later or no early review	30	1,000	3.0%

Note: Early review is more common among severe patients. Severe illness is also associated with higher mortality.

(Calculations for the mortality risks shown in Table 3 are not shown here.)

Worked example

Preventing and assessing the problems

Plan before collecting data

Bias and confounding should be addressed during protocol development.

Researchers should:

• define the target population and selection process;
• specify exposure, outcome and time origin;
• identify important baseline causes of exposure and outcome;
• choose validated or standardised measurements;
• record reasons for non-participation and loss to follow-up;
• prespecify the primary effect measure;
• identify a small number of clinically justified effect modifiers; and
• preserve the timing of variables so that baseline factors are not confused with mediators.

Compare, but do not rely only on, crude and adjusted estimates

A large change between crude and adjusted estimates can indicate confounding, but a small change does not prove that confounding is absent.

Change-in-estimate rules alone are inadequate because:

• confounders may cancel one another;
• measurement error may weaken adjustment;
• non-collapsible effect measures can change after adjustment without confounding;
• model misspecification can hide or create differences; and
• unmeasured confounding remains invisible.

Conduct sensitivity analyses

A sensitivity analysis examines how conclusions change under alternative reasonable assumptions.

Examples include:

• different exposure definitions;
• alternative missing-data assumptions;
• restriction to records with validated timing;
• quantitative bias analysis for misclassification;
• alternative confounder-adjustment sets; and
• effect modification on both absolute and relative scales.

Sensitivity analysis does not prove that the primary estimate is unbiased. It shows how dependent the conclusion is on specified assumptions.

Confounders, mediators and colliders MMed extension

A causal diagram can clarify whether adjustment is appropriate.

• A confounder is a cause of exposure and outcome, such as baseline severity influencing both early review and mortality.
• A mediator is caused by the exposure and lies on a pathway to the outcome, such as improved antibiotic selection following specialist review.
• A collider is a common consequence of two variables. Conditioning on it can create a non-causal association between its causes.

Adjusting for every measured variable is therefore unsafe. A mediator may remove part of the total effect that the study intends to estimate. A collider may introduce selection bias. The choice of adjustment variables should follow the estimand and assumed causal structure rather than an automated significance-based procedure.

Causal diagrams and directed acyclic graphs are developed in BIO-P16-M02.

Suggested reporting

The Strengthening the Reporting of Observational Studies in Epidemiology statement recommends transparent reporting of participant selection, variable definitions, bias, confounding control, subgroup analyses and limitations [8].

The Results wording distinguishes the crude association, adjusted association and scale-specific effect modification. It does not claim that standardisation alone proves causality.

Common mistakes

Knowledge check

Why can a very large study still produce a seriously biased estimate?

Answer and reasoning

Increasing sample size mainly reduces random sampling variability. It does not correct a systematically distorted selection process, inaccurate measurement or uncontrolled confounding. The confidence interval may become narrow around an incorrect target.

Why is baseline pneumonia severity a confounder in the simulated cohort?

Answer and reasoning

Severity is determined before specialist review, influences the likelihood of receiving early review and causes a higher risk of death. It therefore creates baseline prognostic differences between the review groups that can distort the crude exposure–outcome comparison.

Why is improved antibiotic selection after specialist review not automatically a confounder?

Answer and reasoning

It occurs after the exposure and may be part of the pathway through which specialist review affects mortality. It is therefore a possible mediator rather than a pre-exposure common cause. Adjusting for it could remove part of the total effect.

What does it mean that severity modifies the association on the risk-difference scale but not the risk-ratio scale?

Answer and reasoning

The absolute mortality reduction differs between severe and non-severe patients, at −10.0 and −1.0 percentage points, while the relative risks are both 0.67. Effect modification must therefore be described with reference to a particular effect measure.

Does the severity-adjusted risk difference prove that early specialist review reduces mortality?

Answer and reasoning

No. Standardisation addresses the measured baseline-severity difference under the stated method, but causal interpretation also requires no important unmeasured confounding, valid measurement, appropriate selection, correct model assumptions and a well-defined exposure strategy.

Key take-home messages

References

1. Lash TL, VanderWeele TJ, Haneuse S, Rothman KJ. Modern epidemiology. 4th ed. Philadelphia: Wolters Kluwer; 2021.
2. Hernán MA, Hernández-Díaz S, Robins JM. A structural approach to selection bias. Epidemiology. 2004;15(5):615–625. https://doi.org/10.1097/01.ede.0000135174.63482.43
3. Kesmodel US. Information bias in epidemiological studies with a special focus on obstetrics and gynecology. Acta Obstet Gynecol Scand. 2018;97(4):417–423. https://doi.org/10.1111/aogs.13330
4. Hernán MA, Hernández-Díaz S, Werler MM, Mitchell AA. Causal knowledge as a prerequisite for confounding evaluation: an application to birth defects epidemiology. Am J Epidemiol. 2002;155(2):176–184. https://doi.org/10.1093/aje/155.2.176
5. VanderWeele TJ. On the distinction between interaction and effect modification. Epidemiology. 2009;20(6):863–871. https://doi.org/10.1097/EDE.0b013e3181ba333c
6. Knol MJ, VanderWeele TJ. Recommendations for presenting analyses of effect modification and interaction. Int J Epidemiol. 2012;41(2):514–520. https://doi.org/10.1093/ije/dyr218
7. Hernán MA, Clayton D, Keiding N. The Simpson’s paradox unraveled. Int J Epidemiol. 2011;40(3):780–785. https://doi.org/10.1093/ije/dyr041
8. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP, et al. Strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ. 2007;335(7624):806–808. https://doi.org/10.1136/bmj.39335.541782.AD