Missing Data in Medical Research

Overview

Missing data occur when a value required for a planned analysis is unavailable. The absence may result from non-attendance, withdrawal, illness, death, device failure, incomplete records, an inapplicable question or a deliberate decision not to measure something.

Missingness matters because the participants with observed data may differ from those without them. An analysis restricted to complete records can lose precision and may estimate the wrong quantity. Replacing missing values with a simple constant can conceal uncertainty and create artificial patterns.

A principled approach begins before data collection:

1. define the estimand and required variables;
2. prevent avoidable missingness;
3. record reasons and timing;
4. describe the observed pattern;
5. state the assumptions underlying the primary analysis; and
6. perform sensitivity analyses for important departures from those assumptions.

Missing-data methods cannot reconstruct unknowable facts without assumptions. Their purpose is to use observed information coherently and make those assumptions visible.

Why this matters

Suppose participants with poor blood-pressure control are less likely to attend a follow-up visit because they feel discouraged or unwell. If the analysis includes only attenders, the observed mean blood pressure may be lower than the mean that would have been seen for all randomised participants.

The percentage missing does not determine the size or direction of bias. Five per cent missing can be important if missingness is strongly related to the outcome, while a larger percentage may produce little bias under a favourable process and a suitable analysis.

In clinical trials, missing outcomes can weaken the benefit of randomisation. Prevention and continued outcome collection after treatment discontinuation are therefore preferable to relying solely on statistical methods [1].

Types and patterns of missing data

Unit and item nonresponse

Unit nonresponse occurs when no study information is obtained from a selected or enrolled person at a required stage.

Item nonresponse occurs when some information is available but a particular variable is missing.

In the trial, a participant may attend week 12 but decline a treatment-burden questionnaire. That is item nonresponse. A participant with no week-12 contact has unit nonresponse for that visit.

Intermittent missingness and dropout

In longitudinal data:

• intermittent missingness occurs when one visit is missing but later visits are observed;
• dropout occurs when outcomes are missing from a point onwards.

These patterns require different investigation. A missed week-4 visit does not prove that week-12 data will also be unavailable.

Structural missingness

A value is structurally missing when it is not applicable by design.

For example, a reason for stopping telemonitoring is not applicable to a usual-care participant. It should not be confused with a missing response from someone who received telemonitoring.

Missingness by variable and pattern

The study should report:

• number and percentage missing for each important variable;
• denominators;
• missingness by treatment group, clinic and time;
• combinations of variables missing together;
• reasons where available; and
• characteristics associated with missingness.

A missing-data map or pattern table can reveal whether the problem is concentrated in one site or visit.

Simulated missing-outcome profile

simulated data

Table 1. Week-12 outcome completeness by allocated group

Allocated group	Observed outcome	Missing outcome	Total	Percentage missing
Telemonitoring	114	6	120	5.0%
Usual care	112	8	120	6.7%
Total	226	14	240	5.8%

Note: Percentages alone do not identify the missingness mechanism.

(The missing-outcome percentages are calculated from the counts and denominators shown in the table.)

Table 2. Recorded reasons for missing week-12 outcome

Reason	Number
Unable to contact	4
Transport or work difficulty	4
Intercurrent illness	2
Withdrew follow-up permission	2
Reason not established	2

Note: A recorded reason may inform, but does not by itself prove, a statistical missingness assumption.

Missingness mechanisms are assumptions

The terms missing completely at random, missing at random and missing not at random describe relationships between missingness and data values. They are assumptions for analysis, not labels established by a single test.

Missing completely at random

Data are missing completely at random, abbreviated MCAR, when the probability of a value being missing is unrelated to observed and unobserved study data.

An example might be a blood sample destroyed in a random laboratory accident unrelated to patient characteristics or results.

MCAR is a strong assumption. Under MCAR, complete-record analysis can be unbiased for many estimands, although it loses information and precision.

Missing at random

Data are missing at random, abbreviated MAR, when, after conditioning on appropriate observed information, the probability of missingness does not depend further on the unobserved value.

For example, week-12 attendance may depend on clinic, age, baseline blood pressure and week-4 blood pressure. If these observed variables adequately explain the relationship between missingness and the unseen week-12 value, MAR may be a useful working assumption.

MAR does not mean that missingness is random in everyday language. Missingness may be strongly related to observed characteristics.

Missing not at random

Data are missing not at random, abbreviated MNAR, when missingness still depends on the unobserved value after conditioning on the observed information used in the model.

For example, participants with unusually high unobserved week-12 blood pressure may be less likely to attend even after accounting for baseline and week-4 measurements.

The observed data generally cannot distinguish MAR from plausible MNAR mechanisms. Sensitivity analysis is therefore essential when departures could change the conclusion.

Prevent missing data before analysis

The most effective missing-data strategy begins with study design and conduct.

Useful measures include:

• collect only variables necessary for the research objectives;
• define acceptable visit windows and alternative collection methods;
• continue outcome collection after intervention discontinuation where consent permits;
• separate withdrawal from treatment from withdrawal from follow-up;
• obtain several secure contact routes;
• train staff to record reasons promptly;
• monitor missingness by site and group;
• correct operational problems early;
• use participant-centred follow-up procedures; and
• avoid burdensome duplicate measurements.

A participant’s right to withdraw must always be respected. Prevention means reducing avoidable loss, not pressuring participants.

Common analytical approaches

Complete-record analysis

A complete-record analysis, also called complete-case analysis, includes only observations with all variables required for that analysis.

Advantages are simplicity and compatibility with many standard procedures.

Limitations include:

• reduced sample size;
• reduced precision;
• possible change in the target population;
• possible bias when completeness is related to the outcome; and
• different analyses using different subsets.

Complete-record analysis is not automatically valid because the missing percentage is small. Its validity depends on the estimand, model and missingness process.

Available-case summaries

An available-case summary uses all observed values for each variable. Denominators can therefore differ across rows or columns.

This may be suitable for descriptive reporting if denominators are explicit. It can produce internally inconsistent covariance or correlation estimates when different pairs use different participants.

Single imputation

Single imputation fills each missing value once, for example using:

• the mean;
• the median;
• the last observation carried forward;
• a best or worst value; or
• one predicted value.

Routine single imputation is generally unsuitable for primary inference because it treats an imputed value as though it were observed, understates uncertainty and can distort relationships.

Last observation carried forward assumes that the outcome would remain at the last observed value. This is often clinically implausible and does not generally preserve randomisation or remove bias.

Likelihood-based methods

Likelihood-based models can use incomplete outcome sequences under model and missingness assumptions. Mixed-effects models are commonly used for repeated numerical outcomes and can include participants with some observed visits without explicitly filling every missing value.

Their validity still depends on the chosen outcome model and a suitable missingness assumption, commonly MAR for likelihood inference based on observed outcomes.

Inverse-probability weighting

Inverse-probability weighting gives more weight to observed participants who resemble those with missing outcomes. It models the probability of being observed using recorded variables.

It can reduce selection caused by missingness when the model is correct and all important predictors are measured. Large weights may reduce precision and signal poor overlap.

Multiple imputation

Multiple imputation creates several completed datasets containing different plausible values for each missing item, analyses each dataset using the planned model and combines the results.

The variation among imputations contributes to uncertainty. Multiple imputation was developed to reflect that missing values are not known with certainty [2], [3].

How multiple imputation works

A practical workflow is:

1. define the analysis model and estimand;
2. identify variables with missing data;
3. include treatment, outcomes, analysis covariates and useful predictors of missingness or missing values;
4. choose imputation models compatible with each variable type;
5. generate multiple imputed datasets;
6. examine convergence and plausibility;
7. fit the planned analysis to every dataset;
8. combine estimates and standard errors using Rubin’s rules; and
9. perform sensitivity analyses.

The imputation model should usually be at least as rich as the analysis model. It should preserve important non-linear terms, interactions, bounds and the multilevel or longitudinal structure where relevant.

Rubin’s rules

For each imputed dataset, the analyst obtains an estimate and variance. Rubin’s rules combine:

• the average estimate;
• average within-imputation variance; and
• between-imputation variance.

The total uncertainty is larger when imputations differ substantially, reflecting limited information about the missing values.

Multiple imputation does not manufacture data. It propagates uncertainty under a specified model.

Align the approach with the estimand

Missing-data handling should follow the question being estimated.

In the trial, the estimand concerns all randomised participants regardless of telemonitoring discontinuation. Therefore:

• treatment discontinuation should not cause automatic exclusion;
• outcome collection should continue where possible;
• allocation must be included in the imputation and analysis models; and
• the imputation should support the planned treatment-policy interpretation.

Death before the target visit is not always ordinary missingness. If the outcome cannot exist after death, the estimand must specify how death is handled. A convenient imputation of a hypothetical blood pressure after death would not answer a clearly defined clinical question.

Worked example

Reference-based and delta-adjusted sensitivity analyses MMed extension

A delta adjustment changes imputed values by a specified amount to represent outcomes being systematically better or worse than predicted under MAR.

A reference-based imputation may assume that after a particular intercurrent event, outcomes in one group follow a distribution related to a reference group. These methods are often used in trials but answer questions tied to their assumptions.

Sensitivity parameters should be:

• clinically interpretable;
• chosen without using the preferred result;
• varied across a plausible range; and
• reported transparently.

A tipping-point analysis identifies the departure required to change a specified conclusion. The tipping point is not an estimate of the true missingness mechanism.

Suggested reporting

Transparent reporting should include missingness by group, reasons, assumptions, variables in the imputation model, number of imputations, diagnostics and sensitivity analyses [3], [4]. Practical guidance for randomised trials also emphasises choosing multiple imputation according to the analysis question and available information rather than applying it mechanically [5]. Prevention, continued follow-up where permitted and sensitivity analysis remain central even when modern statistical methods are used [6].

Common mistakes

Knowledge check

Does a 5% missing-outcome rate guarantee negligible bias?

Answer and reasoning

No. Bias depends on how missingness relates to the outcome and other variables, not only on the percentage missing. A small but selective loss can materially distort an estimate.

What does missing at random mean?

Answer and reasoning

After conditioning on the observed information included in the model, the probability of missingness is assumed not to depend further on the unobserved value. It does not mean that missingness occurs for no reason.

Why is mean imputation usually unsuitable for primary inference?

Answer and reasoning

It gives every missing observation one fixed value, reduces variability, distorts associations and treats imputed values as certain. Standard errors and confidence intervals are therefore usually inappropriate.

Why should treatment discontinuation not automatically remove a randomised participant from the primary analysis?

Answer and reasoning

If the estimand concerns the effect of treatment allocation regardless of discontinuation, removing the participant changes the target question and may break the randomised comparison. Outcome collection should continue where permitted.

What does an MNAR sensitivity analysis establish?

Answer and reasoning

It shows how the result changes under specified departures from the primary assumption. It does not prove that the chosen MNAR scenario is true or cover every possible missingness process.

Key take-home messages

References

1. National Research Council. The prevention and treatment of missing data in clinical trials. Washington (DC): National Academies Press; 2010. https://doi.org/10.17226/12955
2. Rubin DB. Inference and missing data. Biometrika. 1976;63(3):581–592. https://doi.org/10.1093/biomet/63.3.581
3. Sterne JAC, White IR, Carlin JB, Spratt M, Royston P, Kenward MG, et al. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ. 2009;338:b2393. https://doi.org/10.1136/bmj.b2393
4. White IR, Royston P, Wood AM. Multiple imputation using chained equations: issues and guidance for practice. Stat Med. 2011;30(4):377–399. https://doi.org/10.1002/sim.4067
5. Jakobsen JC, Gluud C, Wetterslev J, Winkel P. When and how should multiple imputation be used for handling missing data in randomised clinical trials—a practical guide with flowcharts. BMC Med Res Methodol. 2017;17:162. https://doi.org/10.1186/s12874-017-0442-1
6. Little RJ, D’Agostino R, Cohen ML, Dickersin K, Emerson SS, Farrar JT, et al. The prevention and treatment of missing data in clinical trials. N Engl J Med. 2012;367(14):1355–1360. https://doi.org/10.1056/NEJMsr1203730