Planning a Statistical Analysis

Overview

A statistical analysis should be designed as part of the study, not improvised after the outcome data are visible. Planning forces the research team to state exactly what will be estimated, which participants and observations will contribute, how variables will be defined and which assumptions will support the interpretation.

A statistical analysis plan, commonly abbreviated as SAP, is a detailed document describing how the study data will be analysed. In a clinical trial, it expands the principal statistical methods stated in the protocol into procedures that another qualified analyst could implement. International guidance emphasises prespecification of the primary analysis, analysis sets, missing-data methods, effect estimates, confidence intervals, multiplicity and sensitivity analyses [1], [2].

Prespecification does not mean that exploratory analysis is forbidden. It means that planned confirmatory analyses are distinguished transparently from additional analyses suggested by the observed data.

Why this matters

Many defensible analytical choices are possible in a medical study:

• analyse the final measurement or change from baseline;
• adjust or do not adjust for baseline values;
• include all randomised participants or only those completing treatment;
• use complete records or an explicit missing-data method;
• analyse a continuous outcome or divide it into categories;
• report an absolute effect, a relative effect or both;
• examine one prespecified subgroup or many data-driven subgroups; and
• transform a skewed variable or use a model suited to its original scale.

If investigators choose among these options after seeing which result is most favourable, the reported uncertainty no longer represents the full selection process. A detailed SAP reduces this flexibility, supports reproducibility and makes deviations visible [2].

Planning also prevents contradictions. A protocol may ask about the effect of treatment allocation, while the analysis excludes participants who did not adhere. A primary outcome may be defined as 12-week blood pressure, while the dataset contains several measurements and no rule for selecting the valid visit. These are scientific problems, not merely software decisions.

What should be planned

A statistical analysis plan should provide enough detail to implement the analysis without making important decisions after seeing treatment-group outcomes.

Study objectives and estimands

Each objective should correspond to a defined estimand: the population-level quantity that answers the research question.

For the primary objective, the plan should state:

• population: all adults meeting the eligibility criteria who were randomised;
• treatment conditions: allocation to telemonitoring plus usual care or usual care alone;
• outcome variable: clinic systolic blood pressure at 12 weeks, in mmHg;
• intercurrent events: events after randomisation that affect interpretation, such as stopping telemonitoring or changing medication;
• population-level summary: the adjusted difference in mean 12-week systolic blood pressure, calculated as telemonitoring minus usual care.

The International Council for Harmonisation E9(R1) addendum recommends aligning the clinical objective, estimand, estimation method and sensitivity analyses [3].

The primary estimand uses a treatment-policy strategy for intervention discontinuation and medication changes: participants remain in their allocated groups, and their 12-week blood-pressure values are analysed regardless of these events.

If a participant dies before 12 weeks, systolic blood pressure at 12 weeks does not exist. The protocol and SAP should specify a clinically defensible strategy rather than treating death as ordinary missingness. No deaths occur in the simulated example.

Outcome definitions

The plan must define the primary outcome precisely.

For this trial:

• the target visit is 12 weeks after randomisation;
• an acceptable visit occurs from day 77 to day 91;
• participants rest seated for five minutes;
• three automated readings are obtained one minute apart;
• the first reading is discarded;
• the outcome is the mean of the second and third readings;
• values are recorded in mmHg; and
• if only two valid readings are available, their mean is used according to the prespecified measurement rule.

These details determine the value that enters the analysis.

Analysis populations

An analysis population defines which participants contribute to a particular analysis and how they are grouped.

The primary analysis follows the intention-to-treat principle:

• participants are analysed according to randomised allocation;
• discontinuation or non-adherence does not change group assignment; and
• missing outcomes are addressed using the prespecified missing-data strategy.

A per-protocol analysis may exclude participants with specified major deviations. It answers a different question and should usually be supportive rather than a replacement for the primary randomised comparison.

A safety analysis may classify participants according to treatment actually received if that is clinically appropriate. Each population requires a precise operational definition.

Variables, coding and derivations

The SAP should identify:

• outcome variables;
• treatment or exposure variables;
• baseline covariates;
• stratification or matching variables;
• subgroup variables;
• time variables;
• derived variables;
• reference categories;
• units;
• permissible values; and
• rules for transformations or categorisation.

Continuous variables should not be divided into categories merely because categories appear easier to analyse. A clinical threshold may justify an additional categorical variable, but the original measurement should usually be retained.

Protocol, SAP and related documents

The protocol defines the scientific question, design, outcomes, sample-size basis and principal analysis. The SAP adds the operational detail needed to perform that analysis: estimands, analysis populations, variable derivations, models, assumptions, missing data, multiplicity, sensitivity analyses and planned outputs.

A data-management plan separately governs data capture, validation, coding, database lock and transfer. The final report states what was done, presents the results and identifies deviations from the protocol or SAP.

The main analysis strategy should be developed during protocol preparation. The detailed SAP should be approved before unblinded comparative outcome results are examined. A blinded data review may identify data-quality problems or impractical categories, but any resulting change should be justified, version controlled and approved before unblinding [1], [2].

Research question and variables

The primary objective is to estimate the effect of allocation to telemonitoring on mean 12-week systolic blood pressure.

Table 2. Primary analysis specification

Note: The adjusted mean difference remains the main result; the p-value does not measure clinical importance.

Why adjust for baseline blood pressure?

Baseline systolic blood pressure is strongly related to the 12-week value. Adjustment can improve precision and account for chance baseline imbalance.

Analysing only change from baseline is not automatically preferable. For randomised trials with baseline and follow-up measurements, analysis of covariance commonly provides an efficient comparison when its assumptions are reasonable [4].

Clinic is included because randomisation was stratified by clinic. Including stratification variables in the analysis maintains alignment with the design.

Data structure

Simulated telemonitoring-trial data

simulated data

Table 3. Example participant-level records

participant_id	clinic	allocated_group	sbp_baseline	sbp_12wk	intervention_stopped	medication_changed
HTM001	Clinic A	Telemonitoring	152 mmHg	134 mmHg	No	Yes
HTM002	Clinic B	Usual care	146 mmHg	141 mmHg	Not applicable	No
HTM003	Clinic A	Telemonitoring	155 mmHg	Missing	Yes	Yes
HTM004	Clinic C	Usual care	149 mmHg	138 mmHg	Not applicable	Yes

Note: Each row represents one randomised participant. Treatment discontinuation and medication changes do not alter allocated group.

Primary statistical model

The primary analysis uses linear regression, equivalent to analysis of covariance in this two-group setting.

The model reports:

• the adjusted mean difference;
• its 95% confidence interval;
• the standard error;
• the two-sided p-value; and
• adjusted group means where useful for interpretation.

The treatment coefficient is interpreted in mmHg. A negative value favours telemonitoring because lower systolic blood pressure is the desired outcome.

Statistical assumptions

Assessment should combine clinical knowledge, residual-versus-fitted plots, a quantile–quantile plot and checks for influential observations. A normality-test p-value should not determine the analysis by itself.

The plan must also state the response to important violations. Options may include correcting data errors, using robust standard errors, modelling a non-linear relationship or conducting a prespecified sensitivity analysis. Assumption checks should not become a search for whichever method gives the preferred result.

Missing data

Preventing missing outcomes is preferable to relying on statistical correction. The plan should report missingness by group and reason, continue outcome collection after intervention discontinuation where possible and state the assumptions supporting the analysis.

The primary analysis uses multiple imputation under a missing-at-random assumption: after accounting for recorded variables in the imputation model, the probability of a missing outcome does not depend further on its unobserved value.

The imputation model includes allocated group, clinic, baseline and interim blood pressure, medication change, intervention discontinuation, age and predictors of follow-up completion. Fifty datasets are imputed; the primary model is fitted to each and combined using Rubin’s rules. Multiple imputation remains dependent on model specification and assumptions [5].

Sensitivity analysis for data that may be missing not at random MMed extension

Data are missing not at random when the probability of missingness depends on the unobserved outcome even after conditioning on recorded information.

A prespecified sensitivity analysis can shift imputed 12-week systolic blood pressure by an assumed amount. For example, missing outcomes in the telemonitoring group may be increased by 3 or 5 mmHg relative to values predicted under the primary imputation model.

This analysis asks how severe an unfavourable departure from the missing-at-random assumption must be before the clinical conclusion changes. It does not identify the true missing values.

The reasons for missingness and plausible shift values should be informed by clinical knowledge rather than chosen after viewing the preferred result. Further methods are developed in BIO-P02-M03.

Secondary outcomes, multiplicity and subgroup analyses

Secondary outcomes include blood-pressure control, medication change, treatment burden and adverse events. Each needs its own definition, effect measure and method. For a binary outcome, report group risks and an appropriate contrast; do not describe an odds ratio as a risk ratio.

The trial has one primary outcome and one primary comparison, tested two-sided at 0.05. Secondary analyses are supportive or exploratory and are interpreted using effect estimates and confidence intervals rather than isolated p-values. Several confirmatory outcomes would require a formal multiplicity strategy.

The prespecified subgroup assessment examines whether the treatment effect varies with baseline systolic blood pressure using a treatment-by-baseline interaction. Baseline pressure remains continuous. Separate significance tests within subgroups do not test whether the subgroup effects differ.

Sensitivity and supportive analyses

A sensitivity analysis evaluates whether the primary conclusion is robust to an alternative assumption or analytical choice.

Prespecified sensitivity analyses include:

1. complete-record analysis using the same baseline-adjusted model;
2. analysis using heteroscedasticity-robust standard errors;
3. missing-not-at-random shift analyses;
4. adjustment for additional prespecified prognostic baseline variables; and
5. a per-protocol analysis as supportive evidence.

The complete-record and per-protocol analyses do not replace the primary analysis. Each estimates a potentially different quantity and depends on additional assumptions.

Worked example

Presenting the planned outputs

A SAP should include output specifications or table shells where useful. This reduces ambiguity and ensures that denominators, units and contrast directions are reported consistently.

Table 4. Planned primary-results table

Note: Group summaries use observed outcomes. The adjusted effect is from the primary multiple-imputation analysis of all 240 randomised participants. SD = standard deviation.

(Calculations for the group summaries, adjusted effect and P-value are not shown in this table; the 95% confidence interval calculation is shown in the worked example above.)

The table should distinguish observed descriptive summaries from model-based estimates. Denominators and missing values should be visible.

Suggested reporting

SPIRIT 2025 recommends transparent protocol reporting, while CONSORT 2025 specifies essential items for reporting completed randomised trials [6], [7].

Any deviation from the SAP should be identified, dated, justified and separated from prespecified analyses.

Reproducibility and governance

The analysis record should retain the final protocol and SAP, data dictionary, validated analysis dataset, derivation and analysis code, software and package versions, random seeds, diagnostics, output-generation code, review records and documented deviations.

Tables and manuscript values should be generated reproducibly rather than copied manually where possible. Substantive SAP changes require version control, dates, justification and approval.

Common mistakes

Knowledge check

Why should a statistical analysis plan be completed before unblinded outcome comparisons are examined?

Answer and reasoning

Seeing comparative outcomes can influence choices about exclusions, transformations, covariates, subgroups and models. Prespecification reduces outcome-driven analytical flexibility and makes planned analyses distinguishable from post hoc exploration.

What is the primary estimand in the simulated telemonitoring trial?

Answer and reasoning

It is the adjusted telemonitoring minus usual-care difference in mean 12-week clinic systolic blood pressure among all randomised participants, with discontinuation and medication changes handled under a treatment-policy strategy.

Why are participants analysed according to randomised allocation even if they stop telemonitoring?

Answer and reasoning

The primary question concerns the effect of allocation to the telemonitoring strategy. Reclassifying or excluding participants according to post-randomisation behaviour can break the randomised comparison and answer a different question.

Why is a complete-record analysis not automatically an adequate solution to missing outcomes?

Answer and reasoning

Participants with observed outcomes may differ systematically from those with missing outcomes. Complete-record analysis is valid only under assumptions about the missingness process and may reduce precision. The missing-data strategy should be justified and accompanied by sensitivity analyses.

A subgroup analysis gives p < 0.05 among older participants and p > 0.05 among younger participants. Does this prove that the treatment effect differs by age?

Answer and reasoning

No. Separate subgroup p-values do not test the difference between subgroup effects. Effect modification should be assessed using an interaction estimate and subgroup-specific effect estimates with confidence intervals.

Key take-home messages

References

1. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. Statistical principles for clinical trials E9. Geneva: ICH; 1998. https://database.ich.org/sites/default/files/E9_Guideline.pdf
2. Gamble C, Krishan A, Stocken D, Lewis S, Juszczak E, Doré C, et al. Guidelines for the content of statistical analysis plans in clinical trials. JAMA. 2017;318(23):2337–2343. https://doi.org/10.1001/jama.2017.18556
3. International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use. Addendum on estimands and sensitivity analysis in clinical trials to the guideline on statistical principles for clinical trials E9(R1). Geneva: ICH; 2019. https://database.ich.org/sites/default/files/E9-R1_Step4_Guideline_2019_1203.pdf
4. Vickers AJ, Altman DG. Analysing controlled trials with baseline and follow up measurements. BMJ. 2001;323(7321):1123–1124. https://doi.org/10.1136/bmj.323.7321.1123
5. Sterne JAC, White IR, Carlin JB, Spratt M, Royston P, Kenward MG, et al. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ. 2009;338:b2393. https://doi.org/10.1136/bmj.b2393
6. Chan AW, Boutron I, Hopewell S, Moher D, Schulz KF, Collins GS, et al. SPIRIT 2025 statement: updated guideline for protocols of randomised trials. BMJ. 2025;389:e081477. https://doi.org/10.1136/bmj-2024-081477
7. Hopewell S, Chan AW, Collins GS, Hróbjartsson A, Moher D, Schulz KF, et al. CONSORT 2025 statement: updated guideline for reporting randomised trials. BMJ. 2025;389:e081123. https://doi.org/10.1136/bmj-2024-081123