Sample Size for Comparing Groups

Overview

Comparative studies ask whether outcomes differ between treatments, exposures, groups or occasions.

The required sample depends on:

• outcome type;
• independent or paired structure;
• number of groups;
• effect measure;
• clinically meaningful difference;
• variability or event risks;
• significance level;
• target power;
• allocation ratio;
• adjustment variables; and
• missingness.

The sample-size method must match the planned primary analysis. A calculation for two independent means is not suitable for paired measurements, cluster randomisation or a binary outcome. Established clinical-research texts provide method-specific calculations for these different structures [1], [2].

Why this matters

Different designs can compare the same clinical outcome but contain different information.

A before-and-after study of 80 participants contains 80 paired differences, not 160 independent observations.

A three-group trial cannot be planned accurately using a two-group formula if the primary objective is an overall analysis of variance or multiple pairwise claims.

A binary outcome with control risk 2% requires a different sample from one with control risk 40%, even when the relative effect is the same.

A comparative sample size therefore begins with the estimand, not the name of a statistical test. Transparent justification of the target difference and other assumptions is essential because apparently small planning choices can materially alter recruitment [3].

Independent numerical outcomes

For two independent groups with equal allocation, a common planning standard deviation and a mean-difference estimand:

The standardised difference is $\Delta/\sigma$, but planning should retain clinical units whenever possible.

Worked calculation

A noncentral t-based calculation may differ slightly. The software procedure and distributional assumptions should be reported.

Baseline adjustment

Analysis of covariance can improve precision by adjusting for a prognostic baseline measurement.

Under idealised assumptions, the residual variance may be approximately $\sigma_Y^2(1-\rho^2)$, where $\rho$ is the correlation between baseline and follow-up outcome.

Reducing sample size based on anticipated baseline adjustment requires:

• reliable prior evidence for the correlation;
• the same adjustment in the final analysis;
• allowance for uncertainty;
• correct modelling of baseline; and
• no double counting of precision gains.

A conservative unadjusted calculation is often used when the correlation is uncertain. When baseline adjustment is incorporated, the final analysis and its assumed baseline–follow-up relationship should match the planning model [4].

Unequal allocation

For two independent groups with similar variance and cost, equal allocation is most efficient.

If the allocation ratio is $r=n_1/n_0$, an approximate inflation relative to equal allocation is:

At a 2:1 ratio:

$$IF_{\mathrm{alloc}}=\frac{3^2}{8}=1.125$$

The total sample is approximately 12.5% larger than with equal allocation for the same power under equal variance.

The allocation ratio should be incorporated directly in suitable software rather than applied mechanically when variances, costs or event risks differ.

Paired numerical outcomes

Paired designs include:

• before-and-after measurements;
• matched participants;
• crossover comparisons; and
• two measurements on the same individual.

The primary information is the within-pair difference.

If $\sigma_D=10$ points, $\Delta_D=4$ points, two-sided $\alpha=0.05$ and power 90%:

$$n_{\mathrm{pairs}} = 65.67$$

(The arithmetic producing 65.67 pairs from the displayed formula and inputs is not shown here.)

Round upward to 66 complete pairs.

Inflate for loss of either measurement, not for the number of individual observations.

The standard deviation of baseline values cannot be substituted automatically for the standard deviation of paired differences.

Comparing two proportions

Let $p_1$ and $p_0$ be the anticipated risks in two independent groups.

A common equal-allocation large-sample approximation is:

Suppose the programme is expected to reduce 12-week treatment failure from 20% to 10%.

With two-sided $\alpha=0.05$ and 80% power:

$$n=198.96$$

(The arithmetic producing 198.96 participants per group from the displayed formula and inputs is not shown here.)

Round upward to 199 participants per group before attrition adjustment.

The calculation targets an absolute risk difference of 10 percentage points and a risk ratio of 0.50. Both group risks should be reported because a relative measure alone does not determine sample size.

Binary paired outcomes

For paired binary data, sample size depends on the discordant pairs:

• outcome changes from no to yes;
• outcome changes from yes to no.

McNemar-type planning requires anticipated discordant proportions, not only the marginal percentages before and after.

Ignoring pairing can overestimate or underestimate the required sample.

More than two groups

For three or more independent groups, planning depends on the primary claim:

• overall analysis of variance;
• one prespecified contrast;
• multiple pairwise comparisons;
• ordered trend;
• factorial main effects or interaction;
• non-inferiority against a common control.

The significance level may need allocation across multiple confirmatory comparisons.

Cohen’s $f$ is often used for one-way analysis of variance, but clinically interpretable group means and standard deviation should still be documented.

Planning inputs by comparison structure

simulated data

Table 1. Examples of required inputs

Comparison	Primary effect	Important nuisance input
Two independent means	Mean difference	Group standard deviations
Paired means	Mean paired difference	SD of paired differences
Two independent proportions	Risk difference or ratio	Both group risks
Paired proportions	Difference in marginal proportions	Discordant-pair probabilities
Three means	Overall or contrast effect	Group means and residual SD
Repeated outcomes	Time-specific or trajectory contrast	Within-person covariance
Cluster-randomised groups	Cluster-aware treatment effect	ICC and cluster-size distribution

Note: ICC = intracluster correlation coefficient; SD = standard deviation.

Superiority, non-inferiority and equivalence

Superiority

A superiority study asks whether one intervention differs from or is better than another on a prespecified scale.

Non-inferiority

A non-inferiority study asks whether a new intervention is not worse than the comparator by more than a prespecified non-inferiority margin.

The margin must be clinically and historically justified. It is not the expected treatment difference.

Sample-size planning must address:

• one-sided type I error;
• expected true difference;
• non-inferiority margin;
• outcome variability or event risks;
• analysis population; and
• preservation of the comparator effect.

Equivalence

An equivalence study asks whether the treatment effect lies within both a lower and upper equivalence margin.

Equivalence is not established by a non-significant superiority test.

Part XIII, Module 6 develops these designs. Active-control non-inferiority and equivalence designs require explicit margins and preservation assumptions [5].

Assumptions and sensitivity analysis

Simulation for complex comparisons MMed extension

Simulation is useful when the study includes:

• repeated outcomes with complex covariance;
• non-linear models;
• adaptive designs;
• multiple primary endpoints;
• unequal cluster sizes;
• treatment-by-time interactions;
• small numbers of clusters; or
• missing-data procedures built into the primary analysis.

A simulation-based calculation should report:

• data-generating model;
• parameter values;
• number of simulation runs;
• analysis fitted to each dataset;
• success criterion;
• Monte Carlo error; and
• code or software version.

Suggested reporting

Common mistakes

Knowledge check

Why does a paired study require the standard deviation of the differences?

Answer and reasoning

The paired analysis is based on one difference per pair. Its precision depends on variability of those differences, which incorporates the within-pair correlation.

Why are both event probabilities required for comparing two proportions?

Answer and reasoning

The variance depends on each group’s risk, not only the absolute or relative effect. Different baseline risks can produce different sample sizes for the same risk ratio.

What is the effect of 2:1 allocation under equal variances?

Answer and reasoning

It generally increases the total sample relative to 1:1 allocation. Under the simple approximation, the inflation factor is 1.125.

Why can a failed superiority test not establish equivalence?

Answer and reasoning

The result may reflect insufficient precision. Equivalence requires prespecified margins and confidence-interval evidence that the entire effect lies within them.

Why should the sample-size method match the final analysis?

Answer and reasoning

Different analyses have different variance and information structures. A mismatch can produce a study that does not achieve its intended precision or power.

Key take-home messages

References

1. Chow SC, Shao J, Wang H, Lokhnygina Y. Sample size calculations in clinical research. 3rd ed. Boca Raton (FL): CRC Press; 2018.
2. Machin D, Campbell MJ, Tan SB, Tan SH. Sample sizes for clinical, laboratory and epidemiology studies. 4th ed. Hoboken (NJ): Wiley; 2018. https://doi.org/10.1002/9781118874905
3. Schulz KF, Grimes DA. Sample size calculations in randomised trials: mandatory and mystical. Lancet. 2005;365(9467):1348–1353. https://doi.org/10.1016/S0140-6736(05)61034-3
4. Vickers AJ, Altman DG. Analysing controlled trials with baseline and follow up measurements. BMJ. 2001;323(7321):1123–1124. https://doi.org/10.1136/bmj.323.7321.1123
5. International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use. Choice of control group and related issues in clinical trials E10. Geneva: ICH; 2000. https://database.ich.org/sites/default/files/E10_Guideline.pdf
6. Hopewell S, Chan AW, Collins GS, Hróbjartsson A, Moher D, Schulz KF, et al. CONSORT 2025 statement: updated guideline for reporting randomised trials. BMJ. 2025;389:e081123. https://doi.org/10.1136/bmj-2024-081123