Common Errors in Sample Size Calculation

Overview

A sample-size calculation can be numerically correct yet scientifically wrong.

Common causes include:

• wrong research objective;
• wrong outcome or time point;
• wrong design formula;
• unjustified effect size;
• standard error entered as standard deviation;
• percentages entered on the wrong scale;
• missing clustering or pairing;
• incorrect attrition adjustment;
• rounding down;
• unreported software settings; and
• final analysis differing from the planning method.

The calculation should be reviewed as part of the study design, not as an isolated number in a protocol appendix.

Why this matters

Sample-size errors can produce:

• an underpowered or imprecise study;
• unnecessary recruitment;
• insufficient numbers in the outcome-relevant subgroup;
• an infeasible protocol;
• misleading ethics or funding claims;
• a calculation that cannot be reproduced; and
• a completed analysis answering a different question from the one planned.

Published and reviewed studies can contain substantial sample-size errors. Independent verification is therefore a legitimate scientific quality-control step [1].

Error 1: no primary estimand

A sample size cannot be appraised without knowing:

• population;
• groups or exposure;
• outcome;
• time point;
• effect measure;
• analysis population; and
• primary contrast.

The phrase “detect a significant difference” is insufficient.

A study may have several objectives, but the protocol should identify which one determines sample size or show that all key objectives are adequately addressed.

Error 2: calculation and design do not match

Examples include:

• two-group formula for a three-group study;
• independent formula for paired data;
• individual-level formula for cluster randomisation;
• mean formula for a binary outcome;
• fixed-time risk calculation for survival analysis;
• superiority formula for non-inferiority;
• sensitivity calculation based on total sample rather than condition-positive participants; and
• correlation formula for agreement.

The planned statistical analysis should be named before the formula is selected.

Error 3: effect size lacks clinical justification

A target difference should not be chosen because:

• it gives a convenient sample;
• it is labelled small, medium or large in a generic table;
• it was observed in one small pilot;
• it is the largest published effect;
• or it makes the study statistically significant in software.

The target should be important, plausible and expressed on the primary analysis scale.

Standardised effect sizes require a justified standard deviation and should be translated back into clinical units.

Error 4: wrong variability input

A common mistake is entering a standard error from a previous study as though it were a standard deviation.

For a sample mean:

$$SE=\frac{SD}{\sqrt{n}}$$

If a previous study reports $SE=2.0$ from $n=36$:

$$SD=2.0\sqrt{36}=12.0$$

Using 2.0 as the SD would severely underestimate the required sample.

Other errors include:

• using variance instead of SD;
• mixing change-score SD with follow-up SD;
• using baseline SD for paired differences;
• ignoring unequal group variances;
• and using a pilot SD without uncertainty.

Error 5: percentage and proportion confusion

Software may require proportions from 0 to 1.

Examples:

• 30% entered as 30 rather than 0.30;
• margin of error 5 percentage points entered as 5 rather than 0.05;
• risk reduction confused with percentage-point difference;
• dropout 15% entered as 0.15 in one field and 15 in another.

Every input should retain a label and unit in the calculation record.

Error 6: ambiguous one-sided testing

A one-sided calculation should not be selected only because it reduces sample size.

It may be justified when:

• only one direction can support the claim;
• an effect in the opposite direction would not lead to a different positive claim;
• the decision rule is prespecified; and
• regulatory and scientific standards support it.

Non-inferiority commonly uses one-sided type I error, but superiority trials usually retain two-sided inference.

Error 7: power and type I error are confused

Power is not:

• the probability that the alternative hypothesis is true;
• the probability that the final result will be clinically important;
• one minus the observed p-value; or
• evidence that a completed non-significant study had no effect.

The significance level and power must correspond to the stated one-sided or two-sided test.

Error 8: attrition is added incorrectly

If $n_A$ analysable participants are required and loss proportion is $L$:

$$n_{\mathrm{recruit}} = \frac{n_A}{1-L}$$

Adding $L\times100\%$ directly is insufficient.

Worked correction

Error 9: rounding down or rounding at the wrong stage

Sample sizes should be rounded upward.

For designs with:

• clusters;
• matched sets;
• allocation blocks;
• repeated batches; or
• group-specific targets;

rounding must preserve whole design units and required numbers per arm.

Rounding each intermediate value excessively can inflate the total unnecessarily, while rounding the final value down can violate the target.

Retain sufficient precision during calculation and round upward at the operational stage.

Error 10: dependence is ignored

Several observations from one participant are not independent participants.

Dependence occurs with:

• repeated visits;
• paired organs;
• multiple lesions;
• families;
• clinics;
• schools;
• matched pairs; and
• readers assessing the same images.

Ignoring positive correlation commonly overstates information and understates sample size.

The sample-size unit and analysis unit should be stated explicitly.

Error 11: only the total sample is reported

The reader may need:

• total participants;
• participants per group;
• number of clusters;
• mean cluster size;
• number of events;
• number with and without disease;
• number of complete pairs;
• number expected to provide the primary outcome; and
• number to screen.

For diagnostic and survival studies, the total alone may conceal the actual information target.

Error 12: pilot data are treated as truth

Small pilots produce uncertain estimates of:

• SD;
• event risk;
• ICC;
• recruitment;
• retention; and
• effect size.

The observed treatment effect from a pilot is especially unstable and often optimistic.

Pilot data should support sensitivity analysis, not one falsely precise input. Teare and colleagues found that estimating key nuisance parameters may require more pilot information than common ad hoc rules assume [2].

Error 13: multiplicity and subgroup claims are ignored

A sample powered for one primary comparison may be inadequate when success requires:

• several co-primary outcomes;
• multiple doses;
• several pairwise comparisons;
• interaction tests;
• or separate subgroup claims.

Multiplicity affects type I error, while interaction effects require their own precision or power.

“Subgroup analyses will be conducted if significant” is not a sample-size plan.

Error 14: software output is accepted without verification

Software may differ in:

• exact versus approximate distribution;
• continuity correction;
• pooled versus unpooled variance;
• one-sided versus two-sided alpha;
• allocation definition;
• attrition handling;
• test statistic;
• rounding; and
• default effect-size scale.

A reproducible record should include:

• software and version;
• procedure or function;
• all inputs;
• output before rounding;
• final rounding;
• and an independent cross-check.

Sample-size verification table

simulated data

Table 1. Review of the original protocol statement

Draft item	Problem	Required correction
“Standard error 2.0”	SE used as SD	Recover or justify the SD
“Effect size 0.5”	Scale and clinical meaning absent	State target in mmHg and justify
“95% power”	Possible but unexplained	State why 95% is required
“5% dropout”	Adjustment method absent	Divide by 0.95 after analysable sample
“100 participants”	Per group or total unclear	State allocation and group totals
Three groups later in protocol	Design mismatch	Recalculate for primary contrast or overall test
Binary outcome later	Outcome mismatch	Use binary-outcome assumptions
Clinic clustering	Independence violated	Use cluster-aware calculation
Sex subgroup	Interaction not planned	Define whether confirmatory and power separately
One-sided test	Direction unjustified	Use two-sided unless prospectively defensible

Error 15: post hoc observed power

After a study is complete, power calculated using the observed effect is largely a transformation of the p-value and adds no useful information about result precision.

Hoenig and Heisey described this as the abuse of power for data analysis [3].

A completed study should report:

• effect estimate;
• confidence interval;
• sample achieved;
• missing data;
• deviations from plan; and
• limitations.

A wide confidence interval shows directly which important effects remain compatible with the data.

Error 16: non-significance is called “underpowered” automatically

A non-significant result may arise because:

• the effect is near zero;
• the estimate is imprecise;
• the assumptions are wrong;
• the outcome is noisy;
• fewer events occurred;
• or the study suffered bias or missing data.

The confidence interval distinguishes effects that remain compatible with the data better than retrospective power.

A study can be adequately planned and still obtain a non-significant result.

Error 17: sample size is treated as a quality certificate

An adequately sized study can remain invalid because of:

• biased sampling;
• confounding;
• poor randomisation;
• measurement error;
• outcome switching;
• selective reporting;
• or inappropriate analysis.

Conversely, a constrained study may still provide useful information if its precision and limitations are reported honestly.

Bacchetti argued that rigid sample-size conventions can themselves lead to poor decisions and that value, cost and uncertainty should be considered explicitly [4].

Error 18: reporting is irreproducible

Inadequate wording:

The sample size was calculated using standard methods.

Reproducible wording identifies:

• objective;
• primary outcome;
• effect measure;
• target value;
• nuisance parameters;
• alpha and sidedness;
• power or precision;
• design;
• allocation;
• adjustments;
• software;
• and final target.

CONSORT and SPIRIT require transparent sample-size reporting for randomised trials [5], [6]. Broader methodological critiques also caution against treating conventional power thresholds or a single calculation as substitutes for a reasoned study justification [7], [8].

Prospective power versus retrospective interpretation MMed extension

Prospective power is useful for comparing designs under specified alternatives before data are collected.

After data are observed, the uncertainty about the effect is represented by the likelihood, confidence interval or posterior distribution, depending on the inferential framework.

Observed post hoc power does not rescue interpretation and should not replace direct uncertainty estimates.

Suggested reporting

Common mistakes

Knowledge check

Why is using a previous SE as the planning SD dangerous?

Answer and reasoning

The SE is smaller than the SD by a factor related to the previous sample size. Using it as the SD can dramatically understate outcome variability and required sample size.

What should be done when the protocol changes from a numerical to a binary primary outcome?

Answer and reasoning

Recalculate using assumptions appropriate to the binary effect measure and both group event risks. The original mean-based calculation no longer supports the design.

Why is observed post hoc power unhelpful?

Answer and reasoning

When computed from the observed effect and standard error, it largely restates the result’s p-value. The effect estimate and confidence interval show the magnitude and uncertainty directly.

What is wrong with adding 15% to an analysable sample of 122?

Answer and reasoning

Losing 15% of the inflated number leaves fewer than 122 on average. The correct calculation divides 122 by the 0.85 retention proportion.

Can a correctly powered study still be invalid?

Answer and reasoning

Yes. Power does not correct bias, confounding, measurement error, missing-data mechanisms or an inappropriate analysis.

Key take-home messages

References

1. Shuster JJ. Sample size verification for clinical trials. Clin Transl Sci. 2014;7(1):60–62. https://doi.org/10.1111/cts.12115
2. Teare MD, Dimairo M, Shephard N, Hayman A, Whitehead A, Walters SJ. Sample size requirements to estimate key design parameters from external pilot randomised controlled trials: a simulation study. Trials. 2014;15:264. https://doi.org/10.1186/1745-6215-15-264
3. Hoenig JM, Heisey DM. The abuse of power: the pervasive fallacy of power calculations for data analysis. Am Stat. 2001;55(1):19–24. https://doi.org/10.1198/000313001300339897
4. Bacchetti P. Current sample size conventions: flaws, harms, and alternatives. BMC Med. 2010;8:17. https://doi.org/10.1186/1741-7015-8-17
5. Chan AW, Boutron I, Hopewell S, Moher D, Schulz KF, Collins GS, et al. SPIRIT 2025 statement: updated guideline for protocols of randomised trials. BMJ. 2025;389:e081477. https://doi.org/10.1136/bmj-2024-081477
6. Hopewell S, Chan AW, Collins GS, Hróbjartsson A, Moher D, Schulz KF, et al. CONSORT 2025 statement: updated guideline for reporting randomised trials. BMJ. 2025;389:e081123. https://doi.org/10.1136/bmj-2024-081123
7. Schulz KF, Grimes DA. Sample size calculations in randomised trials: mandatory and mystical. Lancet. 2005;365(9467):1348–1353. https://doi.org/10.1016/S0140-6736(05)61034-3
8. Lakens D. Sample size justification. Collabra Psychol. 2022;8(1):33267. https://doi.org/10.1525/collabra.33267