Superiority, Non-Inferiority and Equivalence Trials

Overview

A superiority trial asks whether a new treatment is better.

A non-inferiority trial asks whether the new treatment is not unacceptably worse than an effective control.

An equivalence trial asks whether the treatments differ by no more than a prespecified amount in either direction.

The margin is a clinical and scientific decision, not a number chosen from the current data.

Why this matters

A non-significant superiority test does not establish non-inferiority.

Poor adherence, insensitive outcomes and weak trial conduct can make treatments appear similar and falsely support non-inferiority.

For an active-control non-inferiority or equivalence comparison, the active control must have reliable evidence of benefit in a setting sufficiently similar to the current trial.

Superiority trials

A superiority trial is designed to determine whether one treatment produces a better outcome than its comparator. The comparator may be placebo, usual care or an active treatment. The direction of benefit must be defined before the trial begins because higher values are desirable for some outcomes, whereas lower values are desirable for others.

Hypotheses

Let $\Delta=\mu_{\text{new}}-\mu_{\text{control}}$, with higher values indicating benefit. A directional superiority test may be written as:

The null hypothesis is composite: it includes no benefit and effects in the unfavourable direction. It should not be described as a “two-sided null”. One-sided refers to the alternative hypothesis and rejection region.

Many confirmatory superiority trials instead use the conventional two-sided formulation:

With this formulation, superiority is concluded only when the test is statistically significant and the estimated effect favours the new treatment. The choice of one- or two-sided inference must be prespecified. A one-sided test requires prospective justification, particularly because an important effect in the harmful direction must not be ignored [1].

Zero is the null value for a difference. For ratio measures, such as a risk ratio or hazard ratio, the corresponding null value is 1. The inequalities reverse when lower outcome values indicate benefit.

Although the statistical alternative is two-sided, the estimated effect must favour the new treatment before superiority can be claimed. A one-sided test should be used only when it was prospectively justified and an important effect in the opposite direction would not be ignored.

Confidence-interval interpretation

Superiority is supported when the confidence interval excludes the null value and lies entirely in the prespecified direction of benefit. If a two-sided 95% confidence interval for a mean difference is entirely above zero when higher values are better, the result supports superiority at the two-sided 5% significance level.

Under the corresponding standard symmetric test, this decision boundary is also equivalent to a directional one-sided test at $\alpha=0.025$. It is not equivalent to a one-sided test at $\alpha=0.05$, which has a less stringent boundary corresponding to a two-sided 90% confidence interval.

An interval that crosses zero does not demonstrate that the treatments are equivalent or equally effective. It means that the data remain compatible with effects on both sides of the null value within the range described by the interval.

Analysis population

The primary superiority analysis usually follows the prespecified intention-to-treat strategy or, more precisely, the analysis set and estimand defined in the protocol. Retaining participants in their randomised groups preserves the protection provided by randomisation and estimates the effect of assignment under the chosen strategy for intercurrent events [2].

A per-protocol analysis may be informative as a sensitivity or supplementary analysis, but excluding participants after randomisation can introduce bias. It should not replace the primary superiority analysis merely because it produces a larger treatment effect.

Statistical and clinical superiority

Statistical superiority does not automatically imply a clinically important advantage. The confidence interval should be compared with the smallest effect considered clinically meaningful, and the interpretation should also consider harms, burden, costs and outcome relevance.

For example, a very precise improvement of 0.4 points may exclude zero but still be too small to influence practice if clinicians and patients consider a 2-point improvement necessary. Conversely, an imprecise estimate may include a clinically important benefit even though statistical superiority has not been established.

When to use these designs

Use superiority when the primary objective is to establish that one treatment produces a better outcome than its comparator. The comparator may be placebo, usual care or an active treatment, depending on the clinical and ethical context.

Use non-inferiority when a new treatment may offer:

• fewer harms;
• lower cost;
• easier administration;
• improved access;
• better adherence;
• other practical advantages.

Use equivalence when interchangeability within a symmetric range is the objective.

The design should not be used simply because superiority is difficult to show.

Research question and variables

If higher values are better and $\Delta$ is new minus control:

Non-inferiority hypotheses are:

Equivalence hypotheses for margins $-\delta$ and $+\delta$ are:

$$H_0:\Delta\le-\delta \quad\text{or}\quad \Delta\ge+\delta$$ $$H_1:-\delta<\Delta<+\delta$$

The confidence level must match the prespecified Type I error. In this lecture, a two-sided 95% confidence interval is used for every comparison. For non-inferiority, this corresponds to a one-sided test at $\alpha=0.025$. For equivalence, requiring the same 95% interval to lie entirely within both margins corresponds to two one-sided tests, each at $\alpha=0.025$. A TOST procedure using $\alpha=0.05$ for each one-sided test would instead use a two-sided 90% confidence interval.

Data structure

Margin-based interpretation

simulated data

Quantity	Value
Estimate	−0.60
95% CI	−2.27 to 1.07
NI margin	−3.00
Equivalence interval	−2.00 to 2.00

The sign convention must be stated clearly.

Statistical assumptions

Margin selection

A defensible margin uses:

• historical active-control effect;
• uncertainty in that effect;
• preservation fraction;
• clinical judgement;
• regulatory context.

The margin should not exceed the active control’s credible benefit over placebo.

Worked example

Worked analysis in R

estimate <- -0.60
standard_error <- 0.85
# A two-sided 95% CI corresponds to one-sided alpha = 0.025
confidence_interval <- estimate +
  c(-1, 1) *
  qnorm(0.975) *
  standard_error

noninferiority_margin <- -3
equivalence_margin <- c(-2, 2)

noninferior <- (
  confidence_interval[1] >
  noninferiority_margin
)

equivalent <- (
  confidence_interval[1] >
  equivalence_margin[1]
) & (
  confidence_interval[2] <
  equivalence_margin[2]
)

superior <- (
  confidence_interval[1] > 0
)

c(
  estimate = estimate,
  lower = confidence_interval[1],
  upper = confidence_interval[2],
  noninferior = noninferior,
  equivalent = equivalent,
  superior = superior
)

Selected analysis output

Estimate = -0.600
95% CI = -2.266 to 1.066
Non-inferior = TRUE
Equivalent = FALSE
Superior = FALSE

Visualising trial conclusions

ITT and PP

In superiority trials, non-adherence often dilutes effects towards no difference.

In non-inferiority trials, dilution can falsely support similarity.

The primary estimand and analysis set must be prespecified. In non-inferiority trials, appropriately defined full-analysis-set and PP analyses commonly provide complementary evidence:

• an intention-to-treat or treatment-policy analysis preserves randomised group assignment;
• a PP analysis may be more sensitive to treatment differences but can be biased by post-randomisation exclusions;
• concordant conclusions strengthen confidence, but concordance does not remove biases shared by both analyses;
• materially different conclusions require investigation rather than selecting the more favourable result.

Assay sensitivity

Assay sensitivity is the ability of the trial to distinguish an effective treatment from a less effective or ineffective one.

It depends on:

• patient population;
• outcome measurement;
• adherence;
• active-control dose;
• follow-up;
• trial quality.

A poorly conducted trial can show apparent non-inferiority of two ineffective strategies.

Interpreting the output

Presenting the results

Suggested reporting

Use the CONSORT extension for non-inferiority and equivalence trials [3].

Report margin justification, effect direction, confidence interval, ITT and PP results, active-control evidence and assay sensitivity.

Regulatory and methodological guidance provides complementary principles for selecting margins, preserving assay sensitivity and interpreting non-inferiority [4], [5], [6].

Common mistakes

Knowledge check

How can a directional superiority hypothesis be written when higher values are better?

Answer and reasoning

With $\Delta$ defined as new treatment minus control, use $H_0:\Delta\le0$ versus $H_1:\Delta>0$.

What one-sided significance level corresponds to requiring a two-sided 95% confidence interval to lie above zero?

Answer and reasoning

Under the corresponding standard symmetric test, it is a one-sided significance level of 0.025.

What establishes non-inferiority here?

Answer and reasoning

The lower confidence limit lies above the prespecified margin of −3.

Why is non-significant superiority insufficient?

Answer and reasoning

Failure to reject no difference can result from imprecision or poor assay sensitivity.

Why analyse both ITT and PP?

Answer and reasoning

Different biases can favour similarity, and concordant results strengthen the conclusion.

What is assay sensitivity?

Answer and reasoning

The ability of the trial to distinguish effective from less effective treatment.

Why is equivalence not established?

Answer and reasoning

The confidence interval is not fully contained within −2 to +2.

Key take-home messages

References

1. International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use. ICH E9: statistical principles for clinical trials. Geneva: ICH; 1998. https://www.fda.gov/media/71336/download
2. International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use. ICH E9(R1): addendum on estimands and sensitivity analysis in clinical trials. Geneva: ICH; 2019. https://database.ich.org/sites/default/files/E9-R1_Step4_Guideline_2019_1203.pdf
3. Piaggio G, Elbourne DR, Pocock SJ, Evans SJW, Altman DG. Reporting of noninferiority and equivalence randomized trials. JAMA. 2012;308(24):2594–2604. https://doi.org/10.1001/jama.2012.87802
4. US Food and Drug Administration. Non-inferiority clinical trials to establish effectiveness. Silver Spring: FDA; 2016. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/non-inferiority-clinical-trials
5. European Medicines Agency. Guideline on the choice of the non-inferiority margin. London: EMA; 2005. https://www.ema.europa.eu/en/non-inferiority-equivalence-comparisons-clinical-trials-scientific-guideline-1
6. Schumi J, Wittes JT. Through the looking glass: understanding non-inferiority. Trials. 2011;12:106. https://doi.org/10.1186/1745-6215-12-106