Parametric and Non-Parametric Methods

Overview

Statistical methods are often divided into parametric and non-parametric families.

A parametric method specifies a probability model described by a finite set of parameters. Examples include a normal model described by a mean and variance, and linear regression described by regression coefficients and an error variance.

A non-parametric method makes fewer assumptions about a specific distributional form. Rank-based procedures, permutation tests and empirical distribution methods are common examples.

The distinction does not mean:

• parametric methods always require perfectly normal raw data;
• non-parametric methods require no assumptions;
• parametric methods are always more powerful; or
• non-parametric methods automatically compare medians.

Method selection should begin with the research question and estimand—the population quantity the study intends to estimate.

Why this matters

A common decision rule is:

If the normality test is significant, use a non-parametric test.

This rule is inadequate.

Formal normality tests can detect trivial departures in large samples and miss important departures in small samples. Many parametric procedures are reasonably robust to moderate non-normality when the design, sample size and variance estimation are appropriate [1]. Conversely, rank-based methods may answer a different question from the mean difference that clinicians intended to estimate [2].

Changing methods solely to obtain a favourable p-value can change the estimand and weaken interpretability.

What parametric methods assume

A parametric analysis specifies a model for the outcome or estimator.

For a two-group numerical outcome, a linear model may assume:

• independent observations or correctly modelled dependence;
• a linear mean structure;
• residuals with a sufficiently appropriate distribution for the chosen inference;
• a suitable variance model; and
• no single observation dominating the estimate.

The raw outcome need not be exactly normally distributed in every group. The relevant assumptions often concern the model residuals, standard errors and sampling distribution of the estimator.

Examples of parametric methods include:

• Student’s and Welch’s t-tests;
• analysis of variance;
• linear regression;
• logistic regression;
• Poisson regression; and
• Cox proportional-hazards regression.

These methods differ substantially. “Parametric” is not a single assumption set.

What non-parametric methods assume

Non-parametric methods avoid specifying every feature of a particular distribution, but they still require assumptions.

A Wilcoxon–Mann–Whitney analysis commonly assumes:

• independent observations between groups;
• an ordinal or numerical outcome that can be ranked;
• comparable measurement across groups; and
• a prespecified interpretation of the distributional contrast.

A permutation test requires exchangeability under the null: the labels can be reassigned according to the randomisation or sampling structure without changing the joint distribution under that null.

Rank methods can be affected by:

• unequal distribution shapes;
• unequal spread;
• many tied values;
• clustering;
• covariate imbalance; and
• missing data.

Mean comparison using Welch’s t-test

Welch’s independent-samples t-test estimates a difference in population means and does not require equal population variances. It is usually preferable to the equal-variance Student procedure when variances or sample sizes may differ [3].

The estimand is:

$$\Delta_\mu = \mu_1-\mu_0$$

where:

• $\mu_1$ is the population mean in the first group; and
• $\mu_0$ is the population mean in the comparator group.

For length of stay, the mean has a direct resource interpretation because total bed-days equal the number of admissions multiplied by mean stay.

A right-skewed distribution does not make the population mean irrelevant. It may, however, require robust standard errors, a suitable generalised model, bootstrap inference or careful influence assessment.

Distributional comparison using Wilcoxon–Mann–Whitney

The Wilcoxon–Mann–Whitney procedure uses ranks to compare two independent outcome distributions.

One useful estimand for continuous outcomes without ties is the probability:

$$P(Y_1<Y_0)$$

where a randomly selected outcome from group 1 is compared with a randomly selected outcome from group 0.

With ties, a commonly reported probability-of-superiority measure assigns half weight to ties.

The procedure can reflect differences in location, spread or shape. It is not automatically a test of medians. A median-shift interpretation requires additional conditions, such as similarly shaped distributions differing primarily by location [2], [4]. Direct alignment of method and clinical estimand is a longstanding principle of medical-statistical practice [5].

The test may reject even when group medians are equal, and it may fail to represent the clinical mean-difference question.

Data structure

Simulated pneumonia length-of-stay summaries

simulated data

Table 1. Hospital length of stay by care pathway

Care pathway	Participants	Mean (SD)	Median (Q1–Q3)	Range
Early specialist review	120	7.8 (8.6) days	5 (3–9) days	1–54 days
Usual pathway	120	10.2 (12.4) days	6 (4–11) days	1–76 days

Note: Q1–Q3 gives the first and third quartiles; SD = standard deviation. Values are simulated.

The group distributions overlap substantially and both have long right tails. The usual-pathway group also has greater spread.

Worked example

Choosing a method by the estimand

When the mean matters

A mean-based method is often appropriate when the question concerns:

• average blood pressure;
• mean change in a laboratory value;
• average cost;
• mean resource use;
• mean treatment effect; or
• a regression coefficient on the original scale.

Possible approaches for skewed data include:

• Welch’s t-test;
• linear regression with robust standard errors;
• bootstrap confidence intervals under a suitable resampling design;
• generalised linear models; or
• transformation with an explicitly changed interpretation.

When ranks or quantiles matter

Rank or quantile methods may be useful when:

• the outcome is ordinal;
• the ordering is meaningful but distances are not;
• extreme values make a mean-based estimand unsuitable for the question;
• a stochastic-ordering contrast is of interest; or
• the objective is a median or another quantile and a direct quantile method is used.

If the objective is a median difference, use a method that estimates that quantity, such as quantile regression or a suitable confidence interval for the median contrast. Do not assume the Mann–Whitney test supplies it.

Paired and repeated observations

For paired numerical data:

• paired t-test estimates the mean within-pair difference;
• Wilcoxon signed-rank procedure uses ranks of non-zero absolute differences and assumes a suitable symmetry condition for common location interpretations;
• sign test uses only the direction of paired differences and can target the median difference under appropriate conditions.

These are not substitutes for independent-samples procedures.

Repeated measurements across several visits require methods that account for within-person correlation. Applying a separate Mann–Whitney test at every time point creates multiplicity and ignores the longitudinal structure.

Transformations

A logarithmic transformation can be useful for positive right-skewed outcomes when ratios or multiplicative effects are scientifically meaningful.

A difference in mean log outcomes typically becomes a ratio of geometric means after exponentiation. It does not become a median difference automatically.

Transformation should be chosen for the scientific model and interpretation, not solely to obtain a non-significant normality test.

Permutation and bootstrap methods

A permutation test obtains a null distribution by rearranging labels according to the design. In a randomised trial, the permutation scheme should reflect the randomisation.

A bootstrap repeatedly resamples observations or clusters to approximate the sampling distribution of an estimator.

These methods are not assumption-free:

• the resampling unit must match independence;
• cluster or paired structure must be preserved;
• small samples may give unstable results;
• the estimator must be suitable; and
• bias or poor design is not repaired by resampling.

Robustness is method-specific MMed extension

Large-sample mean comparisons can remain valid under substantial non-normality when observations are independent and no small number of values dominates the estimate [1].

Robustness does not mean every violation is harmless. Important concerns include:

• severe imbalance with very different variances;
• small samples with heavy tails;
• influential observations;
• cluster dependence;
• informative missingness; and
• model misspecification.

Welch’s method, robust standard errors, transformation and bootstrap procedures address different problems. They should not be treated as one general correction.

Suggested reporting

Common mistakes

Knowledge check

Does a significant normality test require a non-parametric analysis?

Answer and reasoning

No. The choice depends on the estimand, sample size, design, influence of unusual observations and assumptions of the intended model. A normality test alone cannot select the correct method.

What does Welch's t-test estimate in the pneumonia example?

Answer and reasoning

It estimates the difference in population mean length of stay between pathways. The clinical interpretation concerns average bed-days, not ranks or medians.

Why is the Mann–Whitney procedure not automatically a test of medians?

Answer and reasoning

It compares relative ranks or distributions. Differences in spread or shape can affect the statistic even when medians are equal. A median interpretation requires additional assumptions or a direct quantile method.

Are permutation and bootstrap methods assumption-free?

Answer and reasoning

No. They require a valid resampling scheme, the correct independent unit and an estimator suited to the data. They do not remove bias or repair an inappropriate design.

Why might a mean remain important for a right-skewed length-of-stay outcome?

Answer and reasoning

The arithmetic mean determines expected total bed use. Skewness affects analysis and presentation, but it does not make the mean irrelevant when resource use is the estimand.

Key take-home messages

References

1. Lumley T, Diehr P, Emerson S, Chen L. The importance of the normality assumption in large public health data sets. Annu Rev Public Health. 2002;23:151–169. https://doi.org/10.1146/annurev.publhealth.23.100901.140546
2. Fay MP, Proschan MA. Wilcoxon–Mann–Whitney or t-test? On assumptions for hypothesis tests and multiple interpretations of decision rules. Stat Surv. 2010;4:1–39. https://doi.org/10.1214/09-SS051
3. Welch BL. The generalization of Student’s problem when several different population variances are involved. Biometrika. 1947;34(1–2):28–35. https://doi.org/10.1093/biomet/34.1-2.28
4. Divine GW, Norton HJ, Barón AE, Juarez-Colunga E. The Wilcoxon–Mann–Whitney procedure fails as a test of medians. Am Stat. 2018;72(3):278–286. https://doi.org/10.1080/00031305.2017.1305291
5. Altman DG. Practical statistics for medical research. London: Chapman and Hall; 1991.