Non-Parametric Analysis of Numerical or Ordinal Outcomes

Overview

Non-parametric methods do not require the same fully specified distributional model as conventional normal-theory methods. Many common procedures replace observed magnitudes with ranks.

They are useful for:

• ordinal outcomes;
• numerical outcomes with a rank-based estimand;
• small samples with problematic outliers;
• or settings where a mean-based model is not scientifically appropriate.

They are not assumption-free and are not automatic substitutes whenever a normality test is significant.

Common methods are:

• Mann–Whitney or Wilcoxon rank-sum test for two independent groups;
• Wilcoxon signed-rank test for paired observations;
• Kruskal–Wallis test for three or more independent groups;
• Friedman test for repeated or blocked observations.

Why this matters

The statement:

A Mann–Whitney test compared medians.

is often unjustified.

The Mann–Whitney method compares the relative ranks of observations from two groups. Under additional assumptions, such as distributions differing mainly by a location shift, it can support a location or median interpretation. When shapes differ, the test may respond to spread or other distributional differences.

Similarly, the Wilcoxon signed-rank test assumes a symmetric distribution of paired differences for a simple location interpretation.

The method should be selected from the estimand and data structure, not from a normality-test decision tree [1].

Choosing the method

Rank-based method by study structure

simulated data

Structure	Method	Main null concept
Two independent groups	Mann–Whitney rank-sum	Equal rank distributions under the null
Two paired measurements	Wilcoxon signed-rank	Difference distribution centred on the null under symmetry
Three or more independent groups	Kruskal–Wallis	Equal rank distributions across groups
Three or more repeated conditions	Friedman	Equal condition rank distributions within blocks

If the research question concerns:

• an arithmetic mean, use an appropriate mean-based method;
• a median difference, consider quantile regression;
• an ordered category effect with covariates, consider ordinal regression;
• a full distributional difference, state that explicitly.

Statistical assumptions

Mann–Whitney rank-sum test

The Mann–Whitney method compares two independent groups [2].

A useful probabilistic interpretation under continuity is related to:

$$P(X<Y)+\frac{1}{2}P(X=Y)$$

It is not automatically a test of equal medians.

Worked example 1

group_a <- c(2.1,2.4,2.8,3.0,3.2,3.5,3.7,4.0,4.2,4.4,4.8,5.1,5.3,5.8,6.2)
group_b <- c(3.1,3.6,3.9,4.3,4.6,4.9,5.2,5.5,5.7,6.0,6.4,6.7,7.0,7.4,7.8)

wilcox.test(
  group_a,
  group_b,
  paired = FALSE,
  exact = FALSE,
  correct = FALSE
)

Wilcoxon rank sum test
W = 51, p-value = 0.01074
alternative hypothesis: true location shift is not equal to 0

The wording printed by software should not be interpreted more strongly than the design and distribution support.

Wilcoxon signed-rank test

The signed-rank method analyses paired differences and ranks their absolute magnitudes while retaining signs [3].

Zero differences are omitted. Ties affect the reference distribution.

Worked example 2

before <- c(8.2,7.5,6.9,8.8,7.9,6.5,9.1,7.2,8.0,6.8,7.7,8.5,7.0,9.0,6.7,8.3,7.4,8.6)
after  <- c(6.9,6.8,6.1,7.1,7.2,5.9,7.8,6.6,7.1,6.0,6.8,7.4,6.4,7.7,6.0,7.2,6.7,7.5)

wilcox.test(
  after,
  before,
  paired = TRUE,
  exact = FALSE,
  correct = FALSE
)

Wilcoxon signed rank test
V = 0, p-value = 0.000192
alternative hypothesis: true location shift is not equal to 0

Kruskal–Wallis test

The Kruskal–Wallis method extends the rank-sum idea to three or more independent groups [4].

A significant result means at least one rank distribution differs. It does not identify the groups.

Worked example 3

recovery <- data.frame(
  programme = factor(rep(c("Programme 1", "Programme 2", "Programme 3"), each = 12)),
  days = c(
    12.1,13.5,14.0,14.8,15.2,15.7,16.1,16.6,17.0,17.4,18.0,18.5,
    14.2,15.0,15.8,16.4,17.1,17.8,18.3,18.9,19.5,20.1,20.7,21.2,
    16.0,17.2,18.1,19.0,19.8,20.5,21.3,22.0,22.8,23.5,24.1,25.0
  )
)

kruskal.test(days ~ programme, data = recovery)
pairwise.wilcox.test(
  recovery$days,
  recovery$programme,
  p.adjust.method = "holm",
  exact = FALSE
)

Kruskal-Wallis rank sum test
Kruskal-Wallis chi-squared = 15.479, df = 2, p-value = 0.0004353

Friedman test

The Friedman test compares repeated conditions or matched blocks using within-block ranks [5].

It is a rank-based alternative for one-factor repeated measurements.

Worked example 4

ratings <- data.frame(
  participant_id = factor(1:15),
  A = c(4,5,3,4,4,5,3,4,5,4,3,4,5,4,3),
  B = c(3,4,3,3,4,4,2,3,4,3,3,3,4,3,2),
  C = c(2,3,2,3,3,3,1,2,3,2,2,2,3,2,1)
)

friedman.test(as.matrix(ratings[c("A", "B", "C")]))

Friedman rank sum test
Friedman chi-squared = 28.107, df = 2, p-value = 7.882e-07

Reproducibility checks

mw <- wilcox.test(group_a, group_b, exact = FALSE, correct = FALSE)
ws <- wilcox.test(after, before, paired = TRUE, exact = FALSE, correct = FALSE)
kw <- kruskal.test(days ~ programme, data = recovery)
fr <- friedman.test(as.matrix(ratings[c("A", "B", "C")]))

Interpreting the output

Presenting the results

A compact result may state:

Pain scores were generally lower in Group A than Group B (median 4.0, Q1–Q3 3.1–5.0 versus 5.5, Q1–Q3 4.5–6.6; Wilcoxon rank-sum $W=51$, $p=0.011$).

The report should identify whether the analysis used:

• exact or asymptotic inference;
• continuity correction;
• tie handling;
• and multiplicity adjustment.

Suggested reporting

Alternatives that target a clearer clinical quantity MMed extension

Consider:

• quantile regression for a median or other quantile difference;
• ordinal logistic regression for ordered categories and covariate adjustment;
• robust or bootstrap methods for a population mean;
• permutation tests for design-based hypotheses;
• generalised estimating equations for repeated ordinal outcomes;
• probabilistic-index models for rank probabilities.

“Non-parametric” should not be used as a substitute for defining the estimand.

Common mistakes

Knowledge check

Does a Mann–Whitney result always compare medians?

Answer and reasoning

No. It compares rank distributions. A median or location-shift interpretation requires additional assumptions.

Which method is appropriate for paired ordinal measurements at two times?

Answer and reasoning

The Wilcoxon signed-rank test may be appropriate when its symmetry-related rank estimand fits the question. A sign test is another option with different assumptions.

What does a significant Kruskal–Wallis result establish?

Answer and reasoning

At least one group rank distribution differs. It does not identify the differing pair or prove every group differs.

Why must repeated ordinal ratings use Friedman rather than Kruskal–Wallis?

Answer and reasoning

The same participants contribute ratings under each condition. Friedman preserves the within-participant block structure; Kruskal–Wallis assumes independent groups.

When is quantile regression preferable?

Answer and reasoning

When a specific conditional median or another quantile difference is the estimand, especially when covariate adjustment is required.

Key take-home messages

References

1. Altman DG, Bland JM. Parametric v non-parametric methods for data analysis. BMJ. 2009;338:a3167. https://doi.org/10.1136/bmj.a3167
2. Mann HB, Whitney DR. On a test of whether one of two random variables is stochastically larger than the other. Ann Math Stat. 1947;18(1):50–60. https://doi.org/10.1214/aoms/1177730491
3. Wilcoxon F. Individual comparisons by ranking methods. Biom Bull. 1945;1(6):80–83. https://doi.org/10.2307/3001968
4. Kruskal WH, Wallis WA. Use of ranks in one-criterion variance analysis. J Am Stat Assoc. 1952;47(260):583–621. https://doi.org/10.1080/01621459.1952.10483441
5. Friedman M. The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc. 1937;32(200):675–701. https://doi.org/10.1080/01621459.1937.10503522
6. R Core Team. Wilcoxon rank sum and signed rank tests. R documentation, package stats version 4.6.1. Vienna: R Foundation for Statistical Computing; 2026. https://search.r-project.org/R/refmans/stats/html/wilcox.test.html