A master’s student had two groups of 180 people each, rating satisfaction on a 0–100 scale. Shapiro–Wilk gave p = 0.003 for one group. She switched to Mann–Whitney and wrote that “the median of group A was significantly higher”. A reviewer asked two questions: why abandon the t-test with samples that large, and how can the medians differ significantly when they are identical?
Non-parametric tests are good tools, but they are often chosen by reflex rather than reason. Knowing what they actually test helps you choose well and report correctly.
Normality tests often answer the wrong question
The t-test assumes that the sampling distribution of the mean, or the model residuals, is approximately normal, not that every data point is. With moderate or large samples, the central limit theorem usually satisfies that assumption even when the raw data are skewed.
The paradox of Shapiro–Wilk and Kolmogorov–Smirnov:
- With small samples they have little power, so they often miss non-normality precisely when it matters most.
- With large samples they flag tiny departures that make no practical difference to a t-test, precisely when it matters least.
Instead of relying on a normality p-value, look at the data: histograms, Q-Q plots, and sharper questions. Are there extreme outliers? Ceiling or floor effects? How many observations per group?
Mann–Whitney is not a “median test”
The Mann–Whitney U test (also called the Wilcoxon rank-sum test) asks whether a randomly chosen value from group A tends to be larger than a randomly chosen value from group B. It only becomes a comparison of medians if both distributions have the same shape and differ only in location. When shapes differ, two groups can share a median and still produce a significant Mann–Whitney result, as in the opening example.
A safe way to report: give the median and interquartile range of each group as description, then state the test result as “scores in group A tended to be higher”. Add an appropriate effect size, such as the probability of superiority or the rank-biserial correlation.
Common tests side by side
| Situation | Parametric test | Non-parametric counterpart | Effect size |
|---|---|---|---|
| Two independent groups | Independent t (prefer Welch) | Mann–Whitney U | Rank-biserial correlation, probability of superiority |
| Two measurements on the same people | Paired t | Wilcoxon signed-rank | Matched-pairs rank-biserial correlation |
| Three or more independent groups | One-way ANOVA (or Welch) | Kruskal–Wallis | Epsilon squared |
| Three or more repeated measurements | Repeated-measures ANOVA | Friedman | Kendall’s W |
| Correlation | Pearson | Spearman, Kendall’s tau | The coefficient itself |
Note: for two independent groups, Welch’s t-test does not assume equal variances and should be your default rather than Student’s t.
When non-parametric tests genuinely make sense
- Truly ordinal data: rankings, short scales where the gaps between points are not equal.
- Very small samples with clear skew and outliers that cannot be removed because they are real.
- Outcomes with extreme values that a mean describes badly, such as length of hospital stay or days absent.
- Questions that are inherently about ranking or tendency to be larger, rather than about means.
Options beyond “parametric or non-parametric”
When data are skewed, you have other ways to keep information about the size of an effect:
- Transformation, such as a log for right-skewed variables (income, time). Interpret on the transformed scale or back-transform for readers.
- Bootstrap confidence intervals for differences in means or medians, without distributional assumptions.
- Robust methods such as trimmed means and Yuen’s test.
- Models that match the data type: Poisson or negative binomial regression for counts, ordinal regression for single Likert items, gamma models for positive skewed variables.
The big weakness of classic non-parametric tests is that they are hard to extend: adjusting for confounders or modelling interactions is awkward. Once your question is more than a two-group comparison, an appropriate model is usually better.
In practice: choosing a test for two groups
- Plot boxplots and histograms for each group; do not start with Shapiro–Wilk.
- Identify the data type: continuous, count or ordinal.
- If continuous, with a few dozen or more observations per group and no extreme outliers: use Welch’s t-test and report the mean difference with its confidence interval.
- If samples are small, heavily skewed or ordinal: use Mann–Whitney and report medians, interquartile ranges and a rank-based effect size.
- If you need to adjust for other variables: choose a regression model suited to the data type.
- Give a one-sentence reason in your methods, not just “because the data were not normally distributed”.
Next step: search your thesis or paper for the sentence “because the data were not normally distributed, we used…”. Reopen the plots and group sizes, and ask whether that choice still stands, and whether your write-up describes what the test actually measures.
Câu hỏi thường gặp
Do I have to use a non-parametric test if my data are not normal?
No. With moderate or large samples, the t-test is usually robust thanks to the central limit theorem. Look at plots, sample sizes and outliers before deciding.
Does the Mann–Whitney test compare medians?
Only when both distributions have the same shape. In general it tests whether values in one group tend to be larger than in the other, so report it that way.
Should I use Shapiro–Wilk to check normality?
It can be informative, but do not rely on it alone. It lacks power in small samples and is oversensitive in large ones; Q-Q plots are usually more useful.
What should I do after a significant Kruskal–Wallis test?
Run pairwise post hoc comparisons such as Dunn’s test with a correction for multiple comparisons, and report effect sizes for each pair.
Do non-parametric tests need effect sizes?
Yes. Report a rank-biserial correlation, probability of superiority, epsilon squared or Kendall’s W as appropriate, not just the p-value.
