A study of 20,000 pupils reports that a morning reading programme raised comprehension scores, p < 0.001. Impressive, until you see the numbers: a gain of 0.4 points on a 100-point scale, roughly 0.03 standard deviations. Another study of 30 pupils reports a 9-point gain with p = 0.09 and is dismissed as “no effect”. Asked which programme to try next, most educators would pick the second.
A p value answers “are the data compatible with no effect?”. It depends on both the size of the effect and the size of the sample, so it cannot tell you how big anything is. An effect size answers the question readers actually care about: how large is the effect?
Effect sizes with and without units
The best effect size is often one that keeps the original units: 9 points higher, 5 mmHg lower blood pressure, 12 percentage points more passing. Readers grasp it immediately and can judge practical importance themselves. Standardised effect sizes (d, r, η²) are needed when the scale has no intrinsic meaning, such as scores on a bespoke questionnaire, when comparing studies that used different instruments, or for meta-analysis. Report both where you can.
The main families
| Comparison | Measure | Meaning |
|---|---|---|
| Two means | Cohen’s d, Hedges’ g | Difference in means in standard deviation units |
| Two continuous variables | Correlation r, R² | Strength of association; R² is variance explained |
| ANOVA | η², partial η², ω² | Proportion of outcome variance associated with a factor |
| Binary outcome | Risk difference, risk ratio (RR), odds ratio (OR) | How likely the event is in one group relative to another |
| Clinical intervention | Number needed to treat (NNT) | How many people must be treated for one extra good outcome |
Cohen’s d and Hedges’ g
Cohen’s d is the difference between two means divided by the pooled standard deviation. Intervention mean 72, control mean 66, pooled SD 12: d = 6 / 12 = 0.5. Hedges’ g corrects d for small-sample bias; with a few dozen per group the two are nearly identical, but meta-analyses usually use g.
A more intuitive way to express d is the probability of superiority: the chance that a randomly chosen person from the intervention group scores higher than a randomly chosen person from the control group. Under normality, d = 0.5 corresponds to about 64%, d = 0.2 to about 56% and d = 0.8 to about 71%. Telling a committee “a pupil on the new programme has a 64% chance of outscoring one on the old” often lands better than “d = 0.5”.
A caution for repeated-measures designs: d can be computed from the SD of the raw scores or the SD of the difference scores, and the two can differ a great deal. Always state the formula.
r, R² and eta squared
An r of 0.3 sounds small because R² = 0.09, “only 9% of variance explained”. Yet in research on human behaviour many well-established relationships have r between 0.1 and 0.3. Squaring makes every association look smaller than it feels; do not report R² alone and call the relationship “weak”.
In ANOVA, η² is the proportion of total variance associated with a factor. Software often reports partial η², which removes variance due to other factors, so it is usually larger and cannot be compared directly across designs. ω² is less biased than η² in small samples. Say which one you report.
An odds ratio is not a risk ratio
When the event is rare, below roughly 10 per cent, OR and RR are close. When it is common, the OR exaggerates the RR. Example: 80% pass in group A, 60% in group B. RR = 0.8 / 0.6 ≈ 1.33. Odds in A = 0.8 / 0.2 = 4; in B = 0.6 / 0.4 = 1.5; OR ≈ 2.67. Writing “2.67 times as likely to pass” is wrong; the odds are 2.67 times higher. The absolute difference is 20 percentage points, which gives NNT = 1 / 0.2 = 5: for every five pupils taught method A, one more passes.
Logistic regression reports ORs, so this misreading is extremely common. Where possible, add the absolute rates for each group: “from 60% to 80%” is clearer than any ratio.
Small, medium and large: a last resort, not a yardstick
Jacob Cohen’s familiar benchmarks are 0.2, 0.5 and 0.8 for d, and 0.1, 0.3 and 0.5 for r. Cohen himself intended them for when nothing better is available. Three more careful readings:
- Against similar studies in your field. In education, large syntheses show that many rigorously evaluated interventions have effects below 0.2, so 0.25 may be notable. A painkiller with d = 0.25 over placebo may not be worth taking.
- Against cost and reach. A small effect from a cheap intervention delivered to millions can matter more than a large effect from an expensive one that is hard to scale.
- Against an established threshold of practical importance, such as the minimal change patients can perceive on a symptom scale.
Published small studies tend to overstate effects because of publication bias, so a d of 0.8 from a study of 20 people deserves caution.
Converting between measures
When synthesising literature you will meet papers reporting different measures. Some common approximations, assuming two equal-sized groups and roughly normal data:
- From d to r: r = d / √(d² + 4). For d = 0.5, r ≈ 0.24.
- From r to d: d = 2r / √(1 − r²).
- From OR to d: d ≈ ln(OR) × √3 / π ≈ ln(OR) × 0.55. For OR = 2.67, d ≈ 0.54.
These are approximations; for meta-analysis, use dedicated tools such as the metafor or esc packages in R and state which conversions you applied.
In practice: reporting effect sizes with confidence intervals
An effect size is an estimate, so it needs a confidence interval too. d = 0.5 with an interval of 0.1 to 0.9 says something very different from d = 0.5 with an interval of 0.4 to 0.6. Checklist:
- Report the main result as an effect in original units.
- Add a standardised effect size suited to the analysis, stating the formula.
- Give 95% confidence intervals for both.
- For binary outcomes: group rates, absolute difference, then the ratio.
- Interpret magnitude by comparison with your field, not only with small, medium and large labels.
Model sentence: “The intervention group scored 6.0 points higher than the control group (95% CI 1.7 to 10.3), d = 0.50 (0.14 to 0.86), above the median effect of comparable reading interventions in recent reviews.”
Effect sizes are also the key input for planning sample size before data collection, which is covered in a separate article.
Next step: go through the results section of your manuscript, find every sentence that reports only a p value, and add an effect in original units with its confidence interval. Then ask yourself: compared with similar studies, is this large or small?
Câu hỏi thường gặp
What is an effect size?
A number describing how large an effect or association is, such as a mean difference, Cohen’s d, a correlation or an odds ratio. Unlike a p value, it does not depend directly on sample size.
What does a Cohen’s d of 0.5 mean?
The group means differ by half a standard deviation, roughly a 64% chance that a random person from one group scores higher than a random person from the other. Whether that is large depends on comparable studies in your field.
What is the difference between an odds ratio and a risk ratio?
A risk ratio compares probabilities; an odds ratio compares odds. When the outcome is common, the odds ratio is noticeably larger and should not be read as times more likely.
What is the difference between eta squared and partial eta squared?
Eta squared uses total variance, while partial eta squared removes variance due to other factors and is usually larger. The two cannot be compared directly.
Should I report confidence intervals for effect sizes?
Yes. An effect size is an estimate, and its confidence interval shows how precise it is. The same d of 0.5 with a wide or narrow interval leads to very different conclusions.
