A study asks whether private tutoring raises final-year exam scores. The authors put every variable in their dataset into a regression: family income, previous-year grades, hours of self-study, mock exam score, stress level. The tutoring coefficient comes out near zero and they conclude that tutoring is useless. The trouble is that mock exam scores and self-study hours are exactly the paths through which tutoring would work if it works at all. Controlling for them deletes the effect they set out to measure.
Most questions in education, economics and public health cannot be answered with a randomised experiment. That does not mean you can say nothing about causes. It means you must state your assumptions clearly, and the clearest way is to draw them.
Why correlation falls short, and where
An association between X and Y in observational data can come from four sources: X causes Y; Y causes X (reverse causation); a third variable causes both (confounding); or the way the sample was selected created the association (selection bias). Causal inference is the job of ruling out the last three through design and defensible assumptions.
DAGs: assumptions written as arrows
A directed acyclic graph (DAG) represents each variable as a node and each arrow as an assumption that one variable may directly affect another. A missing arrow is also a strong assumption: you are claiming no direct effect.
For the tutoring example, a reasonable DAG might be: family income → tutoring; family income → exam score; prior grades → tutoring; prior grades → exam score; tutoring → mock exam score → exam score. Pen and paper is enough, or use a free online tool such as DAGitty, which will tell you which set of variables to adjust for.
Three kinds of variable, three different treatments
| Type | Structure | Example | Adjust for it? |
|---|---|---|---|
| Confounder | X ← C → Y | Family income affects both access to tutoring and exam scores | Yes |
| Mediator | X → M → Y | Tutoring raises mock scores, which reflect exam performance | No, if you want the total effect |
| Collider | X → K ← Y | Admission to a selective class depends on both tutoring and ability | Never, including through sample selection |
Colliders are the subtlest trap. If you survey only students in a selective class, you have conditioned on a collider by the way you sampled. Within that group, students who got in without tutoring tend to have higher underlying ability, so tutoring can appear negatively related to ability even though no such relationship exists in the population. This is often called Berkson’s paradox.
A procedure for choosing control variables
- State the causal question precisely: the effect of X on Y, total effect or direct effect?
- Draw the DAG from theory and subject knowledge, not from correlations in your data.
- Find every back-door path from X to Y, that is, paths starting with an arrow pointing into X.
- Choose a set of variables that blocks every back-door path without blocking the causal path and without opening a path through a collider.
- Do not add variables “just to be safe”. Every control needs a reason on the diagram.
- Run a sensitivity analysis: how strong would an unmeasured confounder have to be to explain away the result? The E-value is one way to report this.
When the data allow it: quasi-experimental designs
- Difference-in-differences: compare before-and-after change in a group exposed to a policy with a group that was not, assuming the two would have followed parallel trends otherwise.
- Regression discontinuity: when treatment depends on a cut-off, such as a scholarship threshold, compare people just above and just below it.
- Instrumental variables: a variable that affects X but affects Y only through X. Powerful in theory, but the assumption is hard to defend.
- Propensity score methods: balance groups on observed variables. They do nothing about unmeasured confounders, and the choice of variables must still come from the DAG.
Each method swaps one hard-to-test assumption for another. None replaces careful thought about the causal diagram.
Reverse causation and timing
If predictor and outcome are measured at the same moment, ask which direction is more plausible. Does stress lower exam performance, or does poor performance raise stress? Longitudinal data with the predictor measured before the outcome help a great deal, but they do not solve confounding on their own.
A simple rule: leave enough time between measurements for the cause to act, but not so much that many other events intervene. Record the chosen interval and your reasoning in the methods section, because reviewers will ask.
Word conclusions at the right strength
There are two mirror-image errors. The first is claiming causation with no design or assumptions to support it. The second, just as common, is writing “is associated with” while the whole discussion quietly offers policy advice as though the finding were causal. The honest approach is to say it plainly: our question is causal; under the assumptions shown in the diagram (see the appendix), the estimate is as follows; it would be wrong if a confounder of this kind exists.
Next step: take the last regression model you ran, draw its DAG on a sheet of paper and place each control variable into one of the three categories in the table above. If any turns out to be a mediator or a collider, you have just found the reason your results looked odd.
Câu hỏi thường gặp
What is a DAG in research?
A DAG, or directed acyclic graph, represents causal assumptions between variables as nodes and arrows. It helps you decide which variables to adjust for, and which to leave alone, when estimating an effect.
Should I include as many control variables as possible?
No. Adjusting for a mediator removes part of the effect you want to estimate, and adjusting for a collider can create a spurious association. Each control should be justified by the causal diagram.
What is a confounding variable?
A confounder affects both the exposure and the outcome, producing an association even when there is no causal link. Family income, for example, can influence both access to tutoring and exam results.
Can cross-sectional data show causation?
Only to a very limited extent. You need strong assumptions about the direction of effects and about having adjusted for all important confounders. State those assumptions openly and use cautious wording.
What is Berkson’s paradox?
It is a spurious association that appears when a sample is selected on a variable affected by both variables under study. Studying only students in a selective class, for instance, can create a false negative link between tutoring and ability.
