Linear regression: reading coefficients, checking assumptions, avoiding misreadings

A dissertation states: “Each year of experience increases salary by 1,800.” The model also includes education, sex, industry, firm size and region. The accurate sentence is: “Among people with the same education, sex, industry, firm size and region, each additional year of experience is associated with an average salary 1,800 higher.” The two sentences differ in two places: the condition of holding other variables constant, and “associated with” instead of “increases”.

Linear regression is the most widely used analytical tool in the social sciences, economics and public health. Running the model takes seconds; reading and checking it properly is where the work lies.

Reading one row of the coefficient table

Unstandardised coefficients with confidence intervals are a sound default because they keep the original units and readers grasp them immediately. Standardised betas help when comparing variables on different scales, but they are awkward for dummy variables.

Categorical variables and the reference group

A categorical variable with k levels enters the model as k − 1 dummy variables; the remaining level is the reference group. A coefficient of 3,200 for “Master’s degree” means that people with a master’s earn on average 3,200 more than the reference group, say “Bachelor’s”, other variables held constant. Change the reference group and the coefficients change, but the model does not. Choose a meaningful, reasonably large reference group and label it in the table.

The intercept is the predicted value when every variable equals zero, which is often meaningless: zero years of experience, age zero. To make it interpretable, centre continuous variables by subtracting the mean; the intercept then describes a “typical” person.

R squared matters less than you think

R² is the proportion of variance in the outcome explained by the model. Two things to know:

• A low R² does not make a model useless. In research on human behaviour, R² values of 0.10 to 0.30 are very common, and a coefficient can still be estimated precisely and matter.
• R² always rises when you add variables, even junk ones. Use adjusted R² to compare models with different numbers of predictors.

Checking assumptions with residual plots

The main assumptions of linear regression, and how to check them:

1. Linearity: plot residuals against fitted values; a curve suggests adding a squared term or a log transformation.
2. Independence: a matter of design. Repeated measures or clustered data need different models.
3. Constant variance: residuals should not fan out like a funnel. If they do, use robust standard errors or transform the outcome.
4. Approximately normal residuals: check a Q-Q plot. In large samples, mild departures have little effect on inference about coefficients.

Note that normality concerns the residuals, not the outcome or the predictors. Predictors can be skewed or even binary.

Multicollinearity and influential points

Multicollinearity arises when predictors are strongly correlated, such as age and years of experience. Coefficients become unstable, standard errors balloon and signs can flip in puzzling ways. Check variance inflation factors (VIF); thresholds of 5 or 10 are common but are only conventions. Remedies: drop a redundant variable, combine variables into an index, or accept it if the goal is purely prediction.

Influential points are observations whose removal changes the coefficients noticeably; Cook’s distance helps identify them. Do not delete them because they are inconvenient. Check whether they are data errors or genuine values, fit the model with and without them, and report both if conclusions differ.

Five common misinterpretations

• Causal language with observational data: “increases”, “the effect of”. Write “associated with” unless the design supports causal inference.
• Dropping the “other things equal” condition, as in the opening example.
• Comparing raw coefficients across variables with different units: 0.05 for income in thousands is not smaller than 2 for sex.
• Extrapolating beyond the data: a model fitted on people with 0 to 20 years of experience says nothing about someone with 40.
• Stepwise selection by p values, then reporting p values from the final model as though it had been specified in advance. Select variables on theory and causal diagrams rather than algorithms.

How many variables can your sample support?

A common rule of thumb is at least 10 to 20 observations per estimated parameter. With 150 people, a 25-variable model is almost certainly overfitted: it describes the noise in this sample well and predicts poorly in the next. Categorical variables with many levels use up more parameters than people expect.

In practice: a checklist before reporting a model

1. Every variable has a theoretical justification stated in the methods.
2. The reference group for every categorical variable is stated.
3. You have examined residuals against fitted values and a Q-Q plot.
4. You have checked VIFs and Cook’s distance.
5. You report coefficients, 95% confidence intervals, p values, sample size and adjusted R².
6. Every interpretation includes “other things equal” and avoids causal language the design cannot support.

Next step: take your most recent regression model, plot residuals against fitted values straight away, and rewrite the interpretation of your main coefficient using the template “among people with the same…, each unit of… is associated with…”. It takes fifteen minutes and removes two of the most common reviewer complaints.

Câu hỏi thường gặp