A familiar table in theses: five scales with Cronbach’s alpha of 0.82, 0.87, 0.91, 0.94 and 0.96, followed by “all exceed 0.7, so reliability is good”. An experienced reader stops at 0.96 and opens the appendix. Usually that scale contains eight items saying almost the same thing in eight different ways.
Reliability and validity are separate questions. Reliability asks whether repeated measurement gives consistent results. Validity asks whether you are measuring what you intend to measure. A bathroom scale that always reads two kilograms heavy is highly reliable and not valid. This guide explains the common indices and how to read them without treating them as rituals.
Cronbach’s alpha: what it measures and what it does not
Alpha measures internal consistency: whether items on the same scale vary together. Three points are rarely made explicit:
- Alpha rises with the number of items. A 20-item scale can reach a high alpha even when inter-item correlations are modest. A three-item scale with alpha of 0.68 may be perfectly good.
- Alpha does not demonstrate unidimensionality. Two clusters measuring different facets can still yield a high alpha.
- Very high alpha (above roughly 0.95) usually signals redundancy, not excellence. Respondents are asked the same thing repeatedly, get bored and start straight-lining.
McDonald’s omega is increasingly recommended instead of, or alongside, alpha because it does not assume equal loadings for all items. In R, the psych or semTools packages compute omega; JASP and jamovi offer it out of the box.
The “alpha if item deleted” temptation
The SPSS column “Cronbach’s alpha if item deleted” tempts many researchers to drop items one by one until alpha peaks. This causes three problems: the trimmed scale no longer matches the published instrument, so comparisons break; the dropped item may capture an important facet of the construct; and you are optimising to this particular sample, so the result may not replicate.
Drop an item only when there is a substantive reason alongside the statistical one: a mistranslation, a double negative, an item that makes no sense in context. Report which items were removed and why.
Common indices and how to read them
| Index | What it checks | Common threshold | Caveat |
|---|---|---|---|
| Cronbach’s alpha | Internal consistency | ≥ 0.70 | Depends on item count; above 0.95 suggests redundancy |
| Corrected item–total correlation | Does the item move with the scale? | ≥ 0.30 | Read the item before deleting |
| KMO | Suitability for factor analysis | ≥ 0.60 | A necessary, not sufficient, condition |
| Factor loading | How strongly an item reflects its factor | ≥ 0.50 (ideally 0.70) | Watch for cross-loadings |
| Composite reliability (CR) | Reliability within CFA | ≥ 0.70 | Replaces alpha in SEM reporting |
| AVE | Convergent validity | ≥ 0.50 | Sensitive to weak items |
| HTMT | Discriminant validity | < 0.85 (or 0.90) | Preferred over the square-root-of-AVE rule |
| CFI, TLI / RMSEA, SRMR | CFA model fit | ≥ 0.90–0.95 / ≤ 0.06–0.08 | Not hard-and-fast rules |
These thresholds are conventions in the methods literature, not laws of nature. Report the actual values and argue your case when an index falls slightly short, rather than quietly deleting items until it clears the bar.
EFA or CFA, and in which order
Exploratory factor analysis (EFA) suits situations where the structure is uncertain: a new scale, or an established scale used in a new language or culture for the first time. Confirmatory factor analysis (CFA) suits situations with a clear hypothesis about structure, such as a well-validated instrument. A common mistake is running EFA and then CFA on the same dataset and presenting the CFA as confirmation. With a large enough sample, split it randomly: EFA on one half, CFA on the other.
For EFA, use a factor extraction method such as principal axis factoring or maximum likelihood rather than principal components if you are after latent variables, and an oblique rotation (oblimin, promax) when factors may correlate — which in the social sciences is nearly always.
Validity is more than numbers
The indices above mostly describe a scale’s internal structure. A complete validity argument also needs:
- Content validity: experts judge whether the items cover the construct adequately.
- Criterion validity: scores relate as expected to an independent measure (for instance, a turnover-intention scale predicting actual resignations six months later).
- Convergent and discriminant evidence against other instruments: high correlations with measures of similar constructs, low ones with distant constructs.
Common method bias
When every variable comes from one self-report questionnaire at one time point, correlations can be inflated. Harman’s single-factor test is widely reported but weak; it detects only extreme cases. Better protection lies in design: separate the measurement of predictors and outcomes in time, use different data sources (for example, grades from a records system), vary response formats, and guarantee anonymity.
A reporting checklist for scales
- Source of each scale, number of items, response format and any translation procedure.
- Alpha and/or omega for each scale, with the number of items.
- EFA or CFA results with fit indices.
- CR, AVE and HTMT if you use SEM.
- A list of deleted items with substantive and statistical reasons.
- Any validity evidence beyond internal structure.
Next step: open your reliability table, flag every scale with alpha above 0.93 and reread each item in it. If two items are ones a respondent could hardly answer differently, consider dropping one in your next survey — and note the issue as a limitation in the current paper.
Câu hỏi thường gặp
What is a good Cronbach’s alpha value?
A common guideline is 0.70 or higher, but interpret it in light of scale length. Values above about 0.95 often indicate redundant items rather than an excellent scale.
Should I delete items to improve Cronbach’s alpha?
Only with a substantive reason such as a translation problem or confusing wording. Deleting items purely to maximise alpha changes the instrument, may lose part of the construct and harms replicability.
When should I use EFA versus CFA?
Use EFA when the factor structure is uncertain, such as with a new or newly translated scale. Use CFA when you have a clear structural hypothesis. Running CFA on the same data as EFA is not real confirmation.
What are AVE and HTMT used for?
AVE assesses convergent validity, whether items share enough variance with their construct. HTMT assesses discriminant validity between constructs, with values below 0.85 or 0.90 commonly recommended.
Is Harman’s single-factor test enough to rule out common method bias?
No. It only detects severe cases. Design choices such as separating measurement occasions or using different data sources offer much stronger protection.
