Two dissertations both study “teachers’ digital competence”. The first measures it by the hours teachers spend on a computer each week. The second measures whether teachers can build an online quiz with automatic feedback. Their conclusions point in opposite directions, and each is correct for its own measure. The problem is not the statistics; it is operationalisation.
Operationalisation is the process of turning an abstract concept into concrete measurement procedures. It is the bridge between theory and data. If the bridge is faulty, however sophisticated the analysis, it answers a different question from the one you asked.
Four layers from concept to number
| Layer | Question to answer | Example: teachers’ digital competence |
|---|---|---|
| Concept and definition | What do I mean by this, following which framework? | The ability to use digital technology purposefully for teaching, assessment and professional growth |
| Dimensions | What distinct aspects does it have? | Finding and evaluating digital resources; creating content; assessing learners digitally; protecting student data |
| Indicators | What observable signs show each dimension? | Can build an online quiz; can set sharing permissions for the right audience |
| Recording | How are data collected and coded? | A practical task scored on a 0–3 rubric; a five-point self-rating item |
Jumping straight from layer one to layer four, that is, grabbing the first existing questionnaire you find, is the shortest route to measuring the wrong concept.
Layer one: a working definition
Do not open with a dictionary definition. A working definition has to be narrow enough to measure and has to take a side when authors disagree. Many concepts in education and the social sciences have three to five competing meanings in the literature. List them, say which one you adopt and why it fits your question.
A useful test: hand your definition to a colleague and ask, “Under this definition, what would a high scorer look like, and a low scorer?” If they cannot say, the definition is still too vague.
Layer two: dimensions, and whether to combine them
Most concepts worth studying have several dimensions. The key decision is whether those dimensions add up to a total score or must be analysed separately. Two teachers with the same total of 12 may differ completely: one excellent at creating content but careless with data, the other the reverse. If your question concerns data protection, a total score hides exactly what you need.
Also distinguish two relationships between indicators and concept. In a reflective model the concept causes the indicators, as anxiety causes a racing heart and poor sleep, so indicators tend to correlate. In a formative model the indicators make up the concept, as income, education and occupation make up socio-economic status; they need not correlate, and dropping one changes the concept itself. Judging a formative index with Cronbach’s alpha is a familiar mistake.
Layer three: choosing indicators
A good indicator ties directly to the definition, moves when the concept moves, and is little affected by unrelated factors. Test each one with a question: “Could someone score high on this indicator while actually being low on the concept?” Hours on a computer fail immediately: a teacher can spend ten hours a week just entering marks.
Aim for at least two or three indicators per dimension. With a single indicator, all of its error becomes the error of the whole dimension.
Layer four: recording and level of measurement
- Self-report: cheap and fast, but it measures confidence in competence more than competence itself. The least skilled often overrate themselves the most.
- Performance tasks or scenarios: closer to actual ability, more time-consuming, and they need a scoring rubric and a second rater.
- Behavioural or records data: learning-platform logs, real teaching materials; objective, but they need permission and often lack context.
Decide the level of measurement in advance: nominal, ordinal, interval or ratio. Collecting age in bands throws information away for good; collecting exact age and banding later is always possible.
Proxies: convenient but risky
When direct measurement is impossible, researchers use proxies: publication counts for research productivity, entrance exam scores for academic ability, owning a home computer for study conditions. A proxy is acceptable when you can show it tracks the concept closely in your specific setting and you acknowledge the gap in your limitations. It is not acceptable when your independent variable could affect the proxy through some other route.
Check content validity before collecting data
- Build a matrix with indicators as rows and dimensions as columns. Every dimension needs indicators; no indicator should sit outside all dimensions.
- Ask three to five experts to rate each indicator’s relevance to its dimension on a four-point scale. Compute the proportion rating it 3 or 4.
- Run cognitive interviews with a few people from your target group: how do they understand each item, and what are they thinking as they answer?
- Revise, and only then pilot for reliability.
Writing it up
A four-column table in your methods chapter (concept, dimension, indicator, measure and source) lets examiners see the chain of reasoning at a glance. Add a short paragraph on what you deliberately chose not to measure and why. It shows judgement rather than omission.
Do this now: take the central concept of your project, fill in the four-layer table from the start of this article, and apply the “high on the indicator, low on the concept” test to every row. Any row that fails needs replacing before your instrument goes to print.
Câu hỏi thường gặp
What does operationalisation mean in research?
It is the process of turning an abstract concept into concrete measurement: writing a working definition, identifying dimensions, choosing indicators and deciding how data will be recorded. It determines whether your numbers actually reflect the concept.
What is the difference between a concept, a variable and an indicator?
A concept is an abstract idea such as digital competence. An indicator is an observable sign of that concept. A variable is how the indicator is recorded as values in your dataset.
Should I use an existing scale instead of operationalising myself?
A validated scale is often the better choice, but only if its definition matches yours and it suits your participants’ context. You still need to work through the layers of operationalisation to check that match.
What is the difference between reflective and formative indicators?
With reflective indicators the concept causes them, so they usually correlate. With formative indicators they jointly make up the concept, need not correlate, and should not be judged by Cronbach’s alpha.
What is a proxy variable and when is it acceptable?
A proxy is an indirect measure used when the concept cannot be measured directly, such as publication counts for research productivity. Use one only when there is evidence it tracks the concept in your setting, and state its limits.
