A team spent ten months reviewing 38 studies on online learning and achievement. The manuscript came back in five days. The letter was brief: no registered protocol, a single database searched, no risk-of-bias assessment, and two reviews addressing almost the same question in the past three years. None of these problems was about the writing. Every one had been decided in the first week.
Systematic reviews attract citations, so many people want to write them, and editors read them with a very clear checklist in mind. This article is not a full course in review methods; it focuses on what decides whether a review gets published.
Before you start: has it been done?
The most common and most painful reason for rejection is redundancy. Before drafting a protocol, search:
- Published reviews in your field’s main databases, filtered by article type.
- Registered protocols on PROSPERO (for health-related questions) or the Open Science Framework.
- The Cochrane or Campbell libraries if your topic touches health, education or social policy.
An existing recent review does not automatically rule you out. Many new primary studies, methodological weaknesses in the earlier review, or a different population can all justify a new one. That justification must appear in your introduction.
Register a protocol: proof you did not change the rules midway
Registration shows that the question, eligibility criteria and analysis plan were fixed before you saw the results. Many journals expect the registration number in the abstract. A protocol should cover at least:
- The question, framed with PICO (population, intervention, comparison, outcome) or an equivalent such as SPIDER for qualitative evidence.
- Inclusion and exclusion criteria, including language and date limits.
- Sources and the planned search strategy.
- Screening: how many reviewers, how disagreements are resolved.
- The risk-of-bias tool.
- The synthesis plan: meta-analysis or narrative synthesis, and the conditions for each.
Deviations from the protocol are normal if they are reported and explained. What editors distrust are deviations nobody mentions.
A search someone else can rerun
“We searched Scopus using the keywords online learning and achievement” is not enough. Current expectations:
- At least two or three databases suited to the field, plus grey literature and reference list checking.
- The full search string for at least one database, with Boolean operators, brackets and field tags, in a supplement.
- The date of the final search. If more than a year has passed before submission, many editors will ask for an update.
If your institution has a subject librarian, ask them to review the search. Many strong reviews acknowledge librarians or include them as co-authors.
PRISMA 2020 is more than a flow diagram
PRISMA 2020 is a reporting guideline with a multi-item checklist and a flow diagram. Newcomers often draw the diagram and ignore the checklist. Editors open the checklist first.
| Item group | Often missing |
|---|---|
| Title and abstract | “Systematic review” absent from the title; abstract not following PRISMA for Abstracts |
| Methods | Number of independent screeners not stated; data extraction not described |
| Results | Studies excluded at full-text stage not listed with reasons |
| Discussion | No discussion of the certainty of the overall evidence |
| Other information | Registration number, funding or access to extracted data missing |
Submit the completed PRISMA checklist with page numbers for every item. It takes half an hour and makes the editor’s job far quicker.
Risk of bias and certainty of evidence
A review that does not appraise its studies is just an annotated list. Match the tool to the design: RoB 2 for randomised trials, ROBINS-I for non-randomised studies, JBI or CASP checklists for many other designs. Then use the assessment to judge how certain your conclusions are, for example with the GRADE approach. A conclusion of “effective” built on high-risk studies must be worded cautiously.
Narrative synthesis is not a list
When data cannot be pooled, many reviews drift into “Study A found… Study B found…”. A genuine narrative synthesis groups studies by characteristics, compares the direction of effects and explains why results differ. The SWiM guideline (Synthesis Without Meta-analysis) helps you report this transparently.
Dividing the work in a review team
A systematic review needs at least two independent screeners, usually plus a third person to resolve conflicts. Agree in advance who screens titles and abstracts, who reads full texts, who extracts data and who assesses risk of bias. Pilot on the first 50 records and check agreement between screeners; if it is low, sharpen the criteria before continuing. Screening tools such as Rayyan or Covidence log every decision, so the numbers in your PRISMA diagram add up without manual counting.
In practice: a 10-point pre-submission check
- The title says “systematic review” and states the question.
- A protocol registration number, with any deviations explained.
- At least two databases plus supplementary sources.
- Full search strings in a supplement, with dates.
- Two independent screeners and a process for disagreements.
- A PRISMA 2020 flow diagram whose numbers add up in every box.
- A list of full-text exclusions with reasons.
- Risk of bias assessed for each included study.
- Certainty of evidence discussed.
- A completed PRISMA checklist with page numbers.
Next step: if you are still at the idea stage, spend one session searching for existing reviews and registered protocols on your question before doing anything else. If someone has already answered it well, that session has just saved you a year.
Câu hỏi thường gặp
Do I have to register my systematic review on PROSPERO?
Not every journal requires it, but many biomedical journals and a growing number in other fields expect it. For non-health questions, the Open Science Framework is a common alternative.
How many databases should a systematic review search?
The usual expectation is at least two or three databases relevant to the field, plus grey literature and reference list checking. Searching a single database is a frequent reason for rejection.
What changed in PRISMA 2020 compared with PRISMA 2009?
PRISMA 2020 updated the checklist, added items on synthesis methods and certainty of evidence, and introduced new flow diagram templates. Use the 2020 version unless a journal asks otherwise.
Does a systematic review need a meta-analysis?
No. When studies are too diverse to pool, a structured narrative synthesis is appropriate, provided you report how it was done, for example following SWiM.
Should I do a review if one already exists on my topic?
Possibly, if you can justify it: substantial new evidence, methodological limits in the earlier review, or a different population. State that justification clearly in the introduction.
