High-impact journals receive far more systematic reviews than they publish. What separates the accepted ones is almost always methodological rigour that was decided before any searching began.
1. Frame the question
Use a structured framework so the question is answerable. PICO (Population, Intervention, Comparator, Outcome) suits intervention questions; SPIDER and PEO suit qualitative and exposure questions respectively.
A question that cannot be framed this way is usually too broad, and a review with a vague question produces a search that cannot be bounded.
2. Register a protocol before searching
Register with PROSPERO or an equivalent registry, and do it before screening starts. Journals increasingly require a registration number, and its absence invites the suspicion that inclusion criteria were adjusted once the results were visible.
3. Build a reproducible search
Search at least three databases appropriate to your field. Record, for each: the database, the interface, the full search string, any filters, and the date searched.
The full string must appear in the paper, usually as a supplementary file. A reviewer should be able to paste it in and obtain your number of hits. Our guide to searching PubMed effectively covers MeSH terms and field tags in detail.
4. Screen in two stages, in duplicate
Title and abstract first, then full text. Two reviewers screen independently, with disagreements resolved by discussion or a third party. Record the reason for every full-text exclusion, because the PRISMA flow diagram requires it.
5. Appraise risk of bias
Use an established tool rather than inventing criteria: RoB 2 for randomised trials, ROBINS-I for non-randomised studies, QUADAS-2 for diagnostic accuracy, or the Newcastle-Ottawa Scale for observational designs.
Report the judgement per domain per study, not a single overall score. Reviewers want to see where the weaknesses are, not a number.
6. Synthesise honestly
If the studies are comparable, a meta-analysis is appropriate. If they are not, a structured narrative synthesis is the correct choice and should be stated as a decision with a reason, not presented as a shortfall.
7. Assess certainty
GRADE is expected in clinical fields and increasingly elsewhere. It forces you to state how confident the evidence permits you to be, which is what a reader actually needs.
Where reviews fail peer review
- The search strategy is summarised rather than reproduced in full
- Only one database was searched
- Screening was done by a single reviewer with no second check
- Risk of bias was assessed but never referred to again in the discussion
- Heterogeneous studies were pooled because a pooled figure looked stronger
- Conclusions are stronger than the certainty assessment supports
If you would like the method reviewed before submission, our pre-submission peer review covers exactly these points.