Systematic and Scoping Review Protocols

Overview

A review protocol is the prospective methodological plan for identifying, selecting, appraising and synthesising existing evidence. PRISMA-P provides minimum reporting items for systematic review and meta-analysis protocols, supported by an explanation-and-elaboration document.[1,2] Scoping review protocols require a different logic because the objective is usually to map concepts, evidence types, characteristics or gaps rather than estimate a single pooled effect; current JBI guidance provides detailed conduct recommendations, while PRISMA-ScR addresses reporting of completed scoping reviews.[3,4] The Cochrane Handbook remains a major current source for intervention-review methods, and PROSPERO provides prospective registration for eligible systematic reviews.[5,6]

The protocol protects against post hoc changes driven by the studies that are found. It also exposes feasibility problems early: a question may be too broad, an outcome definition may be unusable across studies, or the planned meta-analysis may require effect measures that the literature rarely reports.

Why this matters

Systematic evidence synthesis contains many discretionary decisions: database selection, search terms, eligibility, handling of multiple reports, choice of outcomes, risk-of-bias tools, effect measures, subgroup definitions and synthesis models. If these are decided after study results are known, reviewers can unintentionally favour a preferred conclusion.

Scoping reviews have different risks. Because their questions are broad, investigators may repeatedly expand or contract the scope during screening, producing an undocumented final map that no longer matches the original purpose. A protocol makes these adaptations visible and helps the team decide in advance what will be charted and how evidence will be presented.

Key concepts

A systematic review uses explicit, reproducible methods to answer a focused review question and synthesise eligible evidence. It may or may not include meta-analysis. A scoping review systematically maps the extent, characteristics, concepts or types of evidence in a broader field and is often appropriate when terminology, interventions or evidence types are heterogeneous. A scoping review should not be chosen merely because the team wants to avoid critical appraisal or because the search is difficult.

Question frameworks should fit purpose. PICO is common for intervention-effectiveness reviews; other frameworks may suit diagnostic, prognostic, qualitative or aetiologic questions. JBI often uses Population–Concept–Context for scoping reviews.[3] Eligibility criteria should operationalise the framework while remaining consistent with the review objective.

The protocol must distinguish search sensitivity from screening specificity. Broad searches may intentionally retrieve many irrelevant records to avoid missing eligible evidence. Database selection should reflect subject coverage, not a desire to maximise the number of databases. Supplementary methods—citation searching, trial registries, grey literature or contacting experts—should be justified by the review question.

DECISION GUIDE

Decide the review type before drafting eligibility criteria

• Is the question focused enough to support a defined estimand, effect or evidence statement?
• Are the interventions, populations and outcomes conceptually comparable?
• Is the objective to map concepts, evidence types, methods or gaps rather than estimate effectiveness?
• Will formal risk-of-bias appraisal change the review conclusion or is mapping the primary purpose?
• What sources and languages are required to avoid predictable coverage bias?
• Where can the protocol be publicly registered or deposited before screening and synthesis?

A step-by-step methodological workflow

1. Write the review objective and framework

State the review purpose in one sentence. For a focused systematic review, define population, intervention/exposure, comparator, outcomes and study designs as appropriate. For a scoping review, define population, concept and context and clarify what “mapping” will produce—categories, evidence distribution, conceptual definitions or research gaps.

2. Operationalise eligibility criteria

Specify study designs, participants, settings, interventions/exposures, comparators, outcomes, publication types, dates and languages. Avoid eligibility rules based on information known only after reading the results, such as including studies only when an effect is statistically significant. For scoping reviews, broad inclusion is acceptable when it directly serves the mapping objective.

3. Design and peer-review the search strategy

Identify core bibliographic databases and supplementary sources. Translate concepts into controlled vocabulary and free-text terms for each platform. Save complete search strings, dates and result counts. Consider librarian or information-specialist review for complex searches. Plan citation searching and registry searching where they can recover evidence missed by database indexing.

4. Prespecify screening and record management

Define deduplication, title/abstract screening, full-text review, reviewer numbers and disagreement resolution. Pilot eligibility criteria on a sample before formal screening. Track exclusion reasons at full text. Link multiple reports of the same underlying study so that the unit of analysis is not accidentally duplicated.

5. Plan extraction, appraisal and synthesis

For systematic reviews, define outcomes, effect measures, risk-of-bias tool, data conversions, meta-analysis model if appropriate, heterogeneity assessment, subgroup/sensitivity analyses and certainty-of-evidence methods. For scoping reviews, define charting fields, classification frameworks and mapping outputs. Critical appraisal may be included when it serves the scoping objective, but it should not be inserted automatically.

6. Register, version and report amendments

Use PRISMA-P to improve protocol reporting for systematic reviews.[1,2] Register eligible health-related systematic reviews prospectively in PROSPERO where appropriate, or use another persistent registry/repository when scope or eligibility differs.[6] Preserve protocol versions and explain substantive amendments, particularly changes to eligibility, outcomes or synthesis after screening has begun.

PROTOCOL CHECKLIST

Review-protocol items to prespecify

• review type, rationale and structured question;
• eligibility criteria and unit of inclusion;
• databases, platforms and supplementary search methods;
• complete search documentation and update plan;
• deduplication and screening workflow;
• data extraction or charting fields;
• risk-of-bias or critical-appraisal approach where relevant;
• effect measures, synthesis, heterogeneity and sensitivity analyses for systematic reviews;
• mapping and presentation plan for scoping reviews;
• registration, amendments, conflicts and dissemination.

An auditable planning document

Systematic versus scoping review planning matrix

illustrative

Table 1. Illustrative design decision

Feature	Focused systematic review	Scoping review
Main purpose	Estimate/synthesise answer to a focused question	Map extent, concepts and evidence types
Question framework	Often PICO or design-specific framework	Often Population–Concept–Context
Risk of bias	Usually central to interpretation	Optional when it serves the mapping purpose
Meta-analysis	Possible if studies support it	Usually not the defining aim
Output	Effect/evidence statement and certainty	Evidence map, categories, gaps and concepts
Protocol guidance	PRISMA-P + conduct-specific handbook	JBI scoping guidance; transparent protocol

Note: Both approaches require systematic searching and transparent selection.

Quality, validity and governance

Review quality depends heavily on decisions made before the first forest plot or evidence map is seen. Search strategies should be reproducible by platform. Eligibility criteria should be pilot-tested because disagreements often reveal hidden ambiguity. Data extraction forms should be piloted on representative studies, especially when outcomes are reported in multiple ways.

Systematic reviews should avoid treating meta-analysis as mandatory. Pooling studies that estimate incompatible quantities can create a precise but meaningless average. The protocol should define conditions under which meta-analysis will not be performed and how narrative synthesis will proceed. If meta-analysis is planned, heterogeneity and subgroup analyses should follow clinical and methodological reasoning rather than arbitrary thresholds.

Scoping reviews should avoid “gap inflation”. An empty cell in an evidence map does not automatically mean research is needed; the absence may reflect an unimportant question, inaccessible literature or a deliberately excluded population. Gap statements should describe what evidence is absent and why that absence matters for decisions.

APPRAISAL CHECKLIST

Appraise a review protocol

• Does the review type fit the purpose?
• Are eligibility criteria operational and independent of study results?
• Is the search broad enough for the question and reproducible by platform?
• Are duplicate reports of the same study handled correctly?
• For systematic reviews, are risk of bias, effect measures and synthesis decisions prespecified?
• For scoping reviews, are charting and mapping outputs defined rather than left open-ended?

Practical decision audit

Audit feasibility before registration

Run scoping searches to estimate volume and terminology, but do not use preliminary effect results to tune eligibility toward a preferred conclusion. If the field is enormous, narrow the concept, population or time period for a scientific reason and record that decision before formal screening.

Audit the unit of evidence

A trial may generate an abstract, protocol, primary paper and subgroup report. These are multiple reports of one study, not four independent studies. Establish a study-level identifier and link reports before extraction to prevent double counting.

Audit synthesis compatibility

Before pooling, compare populations, interventions, comparators, outcome definitions, time points and effect measures. Statistical heterogeneity alone does not determine conceptual compatibility. State prospectively which contrasts can be combined and which require separate synthesis.

Audit protocol amendments

Changes discovered during review work are common. Record when eligibility, outcomes, search sources or analysis changed, what new information triggered the change and whether study results were already known. Transparency is preferable to pretending the protocol was perfect from the start.

A protocol reduces researcher degrees of freedom in evidence synthesisMMed extension

Systematic reviews are sometimes treated as secondary work that does not need the same prespecification as primary research. In fact, reviewers face many choices after seeing study characteristics and effects. Protocols and prospective registration reduce the opportunity for selective inclusion, outcome switching and result-driven subgroup analysis.

This does not mean protocols must prohibit adaptation. Searches often reveal terminology or evidence types that were not anticipated. A defensible review preserves the original plan, records amendments and explains how the change affects interpretation. For living or rapidly updating reviews, change control becomes even more important because methods may need to evolve while maintaining continuity across versions.

Advanced protocol decisions for evidence synthesis

A high-quality review protocol specifies not only what will be searched but how evidence units will be interpreted. Primary studies may report multiple outcomes, time points, adjusted and unadjusted models, subgroup estimates and several publications. The protocol should establish a hierarchy for selecting among eligible results or a strategy for retaining multiple dependent estimates. Without such rules, reviewers can unconsciously select the most favourable or convenient result from each study.

Outcome definitions should anticipate multiplicity within studies

If the review concerns depression, studies may report several scales, multiple thresholds and repeated follow-up times. Prespecify the outcome domain, preferred measurement instruments if scientifically justified, and time windows. A hierarchy can avoid selecting whichever scale yields the largest effect. Alternatively, multivariate methods may retain several outcomes when the review team has appropriate expertise.

Search updates are part of the protocol

A search completed a year before submission may no longer represent current evidence. Specify when the search will be rerun and how newly identified records are integrated. Record exact dates for every source. For living reviews, define the surveillance interval and criteria for updating synthesis or publication.

Automation requires validation and transparency

Machine-learning prioritisation, deduplication software and AI-assisted screening can improve efficiency, but the protocol should define how tools are used, what human oversight remains and how false exclusions are prevented. Record software version and settings where they affect reproducibility. Automation does not remove reviewer responsibility for eligibility decisions.

Risk-of-bias appraisal should match the result being synthesised

Choose the appraisal tool according to study design and target result. A systematic review that includes randomised and non-randomised studies may require different tools. Do not collapse domain-based judgements into an unvalidated numeric “quality score” unless the method specifically requires it. Plan how risk of bias influences synthesis, sensitivity analysis and certainty assessment.

Meta-analysis requires a target quantity

Before selecting fixed or random effects, define the effect measure and what variation across studies means. Odds ratios, risk ratios and hazard ratios are not interchangeable. Change scores and post-treatment scores may target different quantities depending on design and adjustment. For continuous outcomes measured by different instruments, standardised effects may facilitate pooling but reduce clinical interpretability. These decisions belong in the protocol.

Random-effects models do not solve conceptual heterogeneity

A random-effects model allows true effects to vary statistically, but it does not justify pooling studies that answer different clinical questions. Assess compatibility of population, intervention, comparator, outcome definition and follow-up before modelling. If the studies are too diverse, structured narrative synthesis or separate meta-analyses may be more defensible.

Small numbers of studies require cautious inference

With few studies, heterogeneity estimates are imprecise, meta-regression is unstable and funnel-plot asymmetry is difficult to interpret. Prespecify subgroup analyses sparingly and require substantive rationale. Avoid promising complex meta-regression in a field likely to contain only a handful of studies.

Scoping-review charting should be analytically purposeful

A scoping review can easily become a large spreadsheet of extracted details with no synthesis logic. Define categories that answer the mapping question. If the objective is to map digital recruitment methods, chart the recruitment stage, technology, target population, study design, implementation setting and evaluation outcome—not every demographic detail reported in every paper.

Critical appraisal in scoping reviews should have a reason

JBI guidance allows flexibility according to purpose. Appraisal may be useful when the scoping review aims to identify whether a field contains sufficiently robust evidence for guidelines or a later systematic review. If the purpose is simply to map terminology or methods, formal risk-of-bias assessment may add burden without changing the map. State the purpose prospectively.

Protocol publication and registry records should remain reconciled

A journal protocol, PROSPERO record and repository registration can coexist. They should agree on the core question, eligibility and outcomes. When the protocol changes, update the relevant registry where possible and explain discrepancies in the final review. Multiple public records strengthen transparency only if they do not tell contradictory stories.

Final protocol decisions before registration

Before registering or publishing the review protocol, simulate the review workflow using a small set of known eligible, borderline and clearly ineligible records. Confirm that two reviewers could apply the eligibility criteria consistently, that the proposed extraction fields can capture the important study variations, and that the planned synthesis can answer the stated question. This rehearsal is not an opportunity to redesign the protocol around desired results; it is a feasibility check that identifies ambiguous rules before formal screening begins.

The protocol should also define how unavoidable amendments will be handled. Record the date, rationale, stage of the review and likely analytical consequence of each change. Maintaining reconciliation between the protocol, registry record, review-management system and final manuscript makes later deviations visible and interpretable rather than hidden within the final report.

Designing the protocol for later auditability

Create a prospective decision log alongside the protocol. Record search-strategy versions, screening-rule clarifications, amendments to extraction fields, newly discovered outcome variants and decisions about combining studies. The log should distinguish clarifications that implement the existing protocol from substantive changes that alter eligibility or synthesis. This record becomes the basis for the amendments section of the final review.

For living or periodically updated reviews, prespecify surveillance frequency, triggers for a full search update, criteria for incorporating new evidence and responsibility for maintaining the review. Without such rules, an “updated review” can quietly change its methods across versions, making changes in conclusions difficult to separate from changes in procedure.

The protocol should also state who adjudicates unresolved reviewer disagreements and whether authors of included studies may be contacted for missing information. These apparently small operational rules affect reproducibility because they determine how ambiguous evidence enters the final synthesis.

Worked example

WORKED EXAMPLE

Reframing the digital recruitment literature as a scoping review

The clinical research unit’s preliminary search finds randomised reminders, observational platform evaluations, qualitative studies of participant preferences, implementation case studies and descriptions of recruitment algorithms. A single pooled effectiveness estimate would combine fundamentally different interventions and outcomes. The team therefore reframes the first project as a scoping review mapping digital recruitment and retention approaches, populations, outcome categories, implementation settings and evaluation designs.

Using Population–Concept–Context, they define clinical-trial participants as the population, digital recruitment/retention strategies as the concept and interventional health research as the context. The protocol specifies databases, citation searching, trial registries, deduplication, dual screening and a charting form. They plan to classify whether each technology targets identification, invitation, consent, retention or follow-up and to map which approaches have comparative effectiveness evidence.

The completed map can then identify a narrower question—for example, automated messaging to improve retention in outpatient trials—that may support a separate focused systematic review and meta-analysis. The scoping review is not an inferior substitute; it is the correct first design for the broader objective.

Suggested protocol and manuscript wording

Common mistakes

Knowledge check

When is a scoping review particularly useful?

Answer and reasoning

When the objective is to map a broad or heterogeneous evidence field, clarify concepts, describe evidence types or identify where focused synthesis may be possible.

Does every systematic review require meta-analysis?

Answer and reasoning

No. Systematic review refers to the methods of evidence identification and synthesis. Meta-analysis is appropriate only when the included evidence supports meaningful quantitative pooling.

What is the role of PRISMA-P?

Answer and reasoning

It provides minimum reporting items for systematic review and meta-analysis protocols, improving transparency of the prospective plan.

Why link multiple reports to one study identifier?

Answer and reasoning

Because protocols, abstracts and secondary papers may describe the same participants; treating them as independent studies can double count evidence.

Key take-home messages

SUMMARY

Key take-home messages

• Select systematic versus scoping review from the purpose of synthesis.
• Protocolise eligibility, searching, screening and extraction before study results drive decisions.
• Use PRISMA-P for systematic review protocol reporting and current JBI guidance for scoping-review conduct.
• Do not force meta-analysis when clinical or methodological compatibility is poor.
• Register or persistently deposit the protocol and report amendments transparently.

References

1. Moher D, Shamseer L, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1. https://doi.org/10.1186/2046-4053-4-1
2. Shamseer L, Moher D, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;349:g7647. https://doi.org/10.1136/bmj.g7647
3. Peters MDJ, Godfrey C, McInerney P, et al. Chapter 10: Scoping reviews. In: JBI Manual for Evidence Synthesis. JBI; 2024. https://doi.org/10.46658/JBIMES-24-09
4. Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–473. https://doi.org/10.7326/M18-0850
5. Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.5.1. Cochrane; 2026. https://www.cochrane.org/handbook
6. Centre for Reviews and Dissemination, University of York. PROSPERO: International prospective register of systematic reviews. https://www.crd.york.ac.uk/prospero/