Implementation, Health-Services and Quality-Improvement Research

Overview

Health systems frequently know what should be done but struggle with how to deliver it reliably, equitably and sustainably. Implementation research studies strategies that promote uptake or integration of evidence-based interventions. Health-services research examines organisation, access, financing, quality, utilisation and outcomes of care. Quality-improvement (QI) work uses systematic methods to improve local care processes and can itself generate generalisable knowledge when studied rigorously. Current frameworks and reporting standards include the updated CFIR, StaRI, SQUIRE 2.0, implementation-outcome taxonomies and RE-AIM.[1–5]

These approaches overlap but are not synonyms. A project can contain a clinical intervention, an implementation strategy and a QI cycle at the same time. The research question should identify which layer is the object of inference. Practical implementation-research guidance likewise emphasises designing the study around the real implementation problem, context and intended use of findings.[6]

Why this matters

If the layers are confused, investigators may conclude that an intervention “does not work” when the real problem is poor implementation, or claim that an implementation strategy is effective when improvement simply reflects a secular trend. A local QI project may demonstrate better performance in one ward without showing that the change caused the improvement or can transfer elsewhere.

Clear distinction also affects outcomes. Clinical effectiveness may be measured by mortality or symptoms; implementation success may be measured by adoption, fidelity, acceptability, feasibility, penetration, cost or sustainability; health-services outcomes may include access, efficiency, equity and patient experience. The evaluation should measure outcomes at the level of the stated question.

Key concepts

An evidence-based intervention is the clinical or public-health practice intended to improve health. An implementation strategy is the method used to promote adoption, fidelity or sustainment of that intervention. Keeping them separate is essential: “staff education” may be the implementation strategy, while “early mobilisation” is the clinical intervention. StaRI specifically encourages reporting both.[2]

A determinant framework such as CFIR helps identify barriers and facilitators across innovation, inner setting, outer setting, individuals and implementation processes.[1] It does not by itself specify outcomes or causal effects. Implementation outcome frameworks provide constructs such as acceptability, adoption, appropriateness, feasibility, fidelity, cost, penetration and sustainability.[4] Evaluation frameworks such as RE-AIM examine reach, effectiveness, adoption, implementation and maintenance.[5]

QI often uses iterative cycles because interventions are adapted to local workflows. Iteration is not incompatible with rigorous evaluation, but changes should be documented with dates and rationale. Context is not noise; staffing, leadership, competing initiatives and policy changes can explain why the same strategy works differently across settings.

DECISION GUIDE

Classify the project by its primary question

• Is the main question whether a clinical intervention improves health outcomes?
• Is the main question how to increase adoption, fidelity, reach or sustainment?
• Is the main question how services are organised, accessed, financed or experienced?
• Is the main aim rapid local improvement with iterative adaptation?
• Is the project also intended to generate transferable knowledge requiring a research evaluation design?
• What unit—patient, clinician, ward, organisation or system—matches the main outcome?

A step-by-step methodological workflow

1. Define the evidence-to-practice gap

Quantify the gap using local data and verify that the target clinical practice is supported by adequate evidence. A low rate is not automatically a problem if eligibility is uncertain or patient preferences justify variation. Specify who should receive the intervention, under what circumstances and what constitutes appropriate delivery.

2. Specify intervention and implementation strategy separately

Describe the clinical intervention using enough detail to assess fidelity. Then describe implementation strategies: education, audit and feedback, reminders, facilitation, workflow redesign, incentives or other components. State who delivers each strategy, to whom, at what intensity and with what adaptation rules.

3. Use theory or frameworks deliberately

Choose a framework for a defined purpose. CFIR may guide determinant assessment; Proctor-type outcomes may define implementation success; RE-AIM may structure population impact. Avoid “framework stacking” in which multiple frameworks are named without changing data collection, analysis or interpretation.

4. Choose an evaluation design that addresses time and context

Randomised or cluster-randomised designs may be feasible for some implementation strategies. Quasi-experimental designs such as controlled interrupted time series or difference-in-differences may be appropriate when randomisation is impossible. Simple before-after comparisons are vulnerable to secular trends, regression to the mean and co-interventions.

5. Measure implementation process and patient outcomes

Collect reach, adoption, fidelity and acceptability alongside clinical outcomes when both matter. Process measures help explain null or heterogeneous effects. Equity should be considered explicitly: an intervention can improve average uptake while widening gaps between patient groups or facilities.

6. Document adaptation, context and sustainment

Record what changed during implementation, why, and whether the core function of the intervention was preserved. Assess maintenance after initial support is withdrawn. Report contextual events such as staffing shortages or policy changes that could explain results. Use StaRI for implementation studies and SQUIRE 2.0 when reporting systematic healthcare-improvement work, according to scope.[2,3]

PROTOCOL CHECKLIST

Implementation/QI protocol items

• evidence-practice gap and target population;
• clinical intervention and implementation strategy as separate objects;
• framework purpose and selected constructs;
• unit of implementation and unit of analysis;
• implementation, service and clinical outcomes;
• evaluation design and comparator or counterfactual;
• context, adaptation and fidelity tracking;
• equity, cost and sustainment where relevant;
• reporting framework appropriate to the project.

An auditable planning document

Intervention–implementation separation matrix

illustrative

Table 1. Illustrative early-mobilisation evaluation

Layer	Example	Outcome
Clinical intervention	Mobilisation within 24 hours when eligible	Functional recovery
Implementation strategy	Education + prompts + scheduling redesign + feedback	Adoption/fidelity
Context	Ward staffing, surgical volume, physiotherapy availability	Moderator/determinant
Reach	Proportion of eligible patients assessed	Equity/coverage
Sustainment	Delivery six months after active support	Maintenance

Note: The layers may interact but should remain conceptually distinguishable.

Quality, validity and governance

Implementation and QI studies are especially vulnerable to history and co-intervention. A new staffing policy, seasonal workload change or simultaneous safety initiative can produce improvement that coincides with the intervention. Repeated pre- and post-intervention measurements, control sites or phased implementation can strengthen causal interpretation.

Fidelity should be interpreted carefully. High fidelity to the core function may coexist with adaptation of the form to local context. Treating every adaptation as protocol failure can make an intervention unusable; allowing unlimited adaptation can make it impossible to know what was implemented. The protocol should specify what must remain stable and what may be adapted.

Quality-improvement teams should also distinguish operational monitoring from research inference. Run charts can guide local decisions quickly, while a formal interrupted time-series model may be needed to estimate a change in level or trend. The same dataset can support both purposes, but conclusions should match the method.

APPRAISAL CHECKLIST

Appraise implementation and improvement research

• Is the clinical intervention distinct from the implementation strategy?
• Does the selected framework have an explicit purpose in design or analysis?
• Is the comparison sufficient to separate change from secular trends?
• Are implementation outcomes measured in addition to clinical outcomes when relevant?
• Are context, adaptation, fidelity and equity reported?
• Does the reporting guideline match implementation research or QI rather than being selected by habit?

Practical decision audit

Audit the counterfactual

Ask what would likely have happened without the implementation strategy. If the only evidence is a pre-post difference in one ward, alternative explanations may dominate. Consider concurrent controls, staggered rollout, interrupted time series or randomisation when feasible.

Audit implementation dose

Education may be offered but not attended; prompts may fire but be ignored. Record exposure to the implementation strategy and delivery of the clinical intervention separately. This helps distinguish strategy failure from intervention failure.

Audit equity and reach

Overall adoption can improve because well-resourced wards improve rapidly while disadvantaged wards remain unchanged. Stratify reach and uptake by relevant populations or settings and investigate structural barriers rather than interpreting average improvement as universal benefit.

Audit sustainability

Short-term performance during intensive project support may not persist. Define a sustainment period, identify which resources continue, and measure whether delivery remains acceptable and feasible once temporary facilitation or funding ends.

Hybrid effectiveness–implementation designs answer two layers at onceMMed extension

Some studies intentionally evaluate both clinical effectiveness and implementation. Hybrid designs vary in emphasis: one may primarily test clinical effectiveness while gathering implementation information, another may test both, and another may primarily test implementation while monitoring clinical outcomes. The choice should follow the maturity of evidence and the decision that stakeholders face.

Hybrid does not mean “measure everything”. The study still needs clear primary questions, sufficient units at the relevant level, and an analysis that respects clustering and implementation structure. Process evaluation is valuable when it helps explain mechanisms and context rather than becoming a descriptive appendix disconnected from the main trial.

Designing studies for complex change in real systems

Implementation and QI interventions are usually complex, meaning that outcomes depend on interactions between components, people and context. A checklist reminder may work only when staffing permits action; audit and feedback may work differently when clinicians trust the data source; a new pathway may fail because pharmacy hours do not match clinic flow. The study should therefore specify the hypothesised mechanism between strategy and outcome, not merely list intervention components.

Logic models can connect strategy, mechanism and outcome

A practical logic model links resources and strategies to proximal mechanisms, implementation outcomes and downstream clinical outcomes. For early mobilisation, education may increase knowledge, prompts may increase recognition of eligible patients, scheduling redesign may increase physiotherapy availability, and feedback may reinforce behaviour. Measuring only the final mobilisation rate cannot distinguish which mechanism failed. A small set of mechanism-aligned process measures is more useful than an indiscriminate dashboard.

Framework selection should follow the question stage

Different frameworks serve different phases. A determinant framework can diagnose barriers before implementation. A process model can guide implementation steps. An outcome framework defines what success looks like. An evaluation framework integrates reach and maintenance. Using CFIR to define every component of an outcome evaluation can stretch the framework beyond its intended purpose. State why each framework is used and which constructs are measured.

Clustered implementation creates analytical dependence

Implementation strategies are often delivered to wards, clinics or clinicians while outcomes are measured in patients. The statistical analysis must respect clustering and the number of implementation units. A study with thousands of patient records but only six wards may have limited information for ward-level effects. Sample-size planning should reflect the unit of assignment and intracluster correlation where relevant.

Interrupted time series requires enough time points and stable measurement

A single pre and post mean does not constitute a strong time-series evaluation. Collect repeated observations before and after implementation, define the intervention point, assess underlying trend and consider seasonality or autocorrelation. If the measurement definition changes at implementation—for example because the intervention itself improves documentation—an apparent outcome improvement may partly reflect ascertainment.

Difference-in-differences requires a credible comparison trend

When comparing an intervention site with a control site, the pre-intervention trends should support the plausibility of a parallel-trends assumption. A control hospital chosen only because it has available data may have different policy, staffing or patient-mix trajectories. Plot pre-period trends and consider event-study style diagnostics where appropriate. Statistical adjustment cannot fully repair a fundamentally incomparable control.

Adaptation should be tracked with function and form

Implementation often requires local adaptation. Record what changed, who authorised it, why and whether the intended function was preserved. For example, a reminder’s function may be to prompt action before discharge; one ward may use EHR alerts while another uses a checklist. Treating these as identical without documenting delivery can obscure heterogeneity; treating them as protocol violations may prevent useful learning.

Sustainability should be designed, not merely observed later

Identify which strategy components depend on temporary project resources. If a dedicated facilitator disappears after funding ends, uptake may decline. Plan a sustainment phase in which routine staff and systems carry the intervention. Measure not only whether the practice persists but whether the burden remains acceptable and whether equity is maintained.

QI ethics and research ethics can overlap

Local improvement work may still create participant or staff risks, use identifiable data or generate generalisable knowledge. Institutional rules differ on when formal research-ethics review is required. Teams should not assume that labelling a project “QI” exempts it from governance. Seek local determination prospectively, particularly when randomisation, publication-oriented data collection or additional participant procedures are involved.

Economic consequences belong in implementation decisions

A strategy may improve uptake but require intensive facilitation that is unaffordable at scale. Record implementation costs and resource use when they affect decisions. Cost should be interpreted alongside reach, effectiveness and sustainment rather than as a detached finance appendix.

Choosing the counterfactual for real-world system change

Implementation and quality-improvement evaluations often fail because the intervention is described in detail while the counterfactual is left vague. Before choosing an interrupted time-series, controlled before-after, difference-in-differences, cluster-randomised or hybrid design, state what would plausibly have happened to the same outcome in the absence of the implementation strategy. Secular trends, policy changes, staffing disruption, seasonal demand and concurrent improvement initiatives can all produce changes that resemble an intervention effect.

For an interrupted time series, therefore, the intervention date, expected form of change, number and spacing of observations, outcome definition and important co-interventions should be prespecified. For difference-in-differences, the comparison group must represent a credible untreated trajectory; simply having another hospital or district is insufficient if its pre-intervention pattern and policy environment differ materially. Where randomisation is feasible, cluster assignment can strengthen causal interpretation but introduces dependence, recruitment and contamination issues that must be handled at design and analysis stages.

Implementation outcomes should also be linked explicitly to the hypothesised mechanism. Low fidelity may explain a null patient outcome, but high fidelity with poor reach may indicate a different implementation failure. Adoption, acceptability, feasibility, fidelity, cost, penetration and sustainability answer different questions and may require different respondents and time points. Equity should be assessed across reach and benefit: an intervention that improves the overall mean while systematically missing disadvantaged groups may widen disparities.

Finally, distinguish local learning from claims intended to generalise beyond the setting. Quality-improvement work can be highly valuable without making broad causal claims. When investigators intend to generate transferable knowledge, the protocol should describe context, theory, comparator logic, adaptation, outcome definitions and analytic methods with the same rigour expected of other health research.

Implementation research should explain failure as well as success

A useful implementation evaluation prespecifies how different patterns of results will be interpreted. If adoption increases but patient outcomes do not, possibilities include an ineffective clinical intervention, insufficient fidelity, delayed clinical effects or measurement limitations. If fidelity is high among participating wards but reach is poor, the strategy may work only for early adopters. If overall performance improves while the comparison site changes similarly, the apparent effect may reflect a system-wide policy rather than the local strategy.

Mixed quantitative and qualitative evidence can help distinguish these explanations when integration is planned rather than added retrospectively. Process measures can identify where the implementation pathway broke down, while interviews or observations can examine mechanisms and contextual barriers. The analysis should preserve discordant evidence instead of forcing all data sources into a single success narrative. Practical implementation-research guidance also emphasises that questions, context, stakeholder engagement and use of findings should be designed around the real implementation problem.[6]

Worked example

WORKED EXAMPLE

Evaluating early mobilisation across surgical wards

The team identifies early mobilisation as the clinical intervention and a four-component package—education, electronic prompts, physiotherapy scheduling and audit-feedback—as the implementation strategy. They use CFIR interviews and observations before rollout to identify barriers, but select only constructs relevant to staffing, workflow and leadership. Adoption and fidelity are primary implementation outcomes; time to functional recovery is a clinical outcome.

Because a hospital-wide simultaneous rollout would provide only a weak before-after comparison, the team stages implementation across wards and collects repeated weekly measures before and after each transition. They document a coincident staffing policy change and include it in interpretation. Reach is reported by age, language and ward to detect inequitable implementation.

At six months, uptake remains higher than baseline but has fallen after monthly feedback stopped. The report distinguishes the initial effect from sustainment and uses StaRI to describe the implementation strategy. A companion QI report uses SQUIRE principles to explain local adaptation and learning.

Suggested protocol and manuscript wording

Common mistakes

Knowledge check

What is the difference between an intervention and an implementation strategy?

Answer and reasoning

The intervention is the health practice intended to improve outcomes; the implementation strategy is the method used to promote its adoption, fidelity or sustainment.

What is CFIR primarily used for?

Answer and reasoning

It is a determinant framework used to understand factors that may influence implementation; it is not by itself an outcome measure or causal design.

Why can a before-after QI comparison be misleading?

Answer and reasoning

Because secular trends, regression to the mean, policy changes and co-interventions can produce change even without the QI intervention.

When might StaRI and SQUIRE both be relevant?

Answer and reasoning

When a project evaluates an implementation strategy and also reports systematic healthcare-improvement work. Their scopes differ, so authors should use each for the reporting purpose it was designed to support.

Key take-home messages

SUMMARY

Key take-home messages

• Separate clinical intervention, implementation strategy and context.
• Choose implementation frameworks for explicit purposes rather than naming them decoratively.
• Use evaluation designs that address secular trends and co-interventions.
• Measure reach, fidelity, adoption, equity and sustainment when they matter to the question.
• Match reporting guidance to implementation research, health-services research or QI work.

References

1. Damschroder LJ, Reardon CM, Widerquist MAO, Lowery J. The updated Consolidated Framework for Implementation Research based on user feedback. Implement Sci. 2022;17:75. https://doi.org/10.1186/s13012-022-01245-0
2. Pinnock H, Barwick M, Carpenter CR, et al. Standards for Reporting Implementation Studies (StaRI) statement. BMJ. 2017;356:i6795. https://doi.org/10.1136/bmj.i6795
3. Ogrinc G, Davies L, Goodman D, Batalden P, Davidoff F, Stevens D. SQUIRE 2.0: revised publication guidelines from a detailed consensus process. BMJ Qual Saf. 2016;25(12):986–992. https://doi.org/10.1136/bmjqs-2015-004411
4. Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76. https://doi.org/10.1007/s10488-010-0319-7
5. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322–1327. https://doi.org/10.2105/AJPH.89.9.1322
6. World Health Organization. Implementation research toolkit. Geneva: WHO; 2014. https://iris.who.int/handle/10665/110523