Quality and Reporting in Mixed-Methods Research

Overview

A mixed-methods study must succeed at three levels: the quantitative component, the qualitative component, and the integration between them. Strong integration cannot rescue biased quantitative measurement or superficial qualitative analysis; equally, two excellent component studies do not automatically produce a strong mixed-methods study if their relationship is unclear.[1,2]

Quality appraisal therefore asks more than whether “both methods were used”. Researchers should examine whether the mixed-methods rationale is justified, whether the design matches that rationale, whether each component is rigorous, whether integration occurs at appropriate points, whether discrepancies are handled transparently, and whether the resulting meta-inferences are supported by the combined evidence.[1]

Reporting is related to, but distinct from, methodological quality. A well-conducted study can be reported poorly, making appraisal impossible. Conversely, a manuscript can report a weak design very clearly. Reporting guidelines improve transparency; they do not certify validity.

Why this matters

Mixed-methods reports often devote substantial space to each component but very little to integration. Readers may see a table of regression results, several qualitative themes and a discussion that asserts the methods “triangulated” without explaining what was actually compared. This hides the main methodological contribution of mixed methods.

The Good Reporting of A Mixed Methods Study (GRAMMS) framework was introduced in 2008 to improve reporting in health-services research and includes six broad reporting domains covering rationale, design, component methods, integration, limitations created by mixing, and insights gained from integration.[2] The field has advanced substantially since then. A formal project to develop GRAMMS 2.0 published its methodological-review and modified-Delphi protocol in March 2026.[3] As of 19 August 2026, EQUATOR lists GRAMMS 2.0 as a reporting guideline under development, so researchers should not present a final GRAMMS 2.0 checklist as though it were already published.[4]

Key concepts

Quality is multidimensional

O’Cathain and colleagues emphasised appraisal of the success of the overall study, the mixed-methods design, the individual components, integration and inferences.[2] A useful practical structure is:

1. Question and rationale quality — Is mixed methods genuinely needed?
2. Design quality — Does timing, priority and sample relationship fit the purpose?
3. Component quality — Are the quantitative and qualitative methods independently defensible?
4. Integration quality — Are the strands connected, built, merged or embedded in a way that answers the mixed-methods question?
5. Inference quality — Are meta-inferences supported, appropriately bounded and transparent about uncertainty?
6. Reporting quality — Is there enough information for a reader to understand and appraise all of the above?

Reporting quality is not risk of bias

A reporting checklist asks whether key information is present. A critical-appraisal tool asks whether methods are likely to produce credible findings. These functions overlap but are not equivalent.

The Mixed Methods Appraisal Tool (MMAT) version 2018 was designed to appraise qualitative, quantitative and mixed-methods studies in systematic mixed-studies reviews.[5] It includes design-specific criteria and mixed-methods criteria. It should not be reduced to a simple overall percentage score when the tool’s guidance discourages uncritical scoring; item-level judgements are more informative for identifying where concerns arise.

Mixed-methods added value

A mixed-methods report should identify what was learned from integration that would not have been learned from the separate studies alone. This may include:

• explaining why a quantitative pattern occurred;
• discovering that a structured measure under-represented participants’ experience;
• identifying a subgroup or mechanism for further testing;
• revealing that implementation context changed the meaning of an average intervention effect; or
• showing that apparently conflicting results concern different levels or constructs.

If the final conclusion is simply “the survey found X and interviews found Y”, the added value may be limited.

Appraising the rationale and design

DECISION GUIDE

Start appraisal with the mixed-methods purpose

Ask in order:

1. What could not be answered adequately with one methodological approach?
2. Why was this specific mixed-methods design selected?
3. What are the timing and relative priority of the strands?
4. How do the samples relate?
5. Where does integration occur?
6. What meta-inference is expected from integration?

If these questions cannot be answered from the protocol or report, the study’s mixed-methods logic is difficult to appraise.

A design label should not substitute for description. “Convergent”, “explanatory sequential” and “embedded” are useful only when the actual procedures match the design logic.

Appraising component quality

Quantitative component

The relevant criteria depend on the quantitative design. Appraisal may need to consider:

• sampling and representativeness;
• exposure, outcome or intervention measurement;
• confounding and causal assumptions;
• randomisation and allocation concealment in trials;
• loss to follow-up and missing data;
• clustered or repeated observations;
• model specification and uncertainty; and
• whether inference matches the study design.

Qualitative component

Appraisal should consider:

• alignment between question, paradigm and methodology;
• sampling strategy and information richness;
• appropriateness of data collection;
• analytic depth and transparency;
• reflexivity and researcher positioning;
• evidence linking interpretations to data; and
• attention to negative, contradictory or diverse cases.

A mixed-methods label does not relax these standards.

Appraising integration

Fetters and colleagues describe integration across design, methods and interpretation/reporting.[6] Joint displays can make integration visible and support analytic reasoning.[7]

APPRAISAL CHECKLIST

Appraise the mixed-methods integration

• Is the integration purpose stated before the results?
• Are the relevant constructs, samples and levels sufficiently aligned?
• Is the integration mechanism explicit: connecting, building, merging or embedding?
• Can the reader see how one strand influenced another where claimed?
• Are component findings preserved with their uncertainty and context?
• Are contradictions investigated rather than suppressed?
• Does the meta-inference follow from the combined evidence?
• Is the scope of the meta-inference no broader than the data support?

Fit between strands

Fit refers to how the findings relate when integrated. They may converge, complement, expand or diverge. Poor fit is not automatically poor quality. The quality question is whether the researchers recognised and investigated the relationship appropriately.

For example, a survey may show that 85% of women report receiving discharge advice, while interviews reveal widespread uncertainty about what action to take. These findings may appear contradictory only if “received advice” is assumed to equal “actionable understanding”. Integration may reveal a measurement limitation rather than an irreconcilable conflict.

Meta-inference credibility

A meta-inference should identify which strands support it, what uncertainty remains, and whether it is descriptive, explanatory or causal.

A statement such as “younger women miss testing because they do not prioritise their health” would be an overreach if age is observationally associated with completion and interviews identify multiple structural constraints. A more defensible meta-inference might be that lower completion among younger women is associated with a combination of work transition, childcare and service-access conditions that warrant targeted investigation.

Reporting with GRAMMS

The original GRAMMS framework identifies six broad elements.[2]

1. Justify mixed methods

Explain why the research question requires both qualitative and quantitative approaches. Avoid generic statements such as “to obtain a comprehensive picture” unless the specific added value is defined.

2. Describe the mixed-methods design

State purpose, priority and sequence. Describe whether the design is convergent, sequential, embedded or multiphase, but also explain the actual procedures.

3. Describe each method adequately

Each component requires enough methodological detail to appraise sampling, data collection and analysis.

4. Describe integration

State where integration occurred, how it was performed and who participated in the interpretive process where relevant.

5. Describe limitations associated with mixing

Examples include sample mismatch, timing problems, burden, contamination, unequal strand quality and difficulties resolving discordance.

6. Describe insights gained from mixing

Make the mixed-methods contribution explicit.

GRAMMS 2.0 is an active update, not yet a final checklistMMed extension

The 2026 GRAMMS 2.0 protocol describes a methodological review followed by a modified Delphi process and consensus work to update reporting guidance.[3] EQUATOR currently lists the project under reporting guidelines in development.[4] Until the final guideline is published, use the original GRAMMS framework alongside design-specific reporting guidance and current mixed-methods methodological literature rather than citing an unpublished final GRAMMS 2.0 checklist.

Reporting the protocol

Prospective reporting is especially important because integration decisions are vulnerable to post hoc storytelling. A mixed-methods protocol should specify:

• the overall mixed-methods question and rationale;
• component research questions;
• design type and sequence;
• priority of strands where relevant;
• sampling frame and sample relationship;
• data-collection timing;
• component analysis plans;
• integration points and procedures;
• planned handling of divergence;
• governance of linked data; and
• who will contribute to integrated interpretation.

PROTOCOL CHECKLIST

Minimum mixed-methods protocol decisions

• Why are both qualitative and quantitative methods necessary?
• What exactly will each component answer?
• Which component occurs first, or are they concurrent?
• How will samples relate?
• Which findings will be connected, built, merged or embedded?
• What will the joint display or integration matrix contain?
• How will contradictory findings be examined?
• What criteria will be used to judge each component’s adequacy?
• What will count as the mixed-methods added value?

Reporting the results

A clear results structure can follow one of several approaches.

Separate component results followed by integration

This is straightforward when component analyses are substantial. The manuscript includes quantitative results, qualitative findings, then a dedicated mixed-methods integration section.

Weaving by domain

Quantitative and qualitative evidence are presented together under substantive domains such as access, communication and continuity. This can make integration more visible but requires careful signposting so readers can identify the source and method of each finding.

Joint-display-centred reporting

A joint display becomes the organising structure for integrated results. This is useful when the study’s main contribution lies in comparing or explaining findings across strands.[7]

Regardless of structure, do not let the qualitative component disappear into a few quotations or the quantitative component into p-values without estimates and uncertainty.

Reporting architecture for a mixed-methods manuscript

A transparent paper should make the mixed-methods logic visible from the beginning rather than revealing it only in the discussion. The title and abstract should identify the study as mixed methods when that feature is central and briefly state the design and integration purpose. The introduction should explain why one methodological approach would have been insufficient.

In the Methods, a dedicated mixed-methods subsection can describe the overall design, sequence, priority, sample relationship and integration plan before presenting the quantitative and qualitative component methods. This avoids forcing readers to infer the design from two separate method sections. If a joint display was planned prospectively, state its organising domains and how disagreements were adjudicated or retained.

In the Results, make participant flow transparent across components. Readers should be able to see how many people contributed to the quantitative strand, qualitative strand and linked subset where relevant. When the sample relationship is nested, report how the qualitative subsample was selected from quantitative profiles. When samples are parallel, report the population relationship and avoid implying participant-level linkage.

The Discussion should distinguish component-level conclusions from meta-inferences. A useful structure is to state the principal integrated finding, explain what each strand contributed, discuss discordant evidence, and then describe implications that genuinely follow from integration. Limitations should include mixed-methods limitations—such as sample mismatch, timing or unequal strand quality—not only the limitations of the separate components.

Finally, supplementary material can improve auditability. A joint display, integration matrix, development trail or sampling connection diagram may be too detailed for the main manuscript but can show exactly how evidence was combined. Transparency is particularly important when integration involved several interpretive decisions by a multidisciplinary team.

Bias, validity and quality assurance

Other quality threats include:

• design-label inflation: using sophisticated labels without corresponding procedures;
• methodological tokenism: one strand lacks adequate sampling or analysis;
• team silos: qualitative and quantitative researchers work independently until manuscript drafting;
• premature integration: qualitative data are reduced before proper qualitative analysis;
• unequal uncertainty: statistical confidence intervals are reported but qualitative interpretive uncertainty is ignored, or vice versa;
• generalisation creep: integrated conclusions exceed the population or contexts represented; and
• reporting substitution: a checklist is used as evidence that the study is methodologically valid.

Team science and reflexivity

Mixed-methods studies often require interdisciplinary teams. Different methodological traditions may use terms such as validity, generalisability, saturation, reflexivity and inference differently. Productive integration requires enough shared methodological language to discuss these differences without forcing one tradition to dominate.[8]

Reflexivity should include the integration process itself. Who decided which findings were prioritised? Did quantitative investigators treat qualitative contradictions as “anecdotal”? Did qualitative investigators reject useful quantitative patterns because they appeared reductive? Were patient or stakeholder interpretations included? These decisions can shape the meta-inference.

Interface with statistical analysis

Statistical methods should be reported according to the quantitative design and not abbreviated merely because the study is mixed methods. Estimates, confidence intervals, denominators, missing data and model assumptions remain important.

Integration should not use statistical significance as the gatekeeper for qualitative discussion. A non-significant but clinically important estimate may merit qualitative exploration; a significant association may be too small to matter. Similarly, qualitative frequency should not be interpreted as a probability estimate unless the sampling supports that inference.

When integrated conclusions depend on subgroup patterns, the quantitative uncertainty and multiplicity of exploratory subgroup analysis should be acknowledged.

Worked example

WORKED EXAMPLE

Appraising a weakly integrated convergent study

Reported study: A 600-participant survey and 24 interviews examine postpartum testing. Both components are individually described adequately.

Problem 1 — unclear sample relationship: The manuscript does not state whether interview participants came from the survey.

Problem 2 — undefined integration: The authors say findings were “triangulated” but provide no integration procedure.

Problem 3 — selective agreement: The discussion cites interview quotations that support significant survey associations but ignores a qualitative theme indicating that high risk knowledge did not overcome practical barriers.

Problem 4 — no added-value statement: Conclusions repeat the survey findings and recommend education.

Improved integration: The authors clarify that interviews were a purposive nested sample, construct a joint display, retain the discordant knowledge finding, and revise the meta-inference: informational deficits explain some non-completion, but practical coordination and service continuity remain important even among knowledgeable women.

The example is illustrative.

Suggested protocol or reporting wording

Common mistakes

Knowledge check

Can a mixed-methods study be high quality if one component is methodologically weak?

Answer and reasoning

A meta-inference that depends materially on the weak component will have limited credibility. The strengths of the other component do not automatically compensate. Quality should be judged in relation to the specific conclusion and the evidence supporting it.

What is the difference between GRAMMS and MMAT?

Answer and reasoning

GRAMMS is a reporting framework intended to improve transparency in mixed-methods reports. MMAT is a critical-appraisal tool designed to assess methodological quality of studies in systematic mixed-studies reviews. Reporting completeness and methodological quality are related but different.

As of 20 August 2026, should researchers use a final GRAMMS 2.0 checklist?

Answer and reasoning

No final GRAMMS 2.0 checklist can yet be treated as established guidance. A protocol for its development was published in March 2026, and EQUATOR lists the update as under development.[3,4]

Why is discordance not automatically evidence of poor mixed-methods quality?

Answer and reasoning

Different methods may capture different constructs, levels or contexts. High-quality integration examines why findings differ and may generate a more useful meta-inference from the discrepancy rather than forcing agreement.

What should a mixed-methods results section add beyond separate component results?

Answer and reasoning

It should show the integration process and the resulting meta-inference: what the combined evidence reveals, explains, challenges or changes compared with reading the components separately.

Key take-home messages

SUMMARY

Key take-home messages

• Appraise mixed-methods research at the component, integration and meta-inference levels.
• Reporting completeness does not equal methodological validity.
• Use GRAMMS to structure transparent reporting while recognising that GRAMMS 2.0 is still under development as of August 2026.
• Preserve contradictory evidence and show how it affects the integrated conclusion.
• Report the mixed-methods added value explicitly.
• Protect linked qualitative and quantitative data from deductive disclosure.

References

1. Creswell JW, Klassen AC, Plano Clark VL, Smith KC. Best practices for mixed methods research in the health sciences. Bethesda (MD): National Institutes of Health, Office of Behavioral and Social Sciences Research; 2011, revised 2018. https://obssr.od.nih.gov/sites/g/files/mnhszr296/files/Best_Practices_for_Mixed_Methods_Research.pdf
2. O’Cathain A, Murphy E, Nicholl J. The quality of mixed methods studies in health services research. J Health Serv Res Policy. 2008;13(2):92–98. https://doi.org/10.1258/jhsrp.2007.007074
3. Munce S, Fàbregues S, Hong QN, O’Cathain A, Guetterman T, Creswell J, et al. Updating the Good Reporting of a Mixed Methods Study (GRAMMS) reporting guidelines: protocol for a methodological review and modified Delphi process. JMIR Res Protoc. 2026;15:e82364. https://doi.org/10.2196/82364
4. EQUATOR Network. Reporting guidelines under development for other study designs: GRAMMS 2.0. Oxford: EQUATOR Network; 2026. Accessed 2026-08-19. https://www.equator-network.org/library/reporting-guidelines-under-development/reporting-guidelines-under-development-for-other-study-designs/
5. Hong QN, Fàbregues S, Bartlett G, Boardman F, Cargo M, Dagenais P, et al. The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers. Educ Inf. 2018;34(4):285–291. https://doi.org/10.3233/EFI-180221
6. Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs—principles and practices. Health Serv Res. 2013;48(6 Pt 2):2134–2156. https://doi.org/10.1111/1475-6773.12117
7. Guetterman TC, Fetters MD, Creswell JW. Integrating quantitative and qualitative results in health science mixed methods research through joint displays. Ann Fam Med. 2015;13(6):554–561. https://doi.org/10.1370/afm.1865
8. O’Cathain A, Murphy E, Nicholl J. Multidisciplinary, interdisciplinary, or dysfunctional? Team working in mixed-methods research. Qual Health Res. 2008;18(11):1574–1585. https://doi.org/10.1177/1049732308325535