Data Quality Assurance and Monitoring

Overview

Data quality is not achieved by one final cleaning exercise. It is produced through a lifecycle of quality by design, standardised collection, real-time checks, monitoring, documented correction and reproducible dataset preparation. Current ICH E6(R3) guidance emphasises identifying factors critical to quality and applying proportionate, risk-based controls throughout study conduct.[1,2]

The appropriate quality system depends on the study. A low-risk single-centre survey may use structured forms, daily completeness review and duplicate checks. A multicentre clinical trial may add central monitoring, audit trails, source verification, safety reconciliation and formal data review. The principle is the same: direct effort towards errors that could materially affect participant protection or the credibility of conclusions.

Why this matters

A dataset can be internally tidy yet methodologically wrong. If exposure status was systematically misclassified, no amount of range checking repairs the bias. Conversely, an obsession with perfect completion of low-value fields can consume resources while critical outcomes remain poorly verified.

Quality assurance should therefore start with the research question and risk assessment, not with a generic list of database checks.

Quality assurance, quality control and monitoring

Terms are sometimes used inconsistently, but a practical distinction is:

• Quality assurance (QA): planned, systematic activities that create confidence that the study processes are appropriate and controlled.
• Quality control (QC): operational checks used to detect or prevent errors in specific data or procedures.
• Monitoring: ongoing oversight of study conduct and data to verify that required processes are being followed and important issues are identified.
• Data cleaning: review and transformation used to prepare collected data for analysis while resolving or documenting inconsistencies.

WHO’s data-quality review framework similarly treats quality as multidimensional rather than reducing it to error-free typing.[3]

Identify critical-to-quality data and processes

Start by asking what could change the study’s primary conclusion or harm participants. Critical items commonly include:

• informed consent and eligibility;
• primary outcome components;
• major exposure or intervention assignment;
• key time points;
• serious safety information where applicable;
• variables central to adjustment or stratification;
• identifiers required for correct linkage; and
• processes that prevent duplicate, ineligible or fabricated records.

Assign stronger controls to these than to low-impact exploratory variables. ICH guidance supports proportionate quality management based on the importance of errors rather than exhaustive verification of everything.[1]

Preventive controls

Prevention is usually cheaper and more reliable than correction. Examples include:

• clear operational definitions;
• standardised instruments and SOPs;
• staff competency assessment;
• appropriate field types and branching;
• programmed range and logic checks;
• controlled units;
• required fields only where justified;
• automatic timestamps or audit trails when appropriate;
• restricted edit permissions; and
• pilot testing before production.

A programmed check should correspond to a meaningful rule. Excessive alerts can cause warning fatigue and workarounds.

Real-time and near-real-time review

Some errors should be identified while they can still be corrected. Daily or weekly dashboards might track:

• missing primary outcomes;
• visits outside windows;
• unresolved queries;
• unusually fast interviews;
• duplicate identifiers;
• recruitment and withdrawal patterns;
• distributions of critical measurements; and
• site-specific error rates.

The purpose is not to punish sites. A quality indicator is a signal for investigation. Differences can arise from true case mix, workload or system configuration as well as performance.

DECISION GUIDE

Investigate an unusual data-quality signal

1. Verify that the indicator was calculated correctly.
2. Determine whether the pattern is participant-, collector-, site- or time-specific.
3. Review source data or workflow for representative cases.
4. Identify whether the cause is definition, training, form, system or environment.
5. Correct affected records when justified and traceable.
6. Implement preventive action for future records.
7. Re-measure the indicator to confirm that the problem has improved.

Queries and corrections

A data query should be specific and answerable. “Please check” is less useful than “Specimen date precedes delivery date; verify both dates against source and correct the erroneous field or explain.” Queries should not instruct staff to create a value that the source does not support.

Corrections require traceability. Electronic-source guidance emphasises preserving information about data originators and changes rather than silently replacing the historical record.[4] For paper systems, corrections should similarly preserve the original entry according to the study’s procedure.

Source-data verification and source-data review

Source-data verification (SDV) compares captured study data with source information for specified items. Source-data review is broader and evaluates whether source documentation supports the study process and protocol requirements. Not every study requires 100% SDV of every field.

Risk-based approaches focus verification on critical variables and processes, selected records or signals of concern. When source systems are electronic, the team also needs to understand how access, corrections and audit information are managed.[1,4,5]

Central and site-level monitoring

Central monitoring can identify patterns that are hard to see locally: unusual distributions, improbable uniformity, site differences, missingness, repeated values or inconsistent timing. Site-level review can inspect workflow and source context that central data cannot reveal. The two approaches complement one another.

For observational studies without formal trial monitoring, the same logic can be implemented as scheduled data-quality reviews. The terminology need not mimic regulatory trials, but the process should still be proportionate and documented.

Root-cause and corrective/preventive action

When a significant problem recurs, correcting individual records is insufficient. Root-cause analysis asks why the process allowed the error. Possible causes include:

• ambiguous definition;
• inadequate training;
• form design;
• incorrect branching;
• incompatible local workflow;
• device configuration;
• workload; or
• unclear responsibility.

A corrective action addresses the detected problem. A preventive action reduces recurrence. Document the action and verify whether it worked.

WORKED EXAMPLE

Resolving site-specific outcome misclassification

The central dashboard shows Site C has only 1% unknown postpartum test status versus 8–11% elsewhere. Review of 20 records finds that staff coded absence of a local result as “No test”.

Immediate correction: Re-review affected records using the source hierarchy and issue traceable corrections.

Root cause: The local quick-reference guide omitted the external-result verification step.

Preventive action: Replace the job aid, retrain the affected role and add an electronic prompt when “No local result” is selected.

Effectiveness check: Recalculate unknown-status proportions and audit a new sample after four weeks.

The aim is not to force Site C to match other sites but to confirm that all sites apply the same outcome definition.

Audit trails and data integrity

Electronic systems can record who created or changed a record, when the change occurred and sometimes why. Audit trails are valuable because they preserve traceability, but they are not self-interpreting. A large number of edits may reflect legitimate workflow; a suspiciously clean audit trail may reflect data being entered only after off-system manipulation.

Current FDA guidance on electronic records and signatures emphasises trustworthy, reliable systems and controls suited to their intended use.[5] The study should define authorised users, review significant corrections where necessary, and avoid shared accounts.

Database lock or finalisation

Before final analysis, resolve or formally close critical queries, complete prespecified reconciliations, review protocol deviations relevant to analysis, and document the dataset version. Not every minor discrepancy must be solved at any cost. Some values remain genuinely unknown; the important requirement is that unresolved uncertainty is represented honestly.

Finalisation should distinguish correction of errors from analytical recoding. Preserve collected values and document derivations. The Biostatistics lectures on data cleaning and missing data address downstream analytical handling.

PROTOCOL CHECKLIST

Data-quality monitoring plan

• Critical data and processes.
• Preventive controls and programmed checks.
• Quality indicators and review frequency.
• Responsibilities for central and site-level review.
• Query creation, response and escalation procedures.
• Scope of source-data review or verification where applicable.
• Root-cause and corrective/preventive action process.
• Audit-trail or correction review where relevant.
• Dataset finalisation and release criteria.
• Documentation of unresolved data limitations.

Data-quality dimensions are not interchangeableMMed extension

Completeness, accuracy, consistency, timeliness and uniqueness describe different aspects of quality. A variable can be 100% complete yet inaccurate; highly accurate values can arrive too late for a safety decision; internally consistent records can still be systematically biased.

Quality dashboards should therefore use measures linked to the study’s actual risks. A single overall “data quality score” can conceal important failures unless its construction and consequences are validated.

Designing useful quality indicators

A quality indicator should correspond to a plausible failure mechanism and lead to an action. “Percentage of all fields complete” may look impressive but can be dominated by low-value variables. More informative indicators might include proportion of primary outcomes verified, frequency of eligibility corrections after enrolment, median age of unresolved critical queries, or proportion of visits outside the prespecified window.

Specify the numerator, denominator, review frequency and escalation threshold. If denominators differ across sites because forms are not applicable to every participant, the indicator must account for that structure.

Look at distributions, not only error counts

Central review can identify unusual patterns even when every individual value passes validation. Examples include one site recording nearly identical blood pressures, an interviewer with implausibly short completion times, heaping at round numbers, a sudden shift after a software update or an unexpected absence of missing values.

These signals do not prove fabrication or poor practice. They identify records or workflows for further review. Statistical anomaly detection should therefore support human investigation rather than make accusations automatically.

Reconcile related data sources

Some studies hold related information in separate systems: enrolment logs, clinical records, laboratory files, safety systems or specimen databases. Define which elements require reconciliation. For example, the participant count in an analysis dataset should agree with the enrolment record after documented exclusions; critical laboratory results should not remain unlinked because one system uses a different identifier format.

Reconciliation rules should identify the authoritative source and how discrepancies are documented. Avoid “fixing” one system merely to make totals match when the reason for disagreement is not understood.

Quality review meetings

Regular data-review meetings are useful when they focus on decisions rather than presentation of dozens of metrics. A concise agenda can cover new critical issues, ageing unresolved queries, site trends, protocol deviations affecting data, system changes and effectiveness of previous corrective actions.

Maintain a decision log so the rationale for important data-handling choices remains available. This is especially helpful when the study later explains why a variable definition or cleaning rule changed.

Distinguish errors from genuine uncertainty

Quality processes should not convert unknown data into apparently complete data. If a source genuinely cannot establish an outcome, the correct result may remain unknown. Monitoring should confirm that the prescribed verification process was followed, not pressure sites to eliminate every missing value.

APPRAISAL CHECKLIST

Appraise a data-quality system

• Are indicators linked to critical variables and plausible failure modes?
• Are denominators and thresholds defined consistently?
• Does review examine patterns and distributions as well as individual errors?
• Are related data sources reconciled where disagreement matters?
• Are queries specific, traceable and independent of desired study findings?
• Are recurring problems investigated for root cause?
• Is effectiveness of corrective action reassessed?
• Can genuine uncertainty remain documented rather than being forced into a complete category?

Define when a query is considered closed

A query is not resolved merely because somebody typed a response. Closure criteria should require either a traceable correction supported by the source, confirmation that the original value is correct, or documentation that the uncertainty cannot be resolved. Critical unresolved queries may need investigator review before dataset finalisation.

Track query age as well as query count. A site with many quickly resolved queries may be functioning better than one with few queries that remain unanswered for months. Monitoring should therefore interpret indicators in context rather than ranking sites from a single metric.

Quality review should also document decisions not to change data. Preserving a plausible but unusual value after source confirmation is an important part of data integrity and prevents well-intentioned cleaning from narrowing genuine variation.

Common mistakes

Knowledge check

What is the difference between preventive quality control and data cleaning?

Answer and reasoning

Preventive control reduces the chance that an error is created, for example through clear definitions or validation logic. Data cleaning reviews data after capture to identify and resolve or document errors. Prevention is generally preferable when feasible.

Does every variable require source-data verification?

Answer and reasoning

No. Verification should be proportionate to the study and risk. Critical outcomes, eligibility and high-impact variables may warrant stronger verification than low-priority descriptive fields.

Why should a site-specific outlier trigger investigation rather than immediate correction?

Answer and reasoning

The difference may be real. Investigation determines whether it reflects case mix, workflow, measurement or error before changing valid data merely to make sites look similar.

What is the purpose of root-cause analysis after repeated queries?

Answer and reasoning

To identify why the process keeps generating the error so that the system, definition, training or tool can be improved rather than repeatedly correcting individual records.

Key take-home messages

SUMMARY

Key take-home messages

• Build quality into study design and field operations rather than relying on end-stage cleaning.
• Focus controls on data and processes that matter most to validity and participant protection.
• Use indicators as signals for investigation, not automatic evidence of poor performance.
• Make queries specific, corrections traceable and recurring problems subject to root-cause review.
• Combine central pattern detection with local workflow and source understanding.
• Finalise datasets through documented release criteria while preserving genuine uncertainty.

References

1. International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use. ICH harmonised guideline: Good Clinical Practice E6(R3). Step 4. 6 January 2025. https://database.ich.org/sites/default/files/ICH_E6%28R3%29_Step4_FinalGuideline_2025_0106.pdf
2. International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use. ICH harmonised guideline: General considerations for clinical studies E8(R1). 6 October 2021. https://database.ich.org/sites/default/files/ICH_E8-R1_Guideline_Step4_2022_0204%20%281%29.pdf
3. World Health Organization. Data quality review: a toolkit for facility data quality assessment. Module 1: framework and metrics. Geneva: World Health Organization; 2017. https://www.who.int/publications/i/item/9789241512725
4. US Food and Drug Administration. Guidance for industry: Electronic source data in clinical investigations. Silver Spring (MD): FDA; 2013. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/electronic-source-data-clinical-investigations
5. US Food and Drug Administration. Electronic systems, electronic records, and electronic signatures in clinical investigations: questions and answers. Guidance for industry. Silver Spring (MD): FDA; 2024. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/electronic-systems-electronic-records-and-electronic-signatures-clinical-investigations-questions