Training and Standardising the Research Team

Overview

A strong protocol does not guarantee standardised implementation. Research personnel must understand their roles, be able to perform delegated procedures correctly, and remain consistent as recruitment progresses. ICH E6(R3) states that individuals involved in a clinical trial should be qualified by education, training and experience for their assigned tasks, while responsibility and oversight remain explicit.[1] The same principle is useful across health research: training should be matched to the procedure and verified through performance, not inferred from attendance alone.

Team standardisation is particularly important for procedures that involve judgement, manual measurement or interaction with participants. Two interviewers can ask the same question differently; two abstractors can classify the same medical record differently; two observers can apply an outcome definition differently. Training and standardisation aim to make those differences visible and manageable.

Why this matters

Research errors often have a human-system interface. A collector may have been trained but still face an ambiguous form, insufficient equipment, unrealistic time pressure or conflicting local practice. A useful training programme therefore combines knowledge, practical rehearsal, competency assessment, feedback and ongoing review.

For multicentre research, standardisation also protects comparability. If one site’s staff measure waist circumference over clothing while another follows a standard anatomical landmark, site differences can become measurement differences. WHO field-survey programmes emphasise uniform methods and practical preparation of data collectors for precisely this reason.[2]

Orientation, training, competency and standardisation

These activities are related but distinct.

• Orientation introduces the study purpose, structure, roles, systems and communication routes.
• Training teaches the knowledge and skills required for a task.
• Competency assessment evaluates whether a person can perform the task to the required standard.
• Standardisation assesses and promotes consistency between personnel, sites or time periods.
• Refresher training addresses drift, procedural changes or recurring errors after initial qualification.

A training log records participation; it does not prove competency. For low-risk administrative tasks, acknowledgement of an instruction may be sufficient. For complex or safety-critical procedures, observed performance is more appropriate.[1]

Build a training matrix

A training matrix maps roles to required documents and competencies. An illustrative matrix might include:

The matrix also helps when staff join late. Instead of improvising, the site can identify the exact training package required for a new role.

Adult learning and practical rehearsal

Research training should move beyond passive presentation. Useful methods include:

• demonstration followed by return demonstration;
• worked clinical or field scenarios;
• simulated consent or interview encounters;
• standardised record-abstraction cases;
• practice in a training database;
• paired observation with structured feedback;
• short quizzes for critical definitions; and
• troubleshooting exercises for expected exceptions.

WHO survey guidance similarly emphasises explaining the survey rationale, practising uniform methods and preparing collectors for field procedures.[2]

PROTOCOL CHECKLIST

Specify the training programme prospectively

• Roles and delegated tasks.
• Required protocol, SOP and instrument versions.
• Training modality for each task.
• Practical exercises and reference cases.
• Competency criteria before independent work.
• Trainer or assessor roles.
• Documentation of completion and competency.
• Refresher or retraining triggers.
• Procedure for training on amended documents.
• Ongoing standardisation and supervision activities.

Standardising measurements

For physical measurements, standardisation should address equipment and technique together. Training may cover positioning, anatomical landmarks, preparation, repeat-measure rules, calibration checks and recording. If observers make subjective judgements, use reference examples and assess agreement.

Reliability statistics can be useful when repeated ratings or measurements are central to the study, but a single high coefficient does not replace procedural review. Agreement can deteriorate over time, and a statistic may conceal systematic differences between observers. The Research Methodology lecture on reliability and reproducibility and the related Biostatistics lectures address quantitative assessment in more detail.

Standardising interviews and questionnaires

Standardisation does not mean sounding robotic. Interviewers can be respectful and responsive while preserving the intended meaning of questions. They should know which wording must remain fixed, which neutral probes are allowed, and how to respond when participants ask for clarification.

Common interviewer effects include paraphrasing differently, leading respondents, skipping response options, displaying approval or disapproval, and prematurely interpreting ambiguous answers. Recorded or observed practice, where ethically appropriate, can help identify these behaviours before fieldwork.

Standardising record abstraction

Medical records are rarely organised for research questions. Abstractors need explicit source hierarchy, date rules, definitions and examples. Provide difficult reference cases, not only obvious ones. Compare trainee classifications with an adjudicated standard and discuss the reasoning behind discrepancies.

A disagreement may reveal a training problem, but it may also reveal an inadequate operational definition. If competent staff repeatedly disagree on the same rule, revise the rule rather than repeatedly retraining them.

Competency criteria

Competency should be defined before assessment. Examples include:

• 100% correct classification of a small set of critical eligibility cases;
• completion of a mock consent without omitted required elements;
• measurement within prespecified tolerance of a reference measurement;
• acceptable agreement on a set of abstracted records;
• correct handling of all test database scenarios; or
• satisfactory performance on a structured observation checklist.

The criterion should match the seriousness and complexity of the task. Avoid creating arbitrary pass marks with no link to performance needs.

Documentation without bureaucracy

Training records should show who was trained, on what material or version, when, by whom and whether competency was demonstrated where required. Keep evidence sufficient to reconstruct qualification for the role without generating meaningless paperwork.

When procedures change, assess who requires update training. A typographical correction may need notification only; a new eligibility rule or revised outcome definition may require formal retraining and reassessment.

Ongoing supervision and quality signals

Training continues through field operations. Supervisors can review missingness, query rates, protocol deviations, timing errors, consent documentation, source discrepancies and site-specific patterns. WHO data-quality frameworks encourage routine use of data-quality indicators rather than waiting for end-of-study cleaning.[3]

However, a high error rate does not automatically mean “retrain the staff”. Investigate whether the root cause is:

• unclear definitions;
• an awkward form;
• system latency;
• workload or staffing;
• equipment failure;
• conflicting local workflow; or
• inadequate initial training.

DECISION GUIDE

Respond to a recurring quality problem

1. Confirm that the pattern is real and not a reporting artefact.
2. Identify which role, site, period or procedure is affected.
3. Observe or reconstruct the workflow.
4. Decide whether the cause is knowledge, skill, process, tool or environment.
5. Apply the most appropriate corrective action.
6. Reassess the indicator to confirm improvement.

Worked example

WORKED EXAMPLE

Certifying record abstractors

The study creates 12 de-identified training records covering uncomplicated and ambiguous cases. Each abstractor independently records delivery date, gestational-diabetes treatment, discharge appointment status and postpartum test status.

For critical variables, an adjudicated reference answer is available. A trainee must classify all eligibility and primary-outcome cases correctly and achieve the prespecified agreement threshold for secondary fields. Disagreements are reviewed during feedback. When three trainees all misclassify the same external laboratory scenario, the team discovers that the manual’s source rule is unclear. The manual is revised, the cases are repeated, and only then are staff authorised for independent abstraction.

This approach treats disagreement as information about both the trainee and the system.

Multisite standardisation

Central training can establish common principles, but local constraints must also be understood. Site initiation should map where source information is found, which equipment is used, who can access systems and how participant flow differs. Local adaptation is acceptable only where it does not change the defined research measurement or participant protections.

For critical procedures, consider cross-site calibration sessions or shared reference cases at intervals. Central quality reports can identify site outliers, but interpretation should consider case mix and workload before attributing every difference to performance.

ICH E6(R3) and E8(R1) support proportionate, risk-based quality approaches focused on factors important to participant protection and reliability of study results.[1,4]

Quantifying inter-rater performance appropriatelyMMed extension

When the research outcome depends on human rating, statistical assessment may complement procedural standardisation. The appropriate measure depends on whether the outcome is categorical, ordinal or continuous and whether the goal is reliability or absolute agreement. Cohen’s kappa, weighted kappa, intraclass correlation coefficients and Bland–Altman methods answer different questions.

Do not select a coefficient merely because software offers it. Define the raters, rating process, unit of analysis and intended generalisation. Statistical agreement should be interpreted alongside observed discrepancies and the consequences of disagreement.

Designing a standardisation exercise

A formal standardisation exercise should resemble the decisions staff will face in the study. Begin with a set of reference cases or measurements that cover common situations and important edge cases. Have staff work independently before group discussion; otherwise, apparent agreement may simply reflect shared answers rather than independent application of the rule.

For measurements, repeat the exercise under realistic conditions and record both within-observer and between-observer variation where relevant. For categorical abstraction, preserve the pattern of disagreements, not only an overall agreement percentage. If one category is repeatedly confused with another, the manual may require a more discriminating rule.

Separate knowledge gaps from performance constraints

A trained person can still perform poorly when the environment makes the standard process difficult. Excessive workload, interruptions, unavailable equipment, slow electronic systems and lack of private space can all degrade data collection. Supervision should therefore ask whether staff know the procedure, can perform it and have the conditions required to perform it.

This distinction matters for corrective action. Repeating a lecture will not fix a missing blood-pressure cuff or a database timeout. Conversely, replacing equipment will not correct misunderstanding of an eligibility definition.

Training after amendments

When a procedure changes, assess the impact rather than automatically retraining the entire team. Create a change summary stating what changed, why it changed, which roles are affected and whether competency reassessment is needed. Staff who never perform the changed task may require notification only; those making a newly altered critical judgement may need supervised practice.

Keep the effective date aligned across the SOP, form and training material. If a new definition becomes effective on Monday but the electronic form changes on Wednesday, the study can create two days of structurally inconsistent data.

Supporting new and temporary staff

Staff turnover is predictable in long studies. Maintain a controlled onboarding pathway so new personnel receive the same core standards as the original team. Temporary coverage should not result in unqualified personnel performing specialised procedures simply because experienced staff are absent.

A role-based training matrix makes this scalable. It also helps investigators demonstrate which personnel were qualified for which tasks at any point in the study.

APPRAISAL CHECKLIST

Appraise a team standardisation programme

• Are training requirements linked to delegated tasks?
• Does competency assessment test actual performance rather than attendance?
• Do reference cases include ambiguous and high-consequence situations?
• Are environmental barriers to correct performance assessed?
• Are amended procedures accompanied by targeted update training?
• Is observer drift monitored for critical measurements or judgements?
• Can the study reconstruct who was qualified for a task on a given date?
• Are recurrent discrepancies used to improve both people and processes?

Use feedback that is specific and observable

Feedback should identify the behaviour that needs to change. “Be more careful” is less useful than “read every response option before accepting the participant’s answer” or “verify the specimen date before classifying completion”. Specific feedback can be practised and reassessed.

For critical tasks, record the follow-up assessment rather than assuming feedback alone solved the problem. This closes the loop between detected discrepancy, corrective action and demonstrated improvement.

Common mistakes

Knowledge check

Why is a signed training log insufficient evidence for a complex measurement procedure?

Answer and reasoning

It documents exposure to training, not the ability to perform the procedure correctly. Complex or high-impact tasks should include practical competency evidence such as observed performance or successful reference cases.

Three experienced abstractors repeatedly disagree on the same variable. What should the team investigate first?

Answer and reasoning

The operational definition and source rule. Consistent disagreement among competent staff may indicate that the procedure itself is ambiguous rather than that all three need more of the same training.

What is observer drift?

Answer and reasoning

It is a gradual change in how a procedure or rating rule is applied over time. Drift can create period or site differences even when initial training was successful.

When should refresher training be targeted rather than routine?

Answer and reasoning

Target it when quality indicators, observation, procedural changes or time since last performance suggest a plausible loss of standardisation. Routine refreshers can still be scheduled for high-risk tasks, but they should have a clear purpose.

Key take-home messages

SUMMARY

Key take-home messages

• Match training and competency assessment to delegated research tasks.
• Use practical rehearsal and reference cases, not passive teaching alone.
• Standardise critical measurements, interviews and abstraction procedures across staff, sites and time.
• Define competency criteria before assessment and document qualification proportionately.
• Monitor for drift and use quality indicators to identify where support is needed.
• Investigate the system before assuming every error is solved by retraining.

References

1. International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use. ICH harmonised guideline: Good Clinical Practice E6(R3). Step 4. 6 January 2025. https://database.ich.org/sites/default/files/ICH_E6%28R3%29_Step4_FinalGuideline_2025_0106.pdf
2. World Health Organization. WHO STEPS surveillance manual: the WHO STEPwise approach to noncommunicable disease risk factor surveillance. Geneva: World Health Organization. https://www.who.int/teams/noncommunicable-diseases/surveillance/systems-tools/steps
3. World Health Organization. Data quality review: a toolkit for facility data quality assessment. Module 1: framework and metrics. Geneva: World Health Organization; 2017. https://www.who.int/publications/i/item/9789241512725
4. International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use. ICH harmonised guideline: General considerations for clinical studies E8(R1). 6 October 2021. https://database.ich.org/sites/default/files/ICH_E8-R1_Guideline_Step4_2022_0204%20%281%29.pdf