Internal, External and Construct Validity

Overview

Validity asks whether a research conclusion is trustworthy for the claim being made. It is useful to separate at least three questions. Internal validity concerns whether the study design, conduct and analysis support the intended inference within the study setting. External validity concerns whether that inference can reasonably be generalised or transported to another population, setting, time or implementation context. Construct validity concerns whether the operational variables, instruments and procedures adequately represent the concepts that the researcher intended to study.[1,2]

These domains overlap, but they are not interchangeable. A tightly randomised trial can have strong internal validity yet limited applicability to frail patients who were excluded. A nationally representative survey can describe the population well yet still mismeasure depression with an unsuitable instrument. A study may therefore be strong for one inference and weak for another.

Why this matters

Health research often fails not because investigators chose the wrong statistical test, but because the underlying inference is misaligned with the population, measurement or study process. Statistical precision cannot repair systematic misclassification, and a representative sample cannot rescue an invalid causal comparison.

Thinking explicitly about validity during protocol development helps researchers identify threats before data collection. The ICH E8(R1) framework similarly emphasises identifying factors that are critical to the quality and interpretability of a study rather than attempting to control every operational detail equally.[3]

Key concepts

Validity is best considered claim by claim. Internal validity is not equivalent to “good methodology” in the abstract; it depends on whether bias has distorted the particular estimate or conclusion. External validity is not the same as having a large sample. It depends on the relationship between the study population and the target population, and on whether the intervention, setting and outcome process can operate similarly elsewhere. Construct validity is also broader than a reliability coefficient: a measure can be highly reproducible while consistently measuring the wrong construct.[1,4]

The target of inference should therefore be stated before judging validity. A study may aim to estimate the intervention effect among eligible women attending tertiary hospitals, among all women with gestational diabetes in Malaysia, or among clinics able to implement a particular digital workflow. These are different targets.

Internal validity: is the study comparison credible?

Internal validity is threatened when the observed relationship differs systematically from the relationship the study intended to estimate. Important mechanisms include selection bias, confounding, information bias, deviations from intended interventions, missing outcome data and selective reporting. Randomisation can protect against baseline confounding in trials, but it does not automatically prevent post-randomisation bias. Observational studies require an explicit strategy for confounding and selection.

A useful internal-validity audit follows the study chronology: eligibility and recruitment, assignment or exposure measurement, follow-up, outcome ascertainment, analysis inclusion and reporting. At each stage, ask whether a process could create a systematic difference between compared groups.

External validity: to whom and where does the inference apply?

External validity concerns the relationship between the study conditions and the intended target. It is influenced by eligibility restrictions, recruitment mechanisms, geography, health-system capacity, treatment delivery, calendar time, adherence patterns and baseline risk. A study need not mirror the whole population to be useful; rather, researchers should be explicit about the target and whether effect-relevant characteristics differ.[2]

Representativeness is therefore not a binary property. A pragmatic trial conducted in routine clinics may improve applicability to similar services but still fail to represent rural populations. Conversely, a mechanistic trial in a narrow population can answer a valuable biological question even if its clinical generalisability is limited. Thinking in terms of a defined target population helps separate the validity of the study result from the additional assumptions required to transport that result elsewhere.[5]

Construct validity: are we studying what we say we are studying?

Construct validity connects abstract concepts to observed indicators. “Access burden”, “frailty”, “treatment adherence”, “quality of life” and “clinical improvement” are constructs that must be operationalised. Validity evidence may include content relevance, internal structure, relationships with other variables and consequences of the measurement process.[4]

Construct validity also applies to interventions and exposures. If a “multidisciplinary care programme” varies substantially between sites, the intervention construct may be poorly specified. Clear operational definitions, fidelity assessment and validated instruments strengthen interpretation.

Validity trade-offs are not excuses

Researchers sometimes frame internal and external validity as a simple trade-off: strict control improves internal validity while broad inclusion improves external validity. The reality is more nuanced. Poor protocol adherence can damage both. Unclear intervention delivery can reduce causal interpretability and make replication difficult. Broad eligibility can improve transportability without sacrificing randomisation.

The aim is not to maximise one domain at the expense of all others but to align the study with its purpose and state the resulting limits honestly.

Protocol-stage validity map

Before data collection, create a short validity map containing: the principal claim; study population; target population; exposure or intervention definition; outcome construct; major internal-validity threats; effect modifiers relevant to transportability; and the evidence supporting each critical measurement. This map can be updated when deviations occur, but it should not be invented after results are known.

DECISION GUIDE

Classify a validity concern before choosing a remedy

• If the concern changes the comparison within the analysed study population, examine internal validity.
• If the concern changes whether the result applies to another target population or setting, examine external validity.
• If the concern is whether a variable or intervention represents the intended concept, examine construct validity.
• If more than one domain is affected, state each mechanism rather than forcing the issue into a single label.
• Choose prevention or analysis strategies only after the mechanism is specified.

Illustrative validity map

illustrative

Table 1. Validity map for a postpartum follow-up study

Intended claim	Main validity domain	Potential threat	Design response
Reminder programme increases test completion in participating clinics	Internal	Differential outcome verification	Standardised source hierarchy and blinded adjudication where feasible
Effect applies to district clinics	External	Different appointment systems and connectivity	Include diverse sites or plan transportability assessment
Access burden score represents barriers to testing	Construct	Items omit transport and childcare barriers	Content validation and cognitive interviewing

Note: The categories overlap; the table is used to make the dominant mechanism explicit.

APPRAISAL CHECKLIST

Validity questions for critical appraisal

• What exact inference is the study trying to support?
• Could the compared groups differ systematically for reasons other than the exposure or intervention?
• Do the measurements adequately represent the stated constructs?
• How does the study population differ from the intended target population in factors that could affect the result?
• Are limitations discussed by mechanism and likely consequence rather than as a generic list?

Target validity and transportabilityMMed extension

Modern causal-inference frameworks distinguish the validity of an effect estimate in the study sample from transporting that effect to a target population. Transportability may require data on variables that modify the treatment effect and differ between study and target populations. Statistical weighting can help only when the relevant variables are measured and the required exchangeability and positivity assumptions are plausible. This is why external validity cannot be solved merely by increasing sample size.

Interface with statistical analysis

The statistical analysis should reflect the validity map. Confounding control, missing-data handling, measurement-error correction and transportability weighting address different assumptions and should not be treated as interchangeable “adjustments”. Effect estimates and confidence intervals quantify sampling uncertainty, whereas bias analyses investigate systematic error. Detailed statistical methods are covered in the linked Biostatistics lectures.

Worked example

WORKED EXAMPLE

Appraising the SMS reminder trial across three validity domains

The primary analysis shows 74% postpartum testing in the intervention group and 58% in control. Randomisation was concealed, outcome ascertainment used the same laboratory-source hierarchy, and follow-up was similar between arms. These features support internal validity for the assignment effect in participating hospitals.

External validity is less certain. The study excluded women without a smartphone and included only hospitals with centralised appointment systems. The investigators should therefore avoid claiming that the same effect will occur in clinics without those resources.

Construct validity also requires separate appraisal. The three-item “access burden” score was created for the study. Before interpreting subgroup differences by access burden, the team should establish that the items cover the intended construct and behave appropriately in the study languages.

The study can therefore support one claim strongly while supporting other claims only cautiously. That is a more informative conclusion than labelling the whole study simply “high quality”.

Suggested protocol or reporting wording

Common mistakes

Knowledge check

A trial is well randomised but excludes people older than 75 years. Which validity concern is most immediate when applying the result to frail older adults?

Answer and reasoning

External validity is the immediate concern. Randomisation can support the internal comparison among enrolled participants, but exclusion of older adults limits direct evidence about applicability to that target group.

A depression scale gives almost identical scores when repeated after two days but omits important symptoms relevant to the local population. What does this show?

Answer and reasoning

High repeatability does not establish construct validity. The instrument may measure its limited item set consistently while failing to represent the intended construct adequately.

Can a study have strong external validity but weak internal validity?

Answer and reasoning

Yes. A broadly representative sample can still yield a biased exposure-outcome comparison because of confounding, measurement error or selection during follow-up. Representativeness does not guarantee causal credibility.

Why is “the study was valid” an inadequate appraisal statement?

Answer and reasoning

Because validity is inference-specific. The reader needs to know which claim is being judged, the target population, the operational definitions and the assumptions threatened by study processes.

Key take-home messages

SUMMARY

Key take-home messages

• Internal validity concerns credibility of the intended within-study inference.
• External validity concerns generalisation or transport to a specified target population, setting or time.
• Construct validity concerns whether operational variables and procedures represent the intended concepts.
• Validity should be appraised by mechanism and claim, not collapsed into one study-level label.
• Statistical precision does not compensate for systematic bias or poor measurement.

References

1. Shadish WR, Cook TD, Campbell DT. Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin; 2002.
2. Rothman KJ, Lash TL, VanderWeele TJ, Haneuse S. Modern epidemiology. 4th ed. Philadelphia: Wolters Kluwer; 2021.
3. International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use. ICH E8(R1): General considerations for clinical studies. Geneva: ICH; 2021. https://database.ich.org/sites/default/files/E8-R1_Guideline_Step4_2022_0204%20%281%29.pdf
4. Mokkink LB, Prinsen CAC, Patrick DL, Alonso J, Bouter LM, de Vet HCW, et al. COSMIN study design checklist for patient-reported outcome measurement instruments. Amsterdam: COSMIN; 2019. https://www.cosmin.nl/
5. Westreich D, Edwards JK, Lesko CR, Cole SR, Stuart EA. Target validity and the hierarchy of study designs. Am J Epidemiol. 2019;188(2):438-443. https://doi.org/10.1093/aje/kwy228