Non-Probability Sampling Methods

Overview

Non-probability sampling includes people through mechanisms for which individual selection probabilities are unknown. It is widely used in clinical, behavioural, implementation, early-phase, qualitative and hard-to-reach-population research.

Non-probability sampling is not automatically poor methodology. Its appropriateness depends on the research purpose, the selection process, the claims made and the transparency of reporting. A consecutive clinical series may be highly informative for describing a service workflow, while a large online volunteer poll may be unsuitable for estimating population prevalence.

This lecture compares convenience, consecutive, quota, volunteer, purposive and network-based sampling and explains how to avoid overstating representativeness.

Why this matters

The main problem is not the label “non-probability”. The problem is an unexamined pathway by which some eligible people are much more likely to appear than others.

Selection may depend on:

• where and when recruitment occurs;
• healthcare use;
• internet access and digital literacy;
• interest in the topic;
• symptom severity;
• trust in researchers;
• clinician referral;
• social-network position;
• language and transport;
• incentives; or
• availability during recruitment hours.

If these factors are related to the outcome or association under study, estimates may be biased. Increasing the sample size reduces random error around the sample estimate but does not remove systematic selection error.

Key concepts

Unknown selection probabilities

In a non-probability sample, the probability that each source-population member enters the sample cannot be calculated from the design. This limits design-based population inference.

The sample may still support:

• feasibility assessment;
• instrument pretesting;
• process evaluation;
• exploratory association analysis;
• hypothesis generation;
• recruitment of a clinically defined case series;
• qualitative inquiry; or
• estimation under explicit model-based assumptions.

Convenience sampling

Convenience sampling includes accessible units. Examples include:

• patients present during research hours;
• records easiest to retrieve;
• students in one class;
• staff who respond to an internal message; or
• visitors to one website.

It is fast and inexpensive, making it useful for pilots, usability testing and early feasibility work. Its weakness is that accessibility is rarely unrelated to the study topic.

Researchers should specify the actual convenience mechanism. “Convenience sampling was used” is less informative than “eligible patients attending the clinic between 08:00 and 12:00 on weekdays were approached while one research assistant was available.”

Consecutive sampling

Consecutive sampling attempts to include every eligible unit encountered over a defined period until the target is reached. It is common in clinical cohorts and diagnostic studies.

It is generally stronger than discretionary convenience recruitment because staff should not choose preferred participants. However, it remains non-probability sampling when the recruitment stream does not cover all source-population members or when attendance itself is selective.

Quality depends on:

• continuous coverage of clinic sessions and days;
• clear start and end dates;
• screening logs;
• no undocumented skipping;
• stable referral pathways; and
• adequate duration to cover seasonal or workflow variation.

Quota sampling

Quota sampling sets target numbers for categories such as age, sex, ethnicity, district or disease severity. Recruiters then fill each quota non-randomly.

Quotas can improve sample diversity and prevent domination by one accessible group. They do not create random selection within categories. Selection bias can persist inside every quota, and weighting to known margins does not guarantee correction for unmeasured differences.

Volunteer and self-selected sampling

Participants opt in after advertisements, invitations or open calls. Volunteer samples are useful when engagement, lived experience or rapid reach is important. Selection may be strongly related to interest, available time, health literacy or prior experience.

Researchers should document:

• where and when invitations appeared;
• targeting algorithms or eligibility filters where known;
• number of views or contacts if available;
• duplicate prevention;
• verification of eligibility; and
• incentives.

Purposive sampling

Purposive sampling deliberately selects information-rich cases according to the research aim. It is central to qualitative research and can also support case-study, implementation or instrument-development work.

Purposive selection is not a substitute for probability sampling when the primary objective is a population prevalence. Its logic is conceptual relevance rather than statistical representativeness.

Snowball and respondent-driven approaches

In snowball sampling, participants refer others in their networks. It is useful when no conventional frame exists and the population is difficult to identify. Network dependence can overrepresent people with larger networks, stronger connections or proximity to initial “seeds”.

Respondent-driven sampling adds structured recruitment limits, tracking and estimation assumptions. It is more than informal snowball sampling and should not be claimed unless its procedures and assumptions are met.

DECISION GUIDE

Match the non-probability method to the purpose

• Use convenience sampling for rapid formative work when population estimates are not the main claim.
• Use consecutive sampling for a defined clinical stream when all eligible presentations can be logged.
• Use quota sampling to ensure category coverage when random selection is infeasible.
• Use volunteer sampling when open participation is intrinsic, while limiting population claims.
• Use purposive sampling for information-rich cases and conceptual depth.
• Use network-based sampling when members are identifiable mainly through social connections.
• Do not select a method merely because it is easy; state what inference it can and cannot support.

Comparing non-probability sampling plans

illustrative

Table 1. Methods for studying postpartum follow-up

Plan	Potential use	Main limitation
Recruit attendees at one postpartum clinic	Explore clinic experience	Excludes non-attenders
Recruit every eligible delivery for six months	Describe participating-hospital cohort	Limited to hospital users and period
Fill age and district quotas online	Ensure visible diversity	Unknown selection within quotas
Recruit women with contrasting follow-up experiences purposively	Understand barriers and facilitators	Not a prevalence sample
Ask participants to refer migrant women	Reach a less visible group	Network and seed dependence

Note: The most suitable method depends on the objective.

Selection mechanisms and inference

Researchers should draw a simple selection pathway:

1. membership in the source population;
2. exposure to the recruitment opportunity;
3. eligibility recognition;
4. approach;
5. willingness and ability to consent;
6. completion of data collection; and
7. inclusion in analysis.

At each stage, ask what factors affect progression and whether they relate to the variables being studied.

For descriptive population estimates, unmeasured selection is especially problematic. For analytical associations, non-representative sampling does not automatically create bias, but selection related jointly to exposure and outcome or their causes can distort associations.

Strengthening a non-probability sample

Non-probability designs can be improved through:

• a clearly bounded recruitment period and setting;
• consecutive rather than discretionary inclusion;
• multiple recruitment channels;
• translated and accessible materials;
• tracking exposure, screening and refusal where possible;
• quotas based on substantively important variables;
• comparison with external benchmarks;
• calibration or propensity adjustment using measured variables;
• sensitivity analysis for unmeasured selection; and
• cautious, population-specific interpretation.

These steps can reduce or explore bias but do not automatically establish probability-based representativeness.

PROTOCOL CHECKLIST

Non-probability sampling information to prespecify

• method and reason for choosing it;
• recruitment setting, channel and period;
• eligibility verification;
• who controls or mediates access;
• how all potentially eligible people are identified;
• quotas, seeds or referral limits where relevant;
• monitoring of skipped or missed eligible units;
• incentive and duplicate-prevention procedures;
• available benchmark variables;
• planned analytical adjustments; and
• boundaries of the intended inference.

Ethical and governance considerations

Interface with statistical analysis

Ordinary confidence intervals and p-values address sampling variability under the analytical model. They do not quantify unknown frame and self-selection bias.

Adjustment methods may use:

• post-stratification;
• raking;
• calibration weighting;
• propensity-score adjustment;
• multilevel regression with population post-stratification; or
• Bayesian models.

These require external population information, adequate overlap and assumptions that measured variables explain selection and outcomes sufficiently. Analysts should distinguish design weights from model-based pseudo-weights and explain their construction.

For purposive qualitative samples, numerical population weighting is generally not the objective. Sampling adequacy is assessed through conceptual coverage and information power.

Model-based inference from non-probability samplesMMed extension

Model-based inference can be defensible when the outcome model is correctly specified or when selection is ignorable conditional on measured covariates. These assumptions are stronger than the known-randomisation basis of probability sampling and cannot be verified fully from the sample. Benchmark comparisons and sensitivity analyses should therefore accompany adjusted estimates.

Worked example

WORKED EXAMPLE

Choosing between prevalence estimation and barrier exploration

Objective A: estimate the proportion of all eligible women who completed postpartum testing.

An online volunteer sample is poorly suited because selection probabilities and coverage are unknown. A register-based probability or well-defined consecutive cohort is preferable.

Objective B: explore why some women complete testing and others do not.

A purposive qualitative sample can deliberately include:

• women who completed early;
• women who completed late;
• women who did not complete;
• rural and urban residents;
• women with and without transport or childcare constraints.

Conclusion: The same recruitment channel should not be assumed suitable for both population prevalence and explanatory depth. The method follows the inference.

Suggested protocol or reporting wording

Evidence basis and critical appraisal

Professional guidance on non-probability samples recommends judging the recruitment mechanism, available benchmarks and inferential assumptions rather than relying on sample size alone [1]. Research on web surveys demonstrates how coverage and self-selection can produce bias even when respondent numbers are very large [2]. Statistical reviews describe model- and calibration-based methods that can support inference, while emphasising their dependence on measured covariates and assumptions [3].

General health-research guidance requires the sampling approach to match the objective [4], and observational reporting standards require transparent participant selection and setting [5].

A reviewer should therefore ask:

• Who could see or receive the invitation?
• Who controlled access to the recruitment channel?
• Could repeated or fraudulent participation occur?
• Which variables drive willingness or ability to enrol?
• Are external population benchmarks available?
• Does the analysis claim a population prevalence, an association within the sample or conceptual insight?
• Would the conclusion change under plausible unmeasured selection?

Interpreting adjusted non-probability estimates

Weighting a non-probability sample to match age, sex and region can improve those margins while leaving differences in healthcare use, motivation or outcome risk unresolved. The adjusted estimate should be described as model-dependent. Diagnostics should include weight distribution, effective sample size, overlap and comparison with external sources. Sensitivity analyses should explore selection related to unmeasured outcomes rather than presenting calibration as proof of representativeness.

Reporting the inferential boundary

The Methods section should name the population actually reached and the mechanism of enrolment. The Discussion should then separate conclusions about the observed sample from any model-dependent extension to a wider population. This boundary is more informative than a generic statement that “generalisation is limited”.

Common mistakes

Knowledge check

Why is consecutive clinical recruitment still often non-probability sampling?

Answer and reasoning

The clinic stream may not cover all source-population members, and the probability of presenting during the recruitment period may be unknown. Consecutive inclusion reduces discretionary skipping but does not necessarily create known population selection probabilities.

Do demographic quotas eliminate selection bias?

Answer and reasoning

No. They align the sample counts with selected categories, but people who volunteer within each category may still differ from non-volunteers in unmeasured ways related to the outcome.

When is purposive sampling appropriate?

Answer and reasoning

When the aim is to select cases with specific experiences, characteristics or conceptual relevance, especially for qualitative inquiry, implementation analysis or instrument development.

What does a narrow confidence interval from a volunteer survey show?

Answer and reasoning

It may show high numerical precision under the model for the volunteer sample. It does not demonstrate absence of coverage or self-selection bias relative to the target population.

Key take-home messages

SUMMARY

Key take-home messages

• Non-probability sampling has unknown individual selection probabilities.
• Its appropriateness depends on the research purpose and the claims made.
• Consecutive inclusion is stronger than discretionary convenience recruitment but may still be selective.
• Large volunteer samples can be precisely unrepresentative.
• Adjustment requires external information and untestable assumptions.
• Report the recruitment mechanism and limit conclusions to what it supports.

References

1. Baker R, Brick JM, Bates NA, Battaglia M, Couper MP, Dever JA, et al. Report of the AAPOR task force on non-probability sampling. Oakbrook Terrace, IL: American Association for Public Opinion Research; 2013.
2. Bethlehem J. Selection bias in web surveys. Int Stat Rev. 2010;78(2):161–188. https://doi.org/10.1111/j.1751-5823.2010.00112.x
3. Elliott MR, Valliant R. Inference for nonprobability samples. Stat Sci. 2017;32(2):249–264. https://doi.org/10.1214/16-STS598
4. World Health Organization Regional Office for the Western Pacific. Health research methodology: a guide for training in research methods. 2nd ed. Manila: WHO Regional Office for the Western Pacific; 2001. https://iris.who.int/items/0765fa3d-af1d-43b3-bb36-6576e88a7871
5. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP; STROBE Initiative. The Strengthening the Reporting of Observational Studies in Epidemiology statement: guidelines for reporting observational studies. Lancet. 2007;370(9596):1453–1457. https://doi.org/10.1016/S0140-6736(07)61602-X