Probability Sampling Methods

Overview

Probability sampling uses a random mechanism in which every in-scope unit has a known, non-zero probability of selection. This property allows researchers to connect the observed sample to a defined frame and to quantify sampling uncertainty under the design.

Probability sampling is not one technique. It includes simple random, systematic, stratified, cluster, multistage and probability-proportional-to-size designs. The best choice depends on the frame, geographical distribution, subgroup precision, field costs and intended population estimates.

This lecture explains how the main designs work, when they are useful, what information must be retained and why complex designs require design-aware analysis.

Why this matters

The phrase “participants were randomly selected” is insufficient. A reproducible probability design must identify:

• the sampling frame;
• the sampling unit at each stage;
• the random mechanism;
• selection probabilities;
• stratification or clustering;
• treatment of ineligible units and non-response; and
• the analysis implications.

Two samples of the same size can have very different precision. Stratification may improve precision by ensuring coverage of important subgroups. Cluster sampling may reduce travel but increase variance because people within clusters resemble one another. Unequal probabilities may improve operational efficiency but require weights for population estimation.

Key concepts

Known selection probabilities

Let $\pi_i$ denote the probability that unit $i$ is included in the sample. A probability design requires $\pi_i > 0$ and the selection mechanism to be known. The design weight is commonly proportional to $1/\pi_i$.

Equal probability is not mandatory. A stratified sample may deliberately oversample a small hospital, rural area or minority subgroup. The resulting data can still support population estimates if the unequal probabilities are retained and appropriately handled.

Simple random sampling

In simple random sampling without replacement, every possible sample of a specified size has equal probability. Each frame unit has the same inclusion probability.

Strengths

• conceptually clear;
• easy to implement with a complete list;
• straightforward estimation; and
• no design clustering.

Limitations

• requires a person-level frame;
• may underrepresent small but important subgroups by chance;
• can be geographically inefficient; and
• may be costly when selected units are widely dispersed.

A reproducible procedure should freeze the frame, assign unique identifiers, set and preserve the randomisation method or seed where appropriate, generate the selected list once, and retain an audit trail.

Systematic sampling

Systematic sampling selects every $k$th unit after a random start, where the interval is related to the frame size and desired sample size.

It is efficient for ordered lists or production flows, but the ordering must be examined. Periodicity can bias selection when the interval aligns with a repeating pattern. Sorting by a variable before systematic sampling can create implicit stratification, which may be beneficial if intentional and documented.

Stratified sampling

The frame is divided into non-overlapping strata, and sampling occurs independently within each stratum. Common strata include hospital, district, age group, sex, ethnicity, disease severity or calendar period.

Stratification can:

• guarantee representation of specified groups;
• improve precision when units are relatively homogeneous within strata;
• permit subgroup-specific estimates; and
• support oversampling of small groups.

Allocation may be:

• proportionate, where sample sizes follow stratum population sizes;
• equal, where each stratum contributes the same number;
• disproportionate, where sampling fractions are chosen for precision or operational reasons; or
• optimal, where allocation reflects stratum variability and cost.

Disproportionate allocation generally requires weighting for overall population estimates.

Cluster and multistage sampling

Cluster sampling selects naturally occurring groups such as villages, schools, practices or wards. Researchers may include all elements in selected clusters or sample within them.

Advantages

• reduces travel and listing costs;
• permits staged frame construction; and
• is practical for large geographical populations.

Disadvantages

• observations within clusters tend to be correlated;
• fewer independent clusters can produce unstable variance estimates;
• cluster size may vary; and
• design effects often increase the sample size needed for a given precision.

Multistage sampling applies random selection at several levels. For example:

1. select districts;
2. select clinics within districts;
3. select patients within clinics.

The overall inclusion probability is the product of the stage-specific conditional probabilities, provided the design is correctly defined.

Probability proportional to size

When clusters differ greatly in size, selecting clusters with probability proportional to a size measure can reduce unequal element probabilities. The size measure must be current, positive and related to the number of eligible elements. If the listed size is inaccurate, final weights may require correction using actual counts.

DECISION GUIDE

Choose among probability designs

• Use simple random sampling when a complete frame exists and field distribution is manageable.
• Use systematic sampling for an ordered flow or list without problematic periodicity.
• Use stratified sampling when subgroup representation or stratum-specific precision matters.
• Use cluster sampling when travel and listing costs dominate.
• Use multistage sampling when frames exist at higher levels but not for all individuals.
• Use probability proportional to size when cluster sizes vary substantially.
• Combine methods when the design requires it, but preserve every stage and probability.

Comparison of hospital allocation strategies

illustrative

Table 1. Sampling 360 women from three hospital strata

Hospital	Frame size	Proportionate sample	Equal sample	Inclusion probability under equal allocation
A	610	181	120	120/610
B	390	116	120	120/390
C	210	63	120	120/210
Total	1,210	360	360	Varies by hospital

Note: Under equal allocation, women in Hospital C have a higher selection probability than women in Hospital A. Overall estimates require design-aware weighting if the target is the combined hospital population.

Designing a reproducible procedure

PROTOCOL CHECKLIST

Probability-sampling procedure to prespecify

• final frame and freeze date;
• sampling unit at every stage;
• strata and cluster definitions;
• target sample by stratum or stage;
• random-number generation or random-start procedure;
• selection probabilities and planned weights;
• handling of duplicates and frame errors;
• treatment of selected ineligible units;
• policy on replacements or reserve samples;
• variables retained for analysis; and
• quality checks and audit trail.

Reserve samples can be selected probabilistically in advance, but they should not be used to conceal non-response. The protocol must state when a reserve unit is activated and preserve the originally selected unit’s disposition.

Design effects and effective information

The nominal sample size counts observations. The effective sample size reflects how much independent information the design provides relative to a simple random sample.

Cluster similarity is often summarised by the intracluster correlation coefficient. When cluster sizes are approximately equal, a common planning approximation is:

$$DEFF \approx 1 + (m-1)\rho$$

where $DEFF$ is the design effect, $m$ is the average number of elements per cluster, and $\rho$ is the intracluster correlation coefficient.

This approximation is not a universal analysis formula. Unequal cluster sizes, weights, stratification and multistage features can alter the design effect.

Stratification can reduce variance, whereas clustering and highly variable weights commonly increase it. The net effect depends on the complete design.

Ethical and governance considerations

Interface with statistical analysis

Design variables must be supplied to the analysis:

• final weights;
• strata;
• cluster or primary sampling unit identifiers;
• finite-population information where used; and
• any calibration or non-response adjustments.

Ignoring clustering usually makes standard errors too small. Ignoring unequal weights may bias population estimates. Applying weights without the corresponding variance design can also produce incorrect uncertainty.

Descriptive and regression analyses should use software and methods capable of complex survey estimation. The estimand must be clear: an unweighted sample association, a frame-population mean, a target-population prevalence or another quantity.

Self-weighting designsMMed extension

A design is self-weighting when all ultimate elements have the same overall inclusion probability. Multistage designs can be approximately self-weighting through coordinated choices, such as probability-proportional-to-size cluster selection followed by a fixed number of elements per cluster. Non-response and frame inaccuracies can still break self-weighting, so actual dispositions must be incorporated.

Worked example

WORKED EXAMPLE

Selecting women within hospital strata

The team wants both an overall estimate and hospital-specific descriptions.

Design: Stratify by hospital and select 120 women by simple random sampling within each stratum.

Consequence: The small hospital is oversampled. Inclusion probabilities are:

• Hospital A: $120/610$;
• Hospital B: $120/390$; and
• Hospital C: $120/210$.

Analysis: Use inverse-probability design weights for an overall three-hospital estimate and survey-design variance estimation. Hospital-specific estimates can be calculated within each stratum, subject to non-response adjustments.

Rationale: Equal allocation improves the precision available for comparing hospital patterns, but the raw combined sample no longer reflects the frame’s hospital distribution.

Suggested protocol or reporting wording

Evidence basis and critical appraisal

Probability-sampling guidance for contemporary telephone and population surveys stresses that random selection requires a defensible frame and calculable inclusion probabilities [1]. WHO STEPS materials demonstrate how stratification, cluster selection and within-household procedures are integrated in field surveys [2]. Demographic and Health Survey manuals provide detailed examples of multistage area sampling and household listing [3].

Sampling theory shows why design choice changes variance, weighting and effective information [4]. Applied survey-analysis guidance then connects these design features to estimation, regression and variance calculation [5].

When appraising a probability sample, verify the following chain:

1. The source population is represented by the stated frame.
2. Sampling units are defined at every stage.
3. A genuine random mechanism operates at each probability stage.
4. Units selected with unequal probabilities retain those probabilities.
5. Strata and clusters are preserved in the analytical dataset.
6. Non-response adjustments do not erase the original design.
7. The number of primary sampling units is adequate for variance estimation.
8. Published totals use weights appropriate to the stated target.

The label “multistage random sample” is not enough. The report should provide the number selected at each stage, the measure of size where probability proportional to size was used, and the within-cluster selection procedure.

Field implementation and quality control

Probability designs can be undermined during fieldwork. Common threats include skipping distant households, substituting a nearby unit, allowing fieldworkers to choose one adult, or repeatedly sampling high-volume clinic sessions. Quality control should include:

• locked selected-unit lists;
• documented random within-household selection;
• supervisor review of non-contact and ineligibility codes;
• audit of geographic or temporal coverage;
• prohibition of unapproved substitutions;
• reconciliation of achieved and expected sampling fractions; and
• preservation of reserve samples separately from the main selection.

A design remains probability-based only when field implementation follows the defined probabilities. Unrecorded departures create unknown selection mechanisms even if the original computer selection was correct.

When probability sampling is not feasible

A probability design should not be claimed when the frame or field process cannot support known selection probabilities. In that situation, researchers should redesign the frame, narrow the target population or report a non-probability approach transparently. A partially random process followed by discretionary substitution is not a probability sample.

The decision should be made before fieldwork and supported by the frame audit, not relabelled after recruitment is complete.

Common mistakes

Knowledge check

What is the defining feature of probability sampling?

Answer and reasoning

Every in-scope unit has a known, non-zero inclusion probability under a documented random selection mechanism.

Why might researchers oversample a small hospital?

Answer and reasoning

To ensure enough observations for hospital-specific estimation or comparison. The unequal inclusion probabilities must then be retained and usually weighted for combined population estimates.

What is the main risk of systematic sampling from a periodically ordered list?

Answer and reasoning

If the sampling interval aligns with a repeating pattern, some types of units may be repeatedly selected or missed. The ordering and interval should therefore be examined.

Why does cluster sampling often require a larger nominal sample?

Answer and reasoning

People within the same cluster tend to be more similar than people selected independently. The sample therefore contains less independent information than a simple random sample of the same size.

Key take-home messages

SUMMARY

Key take-home messages

• Probability sampling requires known, non-zero inclusion probabilities.
• Select the design according to the frame, precision needs, subgroup goals and field costs.
• Stratification, clustering and unequal probabilities change both estimation and uncertainty.
• Retain design information from selection through analysis.
• More observations within a few clusters do not fully replace more clusters.
• Report the procedure precisely enough for another team to reproduce it.

References

1. World Health Organization. Sampling for mobile phone surveys in low- and middle-income countries. Geneva: World Health Organization; 2023. https://www.who.int/publications/i/item/9789240077386
2. World Health Organization. WHO STEPS surveillance manual: the WHO STEPwise approach to noncommunicable disease risk factor surveillance. Geneva: World Health Organization. https://www.who.int/teams/noncommunicable-diseases/surveillance/systems-tools/steps/manuals
3. ICF. Demographic and Health Survey sampling and household listing manual. Rockville, MD: ICF; 2012. https://dhsprogram.com/publications/publication-dhsm4-dhs-questionnaires-and-manuals.cfm
4. Lohr SL. Sampling: design and analysis. 2nd ed. Boston: Brooks/Cole; 2010.
5. Heeringa SG, West BT, Berglund PA. Applied survey data analysis. 2nd ed. Boca Raton, FL: CRC Press; 2017.