Saved Search

Reliability & Agreement · KAPPA-1S

Sample Size for Hypothesis Test of Kappa Agreement

Plan the number of subjects needed to test two-rater kappa agreement against a minimum acceptable value.

1. Information obtained from previous studies or literature

Use a defensible minimum acceptable kappa from prior evidence or the protocol.

2. Information decided by the researcher

Defaults to 10%. Enter 0 if no adjustment is required.

3. Results

Enter the planning assumptions and select Calculate.

Formula, assumptions, limitations, and methodological note

Example clinical scenario

Two pathologists independently classify biopsy results, and the researcher wants to test whether Cohen kappa exceeds a minimum acceptable agreement level.

Formula and methodological note

This calculator uses the preserved published approximation documented in the calculator specification. It assumes the selected reliability or agreement model, independent subjects, the stated number of items or ratings, and defensible planning values.

Sample size is rounded upward before the dropout/non-response adjustment, which is also rounded upward. The method does not replace design-specific review for clustered samples, multiple raters beyond the stated model, missing ratings, weighted kappa, ordinal outcomes, or alternative ICC definitions.

Method references

  1. Donner A, Eliasziw M. A goodness-of-fit approach to inference procedures for the kappa statistic. Stat Med. 1992;11(11):1511–1519.
  2. Shoukri MM, Asyali MH, Donner A. Sample size requirements for the design of reliability study: review and new results. Stat Methods Med Res. 2004;13(4):251–271.

Suggested citation

Yaacob, Najib Majdi. “Sample Size for Hypothesis Test of Kappa Agreement (KAPPA-1S).” Web calculator. NajibMY, 2026. Accessed July 29, 2026. http://127.0.0.1:4321/tools/sample-size/reliability-agreement/kappa-agreement/.