Saved Search

Reliability & Agreement · KAPPA-CI

Sample Size for Estimation of Kappa Agreement

Plan a sample size to estimate two-rater kappa agreement with a chosen precision.

1. Information obtained from previous studies or literature

Use a prior study, pilot estimate, or defensible planning value.

2. Information decided by the researcher

Defaults to 10%. Enter 0 if no adjustment is required.

3. Results

Enter the planning assumptions and select Calculate.

Formula, assumptions, limitations, and methodological note

Example clinical scenario

Two clinicians independently classify diabetic-retinopathy status in the same patients, and the researcher wants to estimate Cohen kappa with specified precision.

Formula and methodological note

This calculator uses the preserved published approximation documented in the calculator specification. It assumes the selected reliability or agreement model, independent subjects, the stated number of items or ratings, and defensible planning values.

Sample size is rounded upward before the dropout/non-response adjustment, which is also rounded upward. The method does not replace design-specific review for clustered samples, multiple raters beyond the stated model, missing ratings, weighted kappa, ordinal outcomes, or alternative ICC definitions.

Method references

  1. Donner A, Eliasziw M. A goodness-of-fit approach to inference procedures for the kappa statistic. Stat Med. 1992;11(11):1511–1519.
  2. Shoukri MM, Asyali MH, Donner A. Sample size requirements for the design of reliability study: review and new results. Stat Methods Med Res. 2004;13(4):251–271.

Suggested citation

Yaacob, Najib Majdi. “Sample Size for Estimation of Kappa Agreement (KAPPA-CI).” Web calculator. NajibMY, 2026. Accessed July 29, 2026. http://127.0.0.1:4321/tools/sample-size/reliability-agreement/estimate-kappa-agreement/.