Intraclass Correlation Reliability Analyzer
Estimate reliability between raters or repeated measurements using an intraclass correlation coefficient (ICC).
4. Results
Reliability summary
| Coefficient | Estimate | 95% confidence interval |
|---|
Explore measurements
Measurement profiles
Each line joins measurements from one target.
Column numbers match the summaries below. Full column names and all measurements are available in the accessible data table.
Column summaries
| Column | Mean | SD | Minimum | Maximum |
|---|
Reporting wording
Results
Methods
Results
Methods
Technical details
Selected ICC
Dataset summary
Intervals use classical ANOVA F procedures. Negative estimates and limits are retained. Interval width describes uncertainty; it is not an adequacy score.
Alternative ICC definitions
These are different estimands, not competing scores. Keep the primary choice tied to the study design; do not select the largest estimate.
| Numerical form | Estimate | Confidence interval |
|---|
Residuals and accessible plot data
Large standardized residuals
At most ten observations with absolute standardized residuals of 3 or more are listed. These are exploratory observations, not confirmed outliers.
| Target | Column | Value | Standardized residual |
|---|
Accessible plot data — all targets
Methods, assumptions, limitations, validation, privacy, and references
Method
The analyzer calculates classical ANOVA intraclass correlation coefficients using the McGraw–Wong model, definition, and measurement-unit framework. Ten study-design forms map to six distinct numerical formulas in a complete balanced dataset. Confidence intervals use Shrout–Fleiss and McGraw–Wong F procedures.
ICC study-design and selection flowchart
Choosing an ICC: common examples
| Research situation | Typical ICC choice |
|---|---|
| Different random assessors may rate different participants | One-way random, single — ICC(1,1) |
| The final value is the mean of several exchangeable random ratings | One-way random, average — ICC(1,k) |
| The same radiologists assess all scans; inference extends to similar radiologists and exact values should match | Two-way random, absolute, single — ICC(A,1) / ICC(2,1) |
| The same raters assess every target and only consistency matters | Two-way random, consistency, single — ICC(C,1) |
| The same two named devices are the only devices of interest and their values should match | Two-way mixed, absolute, single — ICC(A,1), fixed panel |
These examples are guides, not substitutes for defining the intended inference and measurement process.
Assumptions and limitations
- Rows are independent targets and columns are numerical measurements on the same targets.
- The Phase 1 analysis requires one finite observation in every target-column cell.
- With one observation per cell, target-by-column interaction is pooled with residual measurement error.
- ICC depends on between-target heterogeneity; a restricted target range can reduce the estimate.
- Fixed-column inference applies only to the specified columns. Average-measure inference applies only when all columns will be averaged.
- Large residuals are exploratory observations, not confirmed outliers.
- No universal qualitative reliability grading or adequacy threshold is applied.
Related analyses
For categorical ratings, use the Categorical Inter-rater Agreement Analyzer. If the question is whether two continuous methods give sufficiently similar numerical values, use the Bland–Altman Agreement Analyzer. ICC and Bland–Altman analysis are complementary; a high ICC does not guarantee narrow limits of agreement.
Learn the concept: Choosing the correct ICC
Validation and privacy
Version 1.3.0 retains statistical method RA02-M1 and reproduces the prespecified classical, systematic-offset, negative-ICC, boundary, and invariance benchmarks. Automated tests cover canonical guided and advanced ICC selection, F quantiles, local imports, interface behaviour, and mobile containment. Public availability supports independent comparison and does not constitute regulatory or universal statistical validation. Files and measurements remain in browser memory and are not intentionally transmitted or retained.
References
- McGraw KO, Wong SP. Forming inferences about some intraclass correlation coefficients. Psychol Methods. 1996;1:30–46.
- Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86:420–428.
- Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15:155–163; erratum 2017;16:346.
Suggested citation
Yaacob, NM & Che Soh, NAA. Intraclass Correlation Reliability Analyzer, version 1.3.0. NajibMY; 2026. Available from: https://najibmy.netlify.app/tools/intraclass-correlation-reliability/
This tool supports research interpretation and does not replace statistical consultation. A high ICC does not establish measurement agreement or clinical acceptability.