Evidence Levels for Neuroradiology Articles: Low Agreement among Raters

American Journal of Neuroradiology

Getting new auth cookie, if you see this message a lot, tell someone!

Inter- and intraobserver agreement

Agreement
1) Interobserver agreement (R1, R2, and R3)
Agreement among 3 raters:
Slight agreement for both reading sessions (Fleiss κ: 0.18 and 0.11)
Agreement between R1 and R2, R2 and R3, and R1 and R3 for 2 sessions:
Slight agreement R1 × R2 (Cohen κ unweighted = 0.20 and 0.04)
Fair agreement R2 × R3 (Cohen κ unweighted = 0.27 and 0.30)
Slight agreement R1 × R3 (Cohen κ unweighted = 0.12 and 0.07)
2) Interobserver agreement (prepublication reviewers)
No agreement (Cohen κ unweighted = −0.22)
3) Intraobserver agreement (R1, R2, and R3)
R1 fair agreement (Cohen κ: 0.23)
R2 fair agreement (Cohen κ: 0.38)
R3 substantial agreement (Cohen κ: 0.66)