We appreciate the interest of Mr Sabour in our study and his knowledgeable comments regarding our article recently published in the American Journal of Neuroradiology.1 We are grateful for the opportunity to address his concerns.⇓
As correctly pointed out by Mr Sabour, we used the Cohen κ to assess interrater reliability between 2 raters with categoric ratings (treatment yes/no), while the Fleiss κ was used to calculate the agreement for all 3 raters (refer to Supplementary Figure 2). The dependence of κ coefficients on prevalence has been frequently criticized and discussed;2 however, the relevance remains controversial.3,4 In the selection of methodologic and statistical test procedures, we referred to previous studies on interrater reliability in patients with vasospasm to ensure sound comparability and coherence.5,6 For the sake of clarity, we concur that a prevalence-adjusted and bias-adjusted interrater analysis may provide additional value. Please see below the prevalence-adjusted and bias-adjusted κ coefficients regarding the detection of severe vasospasm in any arterial segment (>50% narrowing), endovascular treatment decision, and the presence of a perfusion deficit (Table). The results before and after adjustment for prevalence and bias were very similar, leaving the clinical implications of our study unchanged. These findings are also in line with the standards applied in most clinics.
| Interrater Reliability (95% CI) | |||
|---|---|---|---|
| All Raters (n = 3) | Senior (n = 2) | ||
| First rating (CTA) | |||
| Detection of severe vasospasm in any arterial segment (>50% narrowing) | 0.27 (0.11–0.42) 0.27 (0.11–0.43)a | 0.28 (0.10–0.46) 0.21 (−0.02–0.44)a | |
| Endovascular treatment? | 0.23 (0.06–0.39) 0.23 (0.07–0.39)a | 0.23 (−0.01–0.46) 0.23 (−0.01–0.46)a | |
| Second rating (CTA + CTP) | |||
| Detection of severe vasospasm in any arterial segment (>50% narrowing) | 0.31 (0.15–0.46) 0.31 (0.15–0.46)a | 0.46 (0.26–0.66) 0.46 (0.25–0.68)a | |
| Endovascular treatment? | 0.47 (0.30–0.64) 0.50 (0.35–0.66)a | 0.73 (0.55–0.91) 0.77 (0.62–0.93)a | |
| Perfusion deficit? | 0.77 (0.63–0.91) 0.83 (0.73–0.94)a | 0.82 (0.66–0.97) 0.86 (0.74–0.98)a | |
↵a Prevalence-adjusted and bias-adjusted κ coefficients. Note that the values were highly comparable with the results presented in the published article.
Interrater reliability regarding graduation and treatment decisions of vasospasm on CT
References
- © 2024 by American Journal of Neuroradiology
