Reliability of CTA Alone and in Combination with CTP in the Detection of Cerebral Vasospasm: Methodologic Issues on Reliability to Avoid Misinterpretation
==========================================================================================================================================================

* Siamak Sabour

I was interested in the article by Heitkamp et al1 published in the March 2024 issue of the *American Journal of Neuroradiology*. The authors investigated the reliability of CTA alone and in combination with CTP in the detection of cerebral vasospasm and the decision to initiate endovascular treatment. Three neuroradiologists were asked to grade 15 intracranial artery segments in 71 cases using a tripartite scale (no, mild <50%, or severe >50% vasospasm). They reported that the interrater reliability for per-segment analysis of vessels was highly variable (κ = 0.16–0.61). When one focused on senior raters, the addition of CTP images resulted in higher interrater reliability for severe vasospasm (κ = 0.28 versus κ = 0.46) and subsequently higher concordance (κ = 0.23 versus κ = 0.73) for the decision to start endovascular treatment.

Although the article provides insight into the decision about using CTA in combination with CTP to increase the reliability of endovascular treatment, its conclusion is limited in 2 important methodologic and statistical ways. First, the value of κ depends on the prevalence in each category. It is possible to have the prevalence of concordance cells equal to 90% and discordance cells equal to 10% and, however, obtain different κ values (0.44 as moderate versus 0.81 as very good, respectively).2⇓-4 Second, the Fleiss κ is a statistical measure for assessing the reliability of agreement among a fixed number of raters when assigning categoric ratings to a number of items. This use contrasts with other applications of κ such as the simple Cohen κ, which works only when assessing the agreement between not more than 2 raters or the intrarater reliability.

The authors concluded that CTA alone offers only low interrater reliability in the graduation of cerebral vasospasm. However, using CTA in combination with CTP might help, especially senior neuroradiologists, to increase the interrater reliability to identify severe vasospasm following aneurysmal SAH and to increase the reliability regarding endovascular treatment decisions. Considering the above-mentioned methodologic and statistical issues, such conclusions can easily be a misleading message.2⇓-4 Briefly, the prevalence-adjusted and bias-adjusted Fleiss κ should be applied to correctly assess the reliability of CTA alone and in combination with CTP in the detection of cerebral vasospasm and the decision to initiate endovascular treatment.

## Footnotes

*   [Disclosure forms](https://www.ajnr.org/sites/default/files/additional-assets/Disclosures/June%202024/0102.pdf) provided by the authors are available with the full text and PDF of this article at [www.ajnr.org](http://www.ajnr.org).

## References

1.  1.Heitkamp C, Geest V, Tokareva B, et al. CTA supplemented by CTP increases interrater reliability and endovascular treatment use in patients with aneurysmal SAH. AJNR Am J Neuroradiol 2024;45:284–90 doi:10.3174/ajnr.A8110 pmid:38238090
    
    [Abstract/FREE Full Text](http://www.ajnr.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpuciI7czo1OiJyZXNpZCI7czo4OiI0NS8zLzI4NCI7czo0OiJhdG9tIjtzOjM4OiIvYWpuci9lYXJseS8yMDI0LzA1LzIzL2FqbnIuQTgyNDEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

2.  2.Sabour S. Methodologic concerns in reliability of noncalcified coronary artery plaque burden quantification. AJR Am J Roentgenol 2014;203:W343 doi:10.2214/AJR.14.12649 pmid:25148196
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.2214/AJR.14.12649&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=25148196&link_type=MED&atom=%2Fajnr%2Fearly%2F2024%2F05%2F23%2Fajnr.A8241.atom) 

3.  3.Sabour S, Rutten A, van der Schouw YT, et al. Inter-scan reproducibility of coronary calcium measurement using multi detector-row computed tomography (MDCT). Eur J Epidemiol 2007;22:235–43 doi:10.1007/s10654-007-9123-4 pmid:17427036
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1007/s10654-007-9123-4&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=17427036&link_type=MED&atom=%2Fajnr%2Fearly%2F2024%2F05%2F23%2Fajnr.A8241.atom) 
    
    [Web of Science](http://www.ajnr.org/lookup/external-ref?access_num=000246352800004&link_type=ISI) 

4.  4.Sabour S. Reliability of the ASA physical status scale in clinical practice: methodological issues. Br J Anaesth 2015;114:162–63 doi:10.1093/bja/aeu423 pmid:25500399
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1093/bja/aeu423&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=25500399&link_type=MED&atom=%2Fajnr%2Fearly%2F2024%2F05%2F23%2Fajnr.A8241.atom) 

*   © 2024 by American Journal of Neuroradiology