There is Poor Agreement between the Subjective and Quantitative Adjudication of Aneurysm Wall Enhancement

DISCUSSION

This study aimed to assess the interrater agreement of subjective adjudication of AWE and its agreement with a previously validated objective 3D method for the determination of enhancement. The interrater agreement among adjudicators of the academic center for assessing AWE was moderate (κ = 0.63) and even lower compared with community-based adjudicators (κ = 0.52). There was minimal agreement between subjective and objective assessments of AWE. Adjudicators were less likely to precisely characterize AWE when assessing patients with multiple aneurysms and unruptured saccular IAs of >7 mm in size.

HR 3D T1 VWI can provide further insight into evaluating IAs, because AWE may serve as a surrogate biomarker for risk of rupture.²⁴ Edjlali et al²⁵ analyzed 108 IAs with HR-VWI and reported that AWE was the only factor associated with aneurysm instability (OR, 9.20; 95% CI, 2.92–29; P < .001). In clinical practice, AWE is determined by comparing pre- and postcontrast HR-VWI through direct observation, following expert consensus recommendations from the American Society of Neuroradiology.⁴ However, objective measures are required to gauge enhancement accurately. Consequently, different methods for SI estimation have been proposed.²⁰ Interrater adjudication for AWE assessment is subject to variability. Our internal adjudication for the presence of AWE showed moderate agreement (κ = 0.63); while compared with the external assessment, it demonstrated weak agreement (κ = 0.52). Accurate interpretation of HR-VWI relies on reader experience,⁴ and in clinical studies, wall enhancement is commonly visually determined by 2 or 3 blinded adjudicators.²⁶ This approach may be susceptible to observation bias. Moreover, structures near the aneurysm wall such as leptomeningeal vessels, brain parenchyma, vascular plexus, luminal thrombus, or microhemorrhages may lead to a false-positive results.²⁷ Also, AWE adjudication may be prone to different interpretations due to the need for standardized definitions. Edjlali et al⁷ determined AWE by the thickness of the aneurysm wall that showed enhancement (>1 mm). Nagahata et al⁵ suggested classifying AWE as “strong” if enhancement was equal to or higher than that of the choroid or venous plexus and as “faint” if the wall SI increased compared with the precontrast T1 sequence. To potentially improve the subjective adjudication of AWE in our analysis, adjudicators compared the postcontrast HR 3D T1 VWI sequence with the pituitary stalk as a reference. Additionally, focal or circumferential enhancement patterns were defined on the basis of the criteria of Edjlali et al.⁷

However, there was minimal agreement among adjudicators on the enhancement patterns. Despite using standardized adjudication definitions, the subjective assessment of AWE remained inconsistent among trained academic and community-based adjudicators. Fu et al²⁸ reported a high interrater agreement (κ = 0.87) for identifying AWE patterns and a strong interclass correlation coefficient (0.98) for measuring the wall enhancement index. The adjudicators’ extensive experience in neuroradiology, exceeding 20 years, likely contributed to the high concordance among raters. The weak agreement between the community-based radiologist and our internal consensus adjudication may be due to a lack of experience in interpreting HR 3D T1 VWI at the community center, where this technique is not routinely performed. Academic and community-based radiologists should be familiar with the normal appearance and variability of the vessel wall on HR-VWI, the patterns of enhancement after Gd administration, and the technical limitations and pitfalls in assessing AWE.

3D quantification of SI enables a more precise characterization of AWE distribution. Most groups have performed assessments of wall enhancement subjectively. Even the several “objective” methods of enhancement analysis are prone to bias because ROIs deemed enhancing are subjectively selected on 2D views. Sricharan et al developed a 3D quantification pipeline of SI capable of identifying areas of enhancement within the aneurysm wall (area under the curve = 0.74).²⁹ Raghuram et al⁸ proposed a 3D method to map the SI distribution within the aneurysm wall after SI normalization to the corpus callosum, generating 3D colormaps of AWE to identify areas of increased SI. This method samples the aneurysm wall and generates thousands of data points (μ = 4490 spokes/aneurysm). Moreover, the quantification of the SI was previously optimized with 7T-MRI by tailoring the spoke length to the thickness of the aneurysm wall (range, 0.38–0.78 mm), decreasing potential artifacts introduced by nearby structures. These colormaps are correlated with histograms of enhancement from pre- and postcontrast HR 3D T1 VWI sequences (Fig 5).⁸ The subjective and 3D-CAWE methods of determining the presence of enhancement achieved minimal agreement (κ = 0.16). Molenberg et al² suggested that the absence of AWE may indicate aneurysm stability, because growing, symptomatic, or ruptured IAs rarely do not exhibit AWE. Similarly, a metanalysis by Texakalidis et al³⁰ reported that the absence of AWE was highly predictive of aneurysm stability (negative predictive value, 96.2%). However, using the presence of AWE to predict aneurysm growth or rupture is less reliable (positive predictive value, 55.8%). Subjective adjudication of enhancement may lead to more frequent follow-up imaging and treatment. Implementing 3D AWE methods eliminates subjectivity and mitigates variability resulting from subjective assessment of enhancement.

• Download figure
• Open in new tab
• Download powerpoint

FIG 5.

Enhancement versus artifact case. Basilar tip aneurysm with pseudoenhancement. A and B, Axial and sagittal views of a basilar tip aneurysm displaying false-positive enhancement on HR 3D T1 VWI. C, This aneurysm is not actually enhancing as demonstrated by the 3D colormap (blue indicates less enhancing). In this case, the anterior wall of the aneurysm sac is in close proximity to the pituitary stalk, therefore, looking as enhancing on axial (A) and saggital (B) views. The SI ratio of this aneurysm ranges between 0 and 1.6. D, The black curve represents T1, and the red curve represents T1+Gd. The 3D-CAWE value is lower than the objective 3D-CAWE (SI of the aneurysm body/corpus callosum ratio of ≥1), thus objectively labeled as nonenhancing.

Aneurysm morphology can influence the subjective adjudication of enhancement. Backes et al examined 89 unruptured IAs using HR-VWI and found that ≥7 mm in size was the primary determinant of Gd enhancement.³¹ However, HR-VWI may be affected by flow artifacts.^32,33 Khan et al³⁴ analyzed unruptured IAs on HR-VWI and discovered that larger aneurysms had lower blood flow rates, which may induce contrast stagnation.¹⁶ Slow flow can lead to inadequate SI suppression at the periphery of the aneurysm sac, simulating vessel wall thickening and pseudoenhancement.^4,33 Identifying inadequate blood suppression is valuable for distinguishing pseudoenhancement caused by flow artifacts, both subjectively and objectively. Additionally, larger IAs are more likely to compress surrounding structures,³⁵ and compromised veins can be mistaken for wall enhancement.³⁶ We found that aneurysms of >7 mm in size (OR, 0.33; 95% CI, 0.15–0.72; P = .006) and with higher size ratio (OR, 0.69; 95% CI, 0.49–0.95; P = .03) were less likely to be reliably adjudicated as enhancing by raters.

Matouk et al³⁶ found that HR-VWI could effectively detect the ruptured aneurysm in patients presenting with multiple IAs and SAH. In our study, we observed that adjudicators were less likely to precisely characterize enhancement when assessing patients with multiple aneurysms (OR, 0.4; 95% CI, 0.16–0.97; P = .04) and  size >7 mm in size (OR, 0.22; 95% CI, 0.09–0.55; P = .002). Typically, the ruptured aneurysm can be determined on the basis of clinical signs, angiographic findings, aneurysm size and shape, focal hemorrhage on CT,³⁷ and wall enhancement on HR-VWI.⁴ However, unruptured IAs may also show enhancement. 3D quantification of SI can aid in assessing AWE in patients with multiple IAs.

This study has several limitations. First, we lacked histologic validation to correlate objectively enhancing aneurysms with pathologic analysis. Second, the cross-sectional design of this study prevents us from assessing any causal relationships between AWE and subsequent treatment decisions, symptomatic status, or rupture outcomes. Third, we compared the subjective assessment of AWE with a comprehensive method that objectively measures the SI of the aneurysm wall. However, this latter method has not been validated as the criterion standard for determining enhancement. Fourth, inadequate blood suppression can increase the risk of pseudoenhancement, possibly confounding both subjective and objective assessments. However, histogram analysis from 3D color maps could be a useful tool for identifying false-positive enhancement determinations on CAWE. Finally, although we have a standardized protocol for acquiring HR 3D T1 VWI images 5 minutes after the administration of contrast, delays in acquisition may occasionally occur. However, such small delays would not affect the degree of enhancement of the pituitary stalk and compromise subjective assessment.