Article Figures & Data
Figures
- FIG 1.
CapsNet (A) and UNet (B) architectures. The nnUNet architecture was self-configured by the model and is already published.16 All models process 3D images in all layers, with dimensions shown on the left side. The depth, height, and width of the image in each layer is shown by D, H, and W, respectively. A, The number over the Conv1 layer represents the number of channels. The numbers over the capsule layers (ConvCaps, DeconvCaps, and FinalCaps) represent the number of pose components. The stacked layers represent capsule channels. B, The numbers over each layer represent the number of channels. In UNet and nnUNet, the convolutions have stride = 1 and the transposed convolutions have stride = 2. Note that the numbers over the capsule layers show the number of pose components, while the numbers over the noncapsule layers show the number of channels.
- FIG 2.
CapsNet, UNet, and nnUNet segmentation of brain structures that were represented in the training data. Segmentations for three structures are shown: third ventricle, thalamus, and hippocampus. Target segmentations and model predictions are, respectively, shown in red and white. Dice scores are provided for the entire volume of the segmented structure in this patient (who was randomly chosen from the test set).
- FIG 3.
CapsNets outperforms UNets in segmenting images that were not represented in the training data. Both models were trained to segment right-brain structures and were tested to segment contralateral left-brain structures. Target segmentations and model predictions are, respectively, shown in red and white. Dice scores are provided for the entire volume of the segmented structure in this patient. The CapsNet partially segmented the contralateral thalamus and hippocampus (white arrows), but the UNet poorly segmented the thalamus (white arrow) and entirely missed the hippocampus.
- FIG 4.
Comparing CapsNets, UNets, and nnUNets when training data are limited. When the size of the training set was decreased from 3199 to 120 brain MRIs, hippocampus segmentation accuracy (measured by Dice score) of all 3 models did not decrease >1%. Further decrease in the size of the training set down to 60 MRIs led to worsened segmentation accuracy.
- FIG 5.
Comparing the computational efficiency among CapsNets, UNets, and nnUNets, in terms of memory requirements (A) and computational speed (B). A, The bars represent the computational memory required to accommodate the total size of each model, including the parameters plus the cumulative size of the forward- and backward-pass feature volumes. B, CapsNet trains faster, given that its trainable parameters are 1 order of magnitude fewer than UNets or nnUNets. The training times represent the time that each model took to converge for segmenting the hippocampus, divided by the number of training examples and the training epochs (to make training times comparable with test times). The test times represent how fast a fully-trained model can segment a brain image.
Tables
Data Partitions No. of MR Imaging Volumes No. of Patients Age (mean) (yr) Sex Diagnosis Training set 3199 841 76 (SD, 7) 42% F, 58% M 29% CN, 54% MCI, 17% AD Validation set 117 30 75 (SD, 6) 30% F, 70% M 21% CN, 59% MCI, 20% AD Test set 114 30 77 (SD, 7) 33% F, 67% M 27% CN, 47% MCI, 26% AD Note:—F indicates female; M, male; CN, cognitively normal; MCI, mild cognitive impairment; AD, Alzheimer disease.
- Table 2:
Comparing the segmentation efficacy of CapsNets, UNets, and nnUNets in segmenting brain structures that were represented in the training dataa
Brain Structure CapsNet Dice (95% CI) UNet Dice (95% CI) nnUNet Dice (95% CI) Repeated Measures ANOVA P Valueb Third ventricle 95% (94–96) 96% (95–97) 96% (95–97) .03 Thalamus 94% (93–95) 95% (94–96) 94% (92–96) .1 Hippocampus 92% (91–93) 93% (92–94) 92% (91–93) .1
Jump to section
Related Articles
Cited By...
- No citing articles found.