ABSTRACT

PURPOSE

Spontaneous intracerebral hemorrhage (ICH) is the most severe form of stroke. The timely assessment of early hematoma enlargement and its proper treatment are of great significance in curbing the deterioration and improving the prognosis of patients with ICH. This study aimed to develop an automated hybrid approach to predict hematoma expansion in ICH.

METHODS

The transfer learning method was applied to build a hybrid model based on a convolutional neural network (CNN) to predict the expansion of hematoma. The model integrated (1) a CNN for automated hematoma segmentation and (2) a CNN-based classifier for hematoma expansion prediction that incorporated both 2-dimensional images and the radiomics features of the 3-dimensional hematoma shape.

RESULTS

The radiomics feature module had the highest area under the receiver operating characteristic curve (AUC) of 0.58, a precision of 0, a recall of 0, and an average precision (AP) of 0.26. The ResNet50 and Inception_v3 modules had AUCs of 0.79 and 0.93, a precision of 0.56 and 0.86, a recall of 0.42 and 0.75, and an AP of 0.51 and 0.85, respectively. Radiomic with Inception_v3 and Radiomic with ResNet50 had AUCs of 0.95 and 0.81, a precision of 0.90 and 0.57, a recall of 0.79 and 0.17, and an AP of 0.87 and 0.69, respectively.

CONCLUSION

A model using deep learning and radiomics was successfully developed. This model can reliably predict the hematoma expansion of ICH with a fully automated process based on non-contrast computed tomography imaging. Furthermore, the radiomics fusion with the Inception_v3 model had the highest accuracy.

Spontaneous intracerebral hemorrhage (ICH), the most severe form of stroke, accounts for 10%–15% of strokes in high-income nations and 20%–50% in developing nations.¹ Compared with ischemic stroke, ICH is associated with poor prognostic outcomes, with a mortality rate of 40% at 1 month and a disability rate of 80% in survivors.^{2, 3, 4} Generally, ICH prognosis is influenced by many factors, including baseline volume and hematoma location, Glasgow Coma scale score, intraventricular hemorrhage, and age.^{5, 6, 7} Of these factors, hematoma volume is the only one that is controllable and dynamic.⁵ Clinical studies have shown that 33% of patients with ICH develop early hematoma enlargement within 24 h of ICH onset.^{8, 9, 10} Early hematoma enlargement in patients with ICH is independently correlated with poor prognostic outcomes and death.^{3, 8, 11, 12} Therefore, the timely assessment of early hematoma enlargement and appropriate treatment are crucial in improving the outcomes of patients with ICH.

Recent studies have identified numerous radiological features for predicting hematoma enlargement after ICH. For example, the computed tomography angiography (CTA) spot sign is a powerful predictive method of hematoma enlargement.^{13, 14, 15} However, clinical applications have shown that some patients are allergic to the contrast agents used in CTA, and patients with renal insufficiencies are relatively contraindicated.^{16, 17, 18} By contrast, non-contrast computed tomography (NCCT) serves as the first choice for diagnosis of patients with acute stroke. New markers based on NCCT, such as blend,¹⁹ swirl,²⁰ black hole,²¹ and island,²² as well as satellite signs,²³ can reflect the density and shape heterogeneity of hematomas with high specificity. Therefore, they have been proposed as alternative predictors of hematoma enlargement in clinical settings.^{24, 25, 26} However, these predictive indicators are only used for qualitative or semi-quantitative analyses and have relatively low sensitivity and accuracy in predicting hematoma enlargement.²⁵ Alternative quantitative methods with automated execution, which may have superior predictive power, are still needed.

Radiomics is a promising quantitative method that has performed excellently in various biomedical fields, where it has been used to extract large numbers of quantitative characteristics from conventional medical imaging.^{27, 28} As well as being extensively applied in oncology studies, radiomics has recently been used for the prediction of hematoma enlargement after ICH based on NCCT imaging. Although these models exhibit positive predictive performance,^{29, 30, 31, 32, 33} only traditional or feature-based machine learning (ML) was utilized for the studies. Deep learning is an automatic method that can skip object segmentation, feature selection, and extraction from segmented objects to identify ‘‘effective features.’’ Since it allows the whole process to be mapped from raw input images to final classifications and can exclude the requirement for hand-crafted features, deep learning is also referred to as end-to-end ML.²⁷

Convolutional neural networks (CNNs) based on deep learning are increasingly being used worldwide with promising outcomes. However, developing CNN-based methods requires large training datasets, which is challenging and laborious in clinical settings. Two approaches partially overcome this challenge. The first involves data augmentation, which utilizes affine transformations, including translation, scaling, and rotation, to produce more data from the available data. The other approach involves transfer learning, which is promising in medical image analysis.³⁴ In this study, we developed a CNN-based hybrid model using a data augmentation and transfer learning method for predicting hematoma expansion by integrating the following: (1) a CNN for automated hematoma segmentation, and (2) a CNN-centered classifier for hematoma expansion prediction that incorporates 2-dimensional (2D) images as well as radiomics features for the 3-dimensional (3D) hematoma shape.

Methods

This was a retrospective study permitted by the Medical Ethics Committee of Xianning Central Hospital (no: 20211126011), and the informed consent was waived.

Patients and image acquisition

Patients with spontaneous ICH diagnosed by the radiologist and admitted to the hospital between September 2017 and September 2021 were enrolled in this study. The inclusion criteria were as follows: (1) patients aged >18 years; (2) patients who had received baseline NCCT within 24 h of symptomatic onset and follow-up computed tomography (CT) at ≤72 h. In this study, 506 patients met the inclusion criteria. The exclusion criteria were as follows: (1) patients diagnosed with trauma, aneurysm, vascular malformation, venous sinus embolism, or tumor-induced cerebral hemorrhage (151 cases); (2) patients with emergency surgery before CT review (53 cases); (3) patients with surgical intervention during the 72-h observation period, even in the absence of hematoma expansion (15 cases); (4) patients whose CT images revealed artifacts. In this study, two different CT scanners were used to minimize variability in image acquisition parameters (7 cases). Following the exclusion criteria, 226 cases were excluded from the study. Finally, 280 cases were included in this study. The original CT image of the patients with ICH was reconstructed into a standardized image with a thickness of 1.0 mm and a spacing of 0.7 mm before being exported from the picture archiving and communication system (PACS).

Study design

The model architecture is shown in Figure 1. Briefly, the hybrid model comprised two deep learning models (models 1 and 2), with an automated pipeline between the two models. Model 1 was developed first for the automated generation of hematoma masks based on NCCT images. A hybrid classifier model (model 2) was then trained for the prediction of hemorrhage expansion based on radiomics and CNN features extracted from the NCCT images and hematoma masks in model 1.

Image processing

The Siemens 64-slice CT (SOMATOM go.Top, Henkestr., Erlangen, Germany) scan parameters were as follows: positioning image, 140 kV, 60 mA; spiral scanning parameters, 120 kV, effective mAs 230, pitch 0.55, layer thickness 5 mm, layer spacing 5 mm. Before exporting the image from the PACS (USA), the original image was reconstructed into a standardized image with a thickness of 1.0 mm and a spacing of 0.7 mm. The CT images were exported from PACS in the Digital Imaging and Communications in Medicine format and were transformed into the Neuroimaging Informatics Technology Initiative format. Images were set to values of -1,024 and 3,052 HU. Image analysis was performed using the Ubuntu 18.04 operating system (London, UK) and Python (USA). First, all input sequences were acquired in a 512 × 512 field. Next, the network output was set to only whole hematoma segmentation. After determining the hematoma volume and matching the original residual network, the original NCCT image and segmentation file (128 × 128 voxel) were reconstructed.

Automated segmentation of the hemorrhage (model 1)

DeepBleed is an open-source tool for quick hemorrhage segmentation.³⁵ This automated deep neural network model preprocesses NCCT scans (including skull stripping), segments the hemorrhage area, and outputs a binary hemorrhage mask. Training and validation of this model using data from the Minimally Invasive Surgery Plus Alteplase for Intracerebral Hemorrhage Evacuation trial showed that the dice similarity coefficient, which evaluates spatial overlap extent between ground-truth segmentation by radiologists and the automatic model, was 0.919.^{35, 36} Contrary to manual segmentation, this model can rapidly and precisely segment ICH with a high level of agreement. To adapt it to this study, the code was partially modified and an interface was added for the automated calculation of hematoma volume.

Labels

Previous studies have defined hematoma expansion as an outright volumetric ICH growth of >6 mL or an increase of >33% from the initial CT scan to the follow-up CT scan.^{3, 12} The binary label (negative or positive ICH expansion) was masked for each included study.

Hybrid classifier model for hematoma expansion prediction (model 2)

The hybrid model for ICH hematoma expansion combines well-known radiomics characteristic analysis and CNNs. Radiomics characteristic analysis was used to determine a wide range of researcher-defined quantitative features including shape, intensity, and texture of regions of interest on images. The CNN features were extracted based on CNN training.

The radiomics characteristic was extracted using the Python “pyradiomics” package (https://www.radiomics.io/pyradiomics.html).³⁷ The transformations were included with gradient, wavelet, original, square, exponential, logarithm, square root, and local binary pattern 3D. After image transformation, radiomics features were extracted.

The CNN model was executed using the well-known Inception_v3³⁸ and ResNet50 modules to extract comprehensive 2D image information from the hematoma. Axial slices with maximum hematoma areas were automatically selected as “maximum ICH image”, and the other two images were extracted from three upper (+3) and three lower (−3) slices from the maximum ICH image. For small ICHs with a volume <1 cm, the upper and lower slices may not be within the hematoma range; the extracted features after separation were therefore the same and did not affect the data results. Hematomas <1 cm are often negative. Typically, CNN features are extracted from the output of fully connected last hidden layers.³⁹ During CNN training, “warm-up training” of the convolutional residual network was achieved using image inputs to identify relevant imaging features. The weights of the pretrained conventional ResNet50 and Inception_v3 modules were imported into model 2 and set as “non-trainable”. Meanwhile, data augmentation was performed by rotating, flipping, and resizing the training images to enrich the training data. This regular operation reduced the overfitting risk and boosted a classifier’s performance. The network was optimized using the Adam optimizer (beta1: 0.9, beta2: 0.999, initial learning rate: 0.0001), with an L2 penalty of 0.01, batch size of 50, and cross-entropy cost function. The maximum training epoch was set to 100, and the model was saved when the maximum accuracy was achieved on the testing set. The CNN models were then implemented using the “keras” and “tensorflow” packages and trained on an AMD Ryzen Threadripper 2970WX processor with 48 Gb RAM (USA).

The features selected are shown in Figure 2. The feature-level fusion approaches were used to fuse the features to collect complementary information from radiomics and CNNs and improve diagnostic accuracy (Figure 3).⁴⁰ Following the fusion pipeline, sequential forward feature elimination was chosen for feature selection, which was able to automatically identify the subset of features that are highly appropriate to the problem. The grid search parameters consisted of many features that account for data complexity and separability. Next, the “support vector machine” was used as the ML model for classification. The training set was divided into two stratified sets for cross-validation, and the estimator with the best average accuracy in the cross-validation was chosen as the best estimator. Subsequently, 100 non-expanding (NEG-ICH) and expanding (POS-ICH) cases were randomly selected as the test set, and the remaining cases were used as training and validation sets. The radiomics-specific reporting [checklist for evaluation of radiomics research (CLEAR)⁴¹ and radiomics quality score (RQS)⁴²] are presented as Supplementary Materials S1 and S2. Thirty-seven items were addressed in the CLEAR checklist, and the RQS score was 10 (27.78%).

Statistical analysis

Model performance was evaluated by metrics. The area under the receiver operating characteristic curve (AUC) was determined to assess the ability of a classifier to distinguish between classes. The confusion matrix (CM) denotes instances in a predicted class, whereas columns denote instances in an actual class. Precision indicates the accuracy of the classifier in identifying positive samples. Recall indicates the classifier’s ability to find all positive samples. The area under the precision-recall curve is the average precision (AP). Python software with the “Scikit-Learn” package (https://scikit-learn.org/stable/) was used for statistical analyses.

Results

The schematic of patient recruitment is shown in Figure 4. There were 280 cases included in the analysis, of which 180 were randomly grouped into the training and validation sets (43 POS-ICH cases and 137 NEG-ICH cases) and 100 were randomly grouped into the test sets (24 POS-ICH cases and 76 NEG-ICH cases). Model 1, which automatically labeled the hematoma area and calculated the hematoma volume (Figure 5), was able to efficiently mask the hematoma. The 213 NEG-ICH cases and 67 POS-ICH cases were then classified and labeled. The NEG-ICH group had a male/female ratio of 153:60, an average age of 64.4 years (range, 50–85 years), and a hematoma volume ranging from 0.271 to 79.6 mL. The POS-ICH group had a male/female ratio of 37:30, an average age of 68.7 years (range, 53–89 years), and a hematoma volume ranging from 0.464 to 107.01 mL (Table 1a). These features were not markedly different between the training and testing datasets. The baseline features of the training, validation, and testing sets are shown in Table 1b.

In model 2, the “PyRadiomics” package was used for the extraction of shape-based characteristics from the hematoma masks, with some of the features extracted from the library being automatically deprecated. For each case, 107 features belonging to “Shape Features”, “First Order”, “Gray Level Dependence Matrix”, “Gray Level Co-occurrence Matrix”, “Gray Level Size Zone Matrix”, “Gray Level Run-Length Matrix”, and “Neighborhood Gray Tone Difference Matrix” were obtained using PyRadiomics. The results in the test dataset with the optimal estimator are presented in Figure 6 and Table 2. The radiomics feature module had the highest AUC of 0.58, a precision of 0, a recall of 0, and an AP of 0.26. The CM showed that all POS-ICH cases were identified as NEG-ICH cases, indicating that this module could not detect hematoma expansion.

For CNN features, the ResNet50 and Inception_v3 modules were compared using various CNNs as feature extractors for the optimization of predictive performance, and the results of the multi-slice CNN performances in test sets are shown in Figures 7 and 8 and Table 2. The ResNet50 and Inception_v3 modules had AUCs of 0.79 and 0.93, a precision of 0.56 and 0.86, a recall of 0.42 and 0.75, and an AP of 0.51 and 0.85, respectively.

Next, the radiomics and CNN features were fused to assess if the predictive capability could be improved. The CNN features were extracted from the last hidden layer of images before outputting (dimensional feature vectors, 4,096 for Inception_v3, 6,144 for ResNet50). The results of the two fusion models are shown in Figures 9 and 10 and Table 2. Radiomic with Inception_v3 and Radiomic with ResNet50 had AUCs of 0.95 and 0.81, a precision of 0.90 and 0.57, a recall of 0.79 and 0.17, and an AP of 0.87 and 0.69, respectively. A summary of the analysis is included in doi: 10.5281/zenodo.10570452. The current analyses showed that the radiomics fusion with the Inception_v3 model had the highest accuracy, indicating that improved performance was obtained relative to the radiomics and CNN modules alone.

Discussion

In this study, we developed a hybrid model that incorporates data from 2D hematoma signal intensities and 3D hematoma shapes into one CNN, along with CNN-based automated hematoma segmentation and a fully automated pipeline, without any operator-dependent processes. The developed automated hybrid model can predict hematoma expansion prediction with high AUC (0.95), precision (0.90), recall (0.79), and AP (0.87) values. This strategy would also improve understanding of the synergistic nature of fusion classification using CNN-based transfer learning and radiomics features. The data obtained in this study highlight improved classification performance when using fusion strategies, radiomics-related features, or features extracted from CNN transfer learning alone. The performance of the CNN model using the Inception_v3 algorithm based on a training dataset of 180 ICH cases for the prediction of hematoma expansion achieved an AUC of 0.93. With the addition of radiomics features, the hybrid model demonstrated a small improvement in the prediction performance. The performance was moderately effective based on a relatively small case series. The model had both radiological features and internal CNN features, and, therefore, the results were better.

Previous studies on the prediction of ICH expansion can be divided into two categories. The first category is a prediction model based on radiological characteristics, which are subjectively judged by image signs, including blend signs,¹⁹ swirl signs,²⁰ black hole signs,²¹ island signs,²² and satellite signs.²³ These signs were combined with clinical factors to develop a predictive score table, such as the BAT score table,⁴³ NAG score,⁴⁴ HEAVN score,⁴⁵ HEP score,⁴⁶ the 9-point score,⁴⁷ and the BRAIN score.⁴⁸ Generally, these studies differ in indicators in their predictive scoring systems, but their radiological features may be associated with low sensitivity and low-quality results, as these visual assessments are susceptible to inter-observer variability. Furthermore, previous evaluation methods for ICH expansion were separated but not mixed. However, the relationship between predictors and hematoma enlargement is complex, which would make accurate predictions of hematoma enlargement challenging. As ML can potentially overcome these challenges, another prediction model based on radiomics has emerged. Previously, most prediction models using radiomics were based on traditional ML, which requires manual hematoma segmentation, feature extraction, screening, and reduction.^{30, 31, 32, 33, 49, 50} Although these models have effective predictive performance, the execution is not fully automated. Moreover, operator-dependent processes, including manual segmentation, are laborious and could cause interrater variability, which limits their clinical applications. Deep learning strategies, particularly CNN, have been used to solve this problem. CNNs and radiomics analyses are typical quantitative methods for image analysis and can extract high-dimensional as well as abstract numeric data beyond what is perceivable through visual image analysis. Thus, in this study, we first used deep learning to develop a prediction model in which hematoma segmentation and feature extraction were automated, while the radiomics and CNN features were merged into a hybrid prediction model. This fusion model can combine complementary features and improve robustness by reducing the uncertainty of each feature or mode.

Although the radiomics feature module had a high AUC (0.58), 0 accuracy, and 0 recall, the predictive performance of feature-based radiomics in this study was not as effective as that reported in previous studies. This was probably because of a lack of clinical information in the current study. Generally, previous studies obtained various patients’ features, including demographic factors, neurological status, and laboratory test parameters. However, these characteristics may not be accessible at all hospitals and may not cover the full spectrum of predictive information that can be acquired from patients. Notably, the mixed model in this study has effective predictive performance without clinical information, indicating that it has greater clinical feasibility.

There are some limitations to this study. First, it is a single-center retrospective study with a small sample size. Further investigations with large sample sizes from multiple centers are required to verify the current results. In this study, we used CNNs to extract high-level features using two different modules (Inception_v3 and ResNet50). Three axial slices were selected to yield three individual samples per patient. Therefore, the convolutional residual network and the CNN classifier were trained on 540 (180 × 3) samples and tested on 300 (100 × 3) samples. Second, even though CNNs exclude the processes of feature computation as well as selection using convolutions and can directly capture key characteristics from the images, signal intensity-based CNNs cannot directly capture 3D shapes of hematoma. Furthermore, model 2 can integrate a hematoma image and shape it into one CNN, but the developed model is not end to end. Model 1 for hematoma segmentation and model 2 for ICH status prediction were separately trained and pooled. However, the image preprocessing, models 1 and 2, and the pipeline in between are completely based on open-source modules that can be integrated into a Python-based pipeline, and a test sample can be automatically run through each step of our model.

In conclusion, we developed a model using deep learning and radiomics that can reliably predict hematoma expansion in ICH in a fully automated process based on NCCT imaging. Hematoma expansion was the only indicator of ICH outcome, and prognostic factors, such as neurological deterioration and mortality, were not included. Due to its high reproducibility and generalizability, this model can be applied more widely in predicting prognosis in acute ICH.

Conflict of interest disclosure

The authors declared no conflicts of interest.