Glioblastoma is the most dangerous type of brain tumor with a poor prognosis and less than a year survival rate. Determining the O6-methylguanine-DNA-methyltransferase (MGMT) promoter status is crucial for treatment planning, as it indicates chemosensitivity. MGMT promoter status lies within the tumor and serves as a reliable indication of cancer's chemosensitivity. Surgical tissue sample removal is required for genetic testing, which may take a few weeks for characterization. Non-invasive diagnosis using MR images plays a significant role in early detection through deep learning techniques. This study proposes a pipeline using stacked multimodalities of MRI scans for brain tumor segmentation and MGMT classification. The pipeline consists of two phases: segmentation using a 3D Residual U-Net Architecture and classification using a 3D ResNet 10 Classifier. The pipeline enhances the prediction of MGMT status and can assist radiologists in diagnosing brain tumors more efficiently and precisely. It reduces subjectivity, variability, and the need for surgical equipment, potentially improving treatment decision making.
Glioblastoma is a highly aggressive type of brain tumor with a devastating prognosis, characterized by a survival rate of less than a year [1][2]. Among various molecular markers, the O6-methylguanine-DNA-methyltransferase (MGMT) promoter status has emerged as a crucial factor in determining the response to chemotherapy and predicting patient outcomes [2]. MGMT promoter methylation, in particular, has been associated with a more favorable prognosis and increased chemosensitivity in glioblastoma patients [3]. To incorporate MGMT status into treatment planning, it is necessary to obtain surgical tissue samples from brain tumor patients for subsequent genetic testing [4]. This process, however, can be invasive, time-consuming, and may pose additional risks to the patient. Therefore, there is a growing interest in non-invasive diagnostic methods that can provide reliable information about MGMT status, enabling more personalized and effective treatment strategies. Magnetic Resonance Imaging (MRI) has emerged as a valuable imaging technique for brain tumor assessment, allowing for the visualization of tumor characteristics and spatial extent. Recent advancements in deep learning techniques have enabled the development of automated approaches for tumor segmentation and classification using MRI data. These methods leverage the power of artificial intelligence to analyze multimodal MRI scans and provide accurate and efficient tumor delineation. Previous studies have explored brain tumor segmentation and MGMT classification separately, with limited consideration of the relationship between the two tasks [5]. Few studies have attempted to integrate segmentation and classification using selective modalities for MGMT prediction. However, since each modality carries its own unique information, it is important to leverage the combined knowledge from multiple MRI modalities to achieve more accurate and robust MGMT status prediction. In this study, we propose a novel pipeline for brain tumor segmentation and MGMT classification using stacked multimodal MRI scans. The pipeline consists of two phases: the first phase utilizes a 3D Residual U-Net Architecture to segment the brain tumor into sub-regions, while the second phase employs a 3D ResNet 10 Classifier to determine the MGMT promoter status based on the segmented tumor regions. By leveraging the strengths of both segmentation and classification models, our pipeline aims to improve the precision and efficiency of MGMT status prediction. The proposed pipeline has important therapeutic implications as it can assist radiologists and clinicians in diagnosing brain tumors more accurately and efficiently, reducing the subjectivity and variability associated with human interpretation. By providing precise information about the tumorous regions and MGMT status, our pipeline can aid in treatment decision making, enabling personalized therapeutic strategies for glioblastoma patients. In this paper, we present the details of our proposed pipeline and evaluate its performance using a benchmark dataset, BraTS2021. We demonstrate the potential of our approach to enhance the consistency and objectivity of brain tumor diagnoses, paving the way for improved patient outcomes and a better understanding of the role of MGMT status in glioblastoma treatment.
Manual segmentation and classification is an expensive, time-consuming task. But due to the emergence of Deep Learning methods in medical imaging, we can automate this task.
The Radiological Society of North America (RSNA) has teamed up with the Medical Image Computing and Computer Assisted Intervention Society (the MICCAI Society) to improve diagnosis and treatment planning for patients with glioblastoma. Introduced BraTS2021 Challenge for Brain Tumor Segmentation and MGMT promoter status prediction.
To help the radiologist to detect the exact location of the tumor in terms of speed and accuracy.
Early detection of MGMT promoter status, which may reduce the death rate and save the life of the patient in its early stage.
The proposed segmentation-based classification method is trained and evaluated using the publicly available BraTS2021 benchmark dataset. Brain Tumor Segmentation Challenge (BraTS2021) is a joint initiative of the Radiological Society of North America (RSNA), the American Society of Neuroradiology (ASNR), and the Medical Image Computing and Computer-Assisted Interventions (MICCAI) Society. It focuses on two primary objectives: segmenting distinct brain tumor sub-regions and classifying the MGMT promoter methylation status of the tumor. The BraTS2021 dataset contains pre-operative baseline 3D mpMRI images from 2,040 patients [26]. For Segmentation Task 1, there are 1,251 publicly accessible scans with segmentation labels. There are 585 patients in task 2 which contains the MGMT labels. 536 patient's data contain both segmentation masks and MGMT classification labels, as determined by intersecting the datasets for segmentation and classification tasks details are shown in Table
In this study, we make use of task 1 MRI scans and their associated MGMT labels, concentrating on the 536 scans for which segmentation masks and MGMT classification labels are available. The MRI scans consist of 4 modalities and each modality has its significance. The importance of each modality is shown in Table.
For the Segmentation phase the dataset is divided into training and validation sets 80% of the data is used for training and 20% of the data is used for validation and in the classification phase 90% of the data is used for training and 10% of the data is used for validation, since only 536 instances were available for phase 2.Each modality has a dimension of 240 by 240 pixels with 155 different slices. Figure 1 illustrates a sample from the benchmark BraTS2021 dataset, showcasing all four modalities along with the MGMT label of slice number 83.
Brain Tumors may appear in any location inside the human brain with different shape and size.
Ambiguous boundaries between cancer and other brain tissues.
The Boundaries between parts of the tumor structure are often unclear, which leads to disagreement between medical professionals when doing image segmentation.
MGMT status may fluctuate spatially within the tumor, making it difficult to obtain a representative sample.
Predicting MGMT promoter methylation becomes complex due to this variability within different regions of the tumor.
One of the main challenge in Medical Image Analysis is limitation of Labelled Data.
BraTS2021 is the only publicly available dataset for MGMT promoter status classification.
High Computational Resources.
Class imbalance dataset.
Research studies have not fully incorporated all modalities of MRI, such as T1-weighted, T2-weighted, FLAIR, and T1ce, in the analysis, limiting the comprehensive depiction of tumor characteristics and potentially impacting the prediction of MGMT promoter status.
There is a research gap in the development of segmentation models specifically trained on MGMT-related characteristics, which would enable the capture of tumor classes associated with MGMT promoter status and enhance the accuracy of prediction.
Incorporating a dedicated segmentation step that focuses on MGMT-specific tumor classes can significantly improve the reliability and accuracy of MGMT promoter status classification.
Classification of MGMT Promoter status for the prediction of the glioblastoma patient’s survival and treatment planning using modified U-Net and CNN-based Classifier.
Accurate classification of the O6-methylguanine-DNA-methyltransferase (MGMT) promoter status in glioblastoma was critical for treatment planning and predicting patient outcomes. Previous techniques that used segmentation results for MGMT classification were unable to achieve satisfactory results. In this research, we aimed to improve the classification performance by implementing a novel deep learning approach. We utilized a modified version of the 3D Residual U-Net (ResU-Net) architecture for improved segmentation and a 3D ResNet10 model for classification, while also leveraging all four available MRI modalities by stacking them.
The proposed pipeline began with preprocessing steps applied to the magnetic resonance imaging (MRI) scans, including T1-weighted, T2-weighted, T1 gadolinium contrast-enhanced, and fluid-attenuated inversion recovery (FLAIR) images. The images from all four modalities were stacked to capture the complementary information provided by each modality. Subsequently, the ResU-Net segmentation technique was applied to obtain improved segmentation results. This technique aimed to provide more accurate and precise delineation of tumor regions by utilizing the combined information from the stacked modalities.
The resulting segmented images were then fed into the 3D ResNet10 classifier for MGMT promoter status classification. The classifier utilized the segmented tumor regions as input to predict the MGMT status accurately. By incorporating the segmented tumor regions derived from the stacked multimodal MRI scans, our approach aimed to enhance the classification performance and provide a more comprehensive understanding of the MGMT promoter status in glioblastoma.
The primary objective of this research was to enhance the performance of MGMT promoter status classification on the BraTS2021 dataset. By leveraging our novel deep learning approach and utilizing all four MRI modalities through the stacking technique, we hypothesized that our pipeline would achieve improved accuracy and robustness in MGMT classification. The refined segmentation results, obtained from the stacked multimodal MRI scans, were expected to provide the classification model with more precise and comprehensive information, enabling a better understanding of the MGMT promoter status in glioblastoma.
This study aimed to contribute to the field by improving the classification accuracy of MGMT promoter status in glioblastoma using a state-of-the-art deep learning pipeline that incorporated all four MRI modalities. By combining advanced segmentation techniques with powerful classification models and leveraging the complementary information from the stacked modalities, we anticipated achieving significant improvements in the classification performance compared to previous approaches. The results of this research would have important implications for treatment planning, as more accurate classification of MGMT promoter status could guide personalized therapeutic strategies and improve patient outcomes.
[1] Z. Liu, L. Tong, L. Chen, Z. Jiang, F. Zhou, Q. Zhang, et al., "Deep learning based brain tumor segmentation: a survey," Complex & intelligent systems, vol. 9, pp. 1001-1026, 2023.
[2] D. N. Louis, A. Perry, G. Reifenberger, A. Von Deimling, D. Figarella-Branger, W. K. Cavenee, et al., "The 2016 World Health Organization classification of tumors of the central nervous system: a summary," Acta neuropathologica, vol. 131, pp. 803-820, 2016.
[3] D. T. Chen, A. T. Chen, and H. Wang, "Simple and Fast Convolutional Neural Network Applied to Median Cross Sections for Predicting the Presence of MGMT Promoter Methylation in FLAIR MRI Scans," in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 7th International Workshop, BrainLes 2021, Held in Conjunction with MICCAI 2021, Virtual Event, September 27, 2021, Revised Selected Papers, Part I, 2022, pp. 227-238.
[4] S. Moton, M. Elbanan, P. O. Zinn, and R. R. Colen, "Imaging genomics of glioblastoma: biology, biomarkers, and breakthroughs," Topics in Magnetic Resonance Imaging, vol. 24, pp. 155-163, 2015.
[5] J. Roth, J. Keller, S. Franke, T. Neumuth, and D. Schneider, "Multi-plane UNet++ Ensemble for Glioblastoma Segmentation," in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 7th International Workshop, BrainLes 2021, Held in Conjunction with MICCAI 2021, Virtual Event, September 27, 2021, Revised Selected Papers, Part I, 2022, pp. 285-294.
[6] S. Faghani, B. Khosravi, M. Moassefi, G. M. Conte, and B. J. Erickson, "A Comparison of Three Different Deep Learning-Based Models to Predict the MGMT Promoter Methylation Status in Glioblastoma Using Brain MRI," Journal of Digital Imaging, pp. 1-10, 2023.
[7] F. Polly, S. Shil, M. A. Hossain, A. Ayman, and Y. M. Jang, "Detection and classification of HGG and LGG brain tumor using machine learning," in 2018 International Conference on Information Networking (ICOIN), 2018, pp. 813-817.
[8] W. Weng and X. Zhu, "INet: convolutional networks for biomedical image segmentation," Ieee Access, vol. 9, pp. 16591-16603, 2021.
[9] W. Chen, B. Liu, S. Peng, J. Sun, and X. Qiao, "S3D-UNet: separable 3D U-Net for brain tumor segmentation," in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Revised Selected Papers, Part II 4, 2019, pp. 358-368.
[10] P. Ahmad, S. Qamar, L. Shen, and A. Saeed, "Context aware 3D UNet for brain tumor segmentation," in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 6th International Workshop, BrainLes 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4, 2020, Revised Selected Papers, Part I 6, 2021, pp. 207-218.
[11] Z. Li, Z. Shen, J. Wen, T. He, and L. Pan, "Automatic Brain Tumor Segmentation Using Multi-scale Features and Attention Mechanism," in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 7th International Workshop, BrainLes 2021, Held in Conjunction with MICCAI 2021, Virtual Event, September 27, 2021, Revised Selected Papers, Part I, 2022, pp. 216-226.
[12] R. A. Zeineldin, M. E. Karar, J. Coburger, C. R. Wirtz, and O. Burgert, "DeepSeg: deep neural network framework for automatic brain tumor segmentation using magnetic resonance FLAIR images," International journal of computer assisted radiology and surgery, vol. 15, pp. 909-920, 2020.
[13] L. H. Shehab, O. M. Fahmy, S. M. Gasser, and M. S. El-Mahallawy, "An efficient brain tumor image segmentation based on deep residual networks (ResNets)," Journal of King Saud University-Engineering Sciences, vol. 33, pp. 404-412, 2021.
[14] T. Yang, J. Song, L. Li, and Q. Tang, "Improving brain tumor segmentation on MRI based on the deep U-net and residual units," Journal of X-ray Science and Technology, vol. 28, pp. 95-110, 2020.
[15] H. M. Luu and S.-H. Park, "Extending nn-UNet for brain tumor segmentation," in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 7th International Workshop, BrainLes 2021, Held in Conjunction with MICCAI 2021, Virtual Event, September 27, 2021, Revised Selected Papers, Part II, 2022, pp. 173-186.
[16] Q. Jia and H. Shu, "Bitr-unet: a cnn-transformer combined network for mri brain tumor segmentation," in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 7th International Workshop, BrainLes 2021, Held in Conjunction with MICCAI 2021, Virtual Event, September 27, 2021, Revised Selected Papers, Part II, 2022, pp. 3-14.
[17] L. Pei and Y. Liu, "Multimodal brain tumor segmentation using a 3D ResUNet in BraTS 2021," in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 7th International Workshop, BrainLes 2021, Held in Conjunction with MICCAI 2021, Virtual Event, September 27, 2021, Revised Selected Papers, Part I, 2022, pp. 315-323.
[18] G. Singh and A. Phophalia, "Multimodal Brain Tumor Segmentation Using Modified UNet Architecture," in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 7th International Workshop, BrainLes 2021, Held in Conjunction with MICCAI 2021, Virtual Event, September 27, 2021, Revised Selected Papers, Part I, 2022, pp. 295-305.
[19] S. Wang, L. Li, and X. Zhuang, "AttU-Net: attention U-Net for brain tumor segmentation," in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 7th International Workshop, BrainLes 2021, Held in Conjunction with MICCAI 2021, Virtual Event, September 27, 2021, Revised Selected Papers, Part II, 2022, pp. 302-311.
[20] A. S. Akbar, C. Fatichah, and N. Suciati, "Unet3D with Multiple Atrous Convolutions Attention Block for Brain Tumor Segmentation," in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 7th International Workshop, BrainLes 2021, Held in Conjunction with MICCAI 2021, Virtual Event, September 27, 2021, Revised Selected Papers, Part I, 2022, pp. 182-193.
[21] T. S. H. Mun, S. Doran, P. Huang, C. Messiou, and M. Blackledge, "Multi Modal Fusion for Radiogenomics Classification of Brain Tumor," in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 7th International Workshop, BrainLes 2021, Held in Conjunction with MICCAI 2021, Virtual Event, September 27, 2021, Revised Selected Papers, Part I, 2022, pp. 344-355.
[22] D. Lang, J. Peeken, S. Combs, J. Wilkens, and S. Bartzsch, "A Video Data Based Transfer Learning Approach for Classification of MGMT Status in Brain Tumor MR Images," in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 7th International Workshop, BrainLes 2021, Held in Conjunction with MICCAI 2021, Virtual Event, September 27, 2021, Revised Selected Papers, Part I, 2022, pp. 306-314.
[23] N. Saeed, S. Hardan, K. Abutalip, and M. Yaqub, "Is it possible to predict MGMT promoter methylation from brain tumor MRI scans using deep learning models?," in International Conference on Medical Imaging with Deep Learning, 2022, pp. 1005-1018.
[24] S. Das, "Optimizing prediction of MGMT promoter methylation from MRI scans using adversarial learning," arXiv preprint arXiv:2201.04416, 2022.
[25] E. Calabrese, J. D. Rudie, A. M. Rauschecker, J. E. Villanueva-Meyer, J. L. Clarke, D. A. Solomon, et al., "Combining radiomics and deep convolutional neural network features from preoperative MRI for predicting clinically relevant genetic biomarkers in glioblastoma," Neuro-Oncology Advances, vol. 4, p. vdac060, 2022.
[26] U. Baid, S. Ghodasara, S. Mohan, M. Bilello, E. Calabrese, E. Colak, et al., "The RSNA-ASNR-MICCAI BraTS 2021 benchmark on brain tumor segmentation and radiogenomic classification," arXiv preprint arXiv:2107.02314, 2021.
[27] S. Bauer, R. Wiest, L.-P. Nolte, and M. Reyes, "A survey of MRI-based medical image analysis for brain tumor studies," Physics in Medicine & Biology, vol. 58, p. R97, 2013.
[28] O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional networks for biomedical image segmentation," in Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, 2015, pp. 234-241.
[29] N. Ibtehaz and M. S. Rahman, "MultiResUNet: Rethinking the U-Net architecture for multimodal biomedical image segmentation," Neural networks, vol. 121, pp. 74-87, 2020.
[30] R. Raza, U. I. Bajwa, Y. Mehmood, M. W. Anwar, and M. H. Jamal, "dResU-Net: 3D deep residual U-Net based brain tumor segmentation from multimodal MRI," Biomedical Signal Processing and Control, vol. 79, p. 103861, 2023.
[31] H. Li, Y. Iwamoto, X. Han, A. Furukawa, S. Kanasaki, and Y.-W. Chen, "An Efficient and Accurate 3D Multiple-Contextual Semantic Segmentation Network for Medical Volumetric Images," in 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 2021, pp. 3309-3312.
[32] F. Milletari, N. Navab, S. Ahmadi, and V-net, "Fully convolutional neural networks for volumetric medical image segmentation," in 2016 Fourth International Conference on 3D Vision (3DV), pp. 565-571.
[33] P. Guo, L. Li, C. Shang, M. Wang, and Y. Lin, "Glioma grading method based on graph convolutional network," in 7th International Symposium on Advances in Electrical, Electronics, and Computer Engineering, 2022, pp. 1114-1121.
[34] L. Li, S. Qin, Z. Lu, K. Xu, and Z. Hu, "One-shot learning gesture recognition based on joint training of 3D ResNet and memory module," Multimedia Tools and Applications, vol. 79, pp. 6727-6757, 2020.
[35] Y. Gong, Y. Jia, T. Leung, A. Toshev, and S. Ioffe, "Deep convolutional ranking for multilabel image annotation," arXiv preprint arXiv:1312.4894, 2013.
[36] M. J. Awan, M. S. M. Rahim, N. Salim, A. Rehman, and B. Garcia-Zapirain, "Automated knee MR images segmentation of anterior cruciate ligament tears," Sensors, vol. 22, p. 1552, 2022.
[37] S.-S. Su, L.-Y. Li, Y. Wang, and Y.-Z. Li, "Stroke risk prediction by color Doppler ultrasound of carotid artery-based deep learning using Inception V3 and VGG-16," Frontiers in Neurology, vol. 14, 2023.
[38] M. Havaei, A. Davy, D. Warde-Farley, A. Biard, A. Courville, Y. Bengio, et al., "Brain tumor segmentation with deep neural networks," Medical image analysis, vol. 35, pp. 18-31, 2017.
[39] W. Wang, C. Chen, M. Ding, H. Yu, S. Zha, and J. Li, "Transbts: Multimodal brain tumor segmentation using transformer," in Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part I 24, 2021, pp. 109-119.
[40] H. Messaoudi, A. Belaid, M. L. Allaoui, A. Zetout, M. S. Allili, S. Tliba, et al., "Efficient embedding network for 3D brain tumor segmentation," in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 6th International Workshop, BrainLes 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4, 2020, Revised Selected Papers, Part I 6, 2021, pp. 252-262.
[41] J. Colman, L. Zhang, W. Duan, and X. Ye, "DR-Unet104 for Multimodal MRI brain tumor segmentation," in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 6th International Workshop, BrainLes 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4, 2020, Revised Selected Papers, Part II 6, 2021, pp. 410-419.
[42] L. Mora Ballestar and V. Vilaplana, "Brain Tumor Segmentation using 3D-CNNs with Uncertainty Estimation," arXiv e-prints, p. arXiv: 2009.12188, 2020.
usamabajwa@cuilahore.edu.pk
Co-PI, Video Analytics lab, National Centre in Big Data and Cloud Computing,
Program Chair (FIT 2019),
HEC Approved PhD Supervisor,
Tenured Associate Professor
Department of Computer Science,
COMSATS University Islamabad, Lahore Campus, Pakistan
Research Scholar
Muhammad Sohaib Iqbal
Research Associate (RA), COMSATS University Islamabad Lahore Campus
Work Email: fa21-rcs-005@cuilahore.edu.pk / sohaibiqbal@cuilahore.edu.pk
Personal Email: cssohaibiqbal@gmail.com