Brain Tumor Segmentation using 3D Deep Residual U-Net (dResU-Net) from Multimodal MRI Images

Glioma is the most prevalent and dangerous type of brain tumor which can be life-threatening when its grade is high. The early detection of these tumors can improve and save the life of the patients. The automatic segmentation of brain tumors from magnetic resonance imaging (MRI) plays a vital role in treatment planning and timely diagnosis. Automatic segmentation is a challenging task due to the massive amount of information provided by MRI and the variation in the location, type, and size of the tumor. Therefore, a reliable and authentic method to segment the tumorous region from healthy tissues accurately is an open challenge in the field of deep learning-based medical image analysis. This thesis presents an end-to-end framework for automatic 3D Brain Tumor Segmentation (BTS). The proposed model is a hybrid of the deep residual network and U-Net model (dResU-Net). The residual network is used as an encoder in the proposed architecture with the decoder of the U-Net model to handle the issue of vanishing gradient. The proposed model is designed to take benefit from low and high features. In addition, shortcut connections are employed in residual convolutional blocks and skip connections between residual and convolutional blocks are utilized in the proposed architecture to accelerate the training process. Furthermore, the proposed dResU-Net model is also compared with modified 3D U-Net and pre-trained ResU-Net34 and ResU-Net50 to check the performance of the proposed model over the pre-trained models. To demonstrate the robustness of the proposed model in real-world clinical settings, validation of the trained model on an external cohort is performed on randomly selected 50 patients of the BraTS 2021 benchmark dataset.

The human brain is one of the most curious and integral parts of the human body that incorporate reasoning, thinking and is responsible for coordination and parity meaning voluntary movements, coordination and balance, executive planning, language, memory and learning, mathematical logic, and emotional responses. As the brain is the center of the nervous system and protected within the skull, any abnormal conduct or working of it might cause a breakdown of the whole-body functionalities [1]. Several diseases can affect the overall functioning of the brain like stroke, infection, and tumors, from which brain tumor is the most fatal disease. Brain tumors are the collection of abnormal cells in the human brain that can damage the nervous system and harm nearby healthy brain tissues. Brain tumors can affect the overall functioning of the brain and are known as one of the most dangerous diseases in humans [2]. According to the survey of [3], brain tumors cause more than 13000 death every year in the USA. Brain tumors are classified into primary and secondary brain tumors. The tumorous cells that originate inside the brain are termed primary brain tumors. While other cells that grow somewhere else in the body, travel through the blood and belong to metastatic brain tumors are termed secondary brain tumors [2] [4]. Depending on the severity of tumorous cells, brain tumors are also characterized as malignant brain tumors and non-malignant brain tumors. Glioma is a form of primary brain tumor that occurs in the brain and is one of the most prevalent. It develops from glial cells and is divided into two grades: high-grade glioma (HGG) and low-grade glioma (LGG) [5].

According to the report of the World Health Organization (WHO), based on tumor behavior and microscopic images, gliomas are graded from level one to four, from the least aggressive to the most aggressive type of tumor. Grade I and II are slow-growing cases classified as LGG, whereas grades III and IV are cancerous and aggressive and quickly spread themselves are classified as HGG [5]. The survival rate and diagnosis of HGG are still very poor. HGG tumors can be seen with heterogeneous mass with surrounding edema in MRI images. HGG is more malignant, requiring surgery and radiotherapy, and has a two-year life expectancy. In contrast, LGG can be benign or malignant, and grows more slowly, with a life span of several years [6]. LGG can take several years to grow but mostly grows into the HGG over time. LGG tumors have a wide range of imaging features that explain their histology types. LGG tumors can be seen on the T2w MRI image modality and can be seen with diffused boundaries or focal forms with clear boundaries. Moreover, the visibility of the LGG tumors can be further enhanced by tuning the image contrast.

The diagnosis of gliomas includes chemotherapy, radiation therapy, and surgery. Due to the varying size and diverse location of gliomas, the detection of the tumor is a challenging task for radiologists. Since the successful treatment of the tumor is mainly dependent upon the location, type of the tumor, and its grade. Therefore, segmentation of the tumor is necessary for planning the treatment [7] MRI is the most extensively utilized among them in brain tumor diagnosis procedures [5] [7] due to its sharpness and tissue resolution and provide a detailed representation of the internal structure of the body. Different parameters can be used to create a distinct anatomical tomographic image. MRI is a non-invasive and advanced imaging modality in medical imaging analysis that depends on nuclear magnetic resonance (NMR). MRI uses strong magnetic waves ranging from 1.5T to 3T and radiofrequency waves that produce a detailed representation of internal body structure and tissues [5] [9]. MRI is used to extract detailed information about brain cells without worrying about high ionization and radiation effect [9]. MRI contains four types of image modalities: T1- weighted (T1w), T1-weighted contrast-enhanced (T1c), Fluid-Attenuated Inversion Recovery (FLAIR), and T2-weighted (T2w). T1w is mostly used to measure healthy tissues. T2w has a bright tumor region, while T1c has a bright tumor border. The FLAIR scan, on the other hand, is persuasive for isolating the edema distinct from the CSF (cerebrospinal fluid) [10].

In this research work, a novel hybrid approach for the brain tumor sub-regions segmentation is being introduced that is based on a deep residual network and U-Net network (dResU-Net) to make the training process faster and more robust. In the proposed model, residual blocks are utilized in the encoder part of the U-Net network. The central motivation behind using these blocks in the encoder is to utilize both high and low-level features by exploiting skip connections to predict segmentation masks. Furthermore, residual blocks also help to overcome the vanishing and gradient problems during the training of the proposed model.

• Manual segmentation is an expensive, time-consuming and tedious task. But due to the emergence of Deep Learning methods in medical imaging, we can automate this task.

• Multimodal Brain Tumor Segmentation Challenges (MICCAI BRATS).

• To help the radiologist to detect the exact location of the tumor in terms of speed and accuracy.

• Early diagnosis of Brain Tumor which may reduce the death rate and save the life of the patient in its early stage.

• BraTS 2012

• BraTS 2013

• BraTS 2014

• BraTS 2015

• BraTS 2016

• BraTS 2017

• BraTS 2018

• BraTS 2019

• BraTS 2020

• BraTS 2021

Dataset Link (BraTS-2020): https://www.med.upenn.edu/cbica/brats2020/data.html

Dataset Link (BraTS-2021): https://www.med.upenn.edu/cbica/brats2021/

Research Challenges

• Brain Tumors may appear in any location inside the human brain with different shape and size.

• Ambiguous and Fuzzy boundaries between cancer and other brain tissues.

• The Boundaries between parts of the tumor structure are often unclear, which leads to disagreement between medical professionals when doing image classification.

• One of the main challenge in Medical Image Analysis is the limitation of 3D Labelled Data.

• High Computational Resources.

• High class imbalance dataset.

Research Gap

• Usage of MRI modalities independently

• Loss of contextual information

  • • • Uses of 2D Slices

      • Patch-wise Strategy

• Vanishing Gradient Issue

• High Class Imbalance (WT=61%, TC=24%, ET=15%)

A multi-modal magnetic resonance imaging-based segmentation of brain tumor for the accurate segmentation of the tumorous region from healthy tissues and timely diagnosis of brain tumor by using hybrid encoder-decoder-based Residual U-Net model.

1. Dice Score

2. Hausdorff  Distance - 95

3. Sensitivity

4. Specificity

5. Jaccard Simalarity

1. The performance of the proposed model can be improved by using different kind of Augmentation Techniques or by using a large benchmark dataset.

2. In future, the efficiency and effectiveness of ET class can be increase by removing False Positives Rates (FPR) by using post-processing techniques.

3. The results of proposed method can be improved by exploring and developing more 3D-based architectures and by using more powerful computational resources.

4. This research work can be extended to clinically challenged medical imaging problems and other segmentation problems i.e., LITS.

[1] Z. Liu et al., “Deep Learning Based Brain Tumor Segmentation: A Survey,” vol. 14, no. 8, pp. 1–21, 2020, [Online]. Available: http://arxiv.org/abs/2007.09479.

[2] M. Ghaffari, A. Sowmya, and R. Oliver, “Automated Brain Tumor Segmentation Using Multimodal Brain Scans: A Survey Based on Models Submitted to the BraTS 2012-2018 Challenges,” IEEE Rev. Biomed. Eng., vol. 13, no. c, pp. 156–168, 2020, doi: 10.1109/RBME.2019.2946868.

[3] E.-E. Mohd. Azhari, M. M. Mohd. Hatta, Z. Zaw Htike, and S. Lei Win, “Tumor detection in medical imaging: a Survey,” Int. J. Adv. Inf. Technol., vol. 4, no. 1, pp. 21–30, 2014, doi: 10.5121/ijait.2013.4103.

[4] S. A. L. I. R. Al-qazzaz, “Deep Learning-based Brain Tumour Image Segmentation and its Extension to Stroke Lesion Segmentation,” 2020.

[5] R. V. Tanneedi, P. Pedapati, and S. Johansson, “Brain tumour detection using HOG by SVM,” no. December, 2017, [Online]. Available: www.bth.se.

[6] X. Zhang et al., “A radiomics nomogram based on multiparametric MRI might stratify glioblastoma patients according to survival,” Eur. Radiol., vol. 29, no. 10, pp. 5528–5538, 2019, doi: 10.1007/s00330-019-06069-z.

[7] U. Baid et al., “A Novel Approach for Fully Automatic Intra-Tumor Segmentation With 3D U-Net Architecture for Gliomas,” Front. Comput. Neurosci., vol. 14, no. February, pp. 1–11, 2020, doi: 10.3389/fncom.2020.00010.

[8] J. Zhuang, J. Yang, L. Gu, and N. Dvornek, “Shelfnet for fast semantic segmentation,” Proc. - 2019 Int. Conf. Comput. Vis. Work. ICCVW 2019, pp. 847–856, 2019, doi: 10.1109/ICCVW.2019.00113.

[9] M. Havaei et al., “Brain tumor segmentation with Deep Neural Networks,” Med. Image Anal., vol. 35, pp. 18–31, 2017, doi: 10.1016/j.media.2016.05.004.

[10] M. I. Razzak, M. Imran, and G. Xu, “Efficient Brain Tumor Segmentation with Multiscale Two-Pathway-Group Conventional Neural Networks,” IEEE J. Biomed. Heal. Informatics, vol. 23, no. 5, pp. 1911–1919, 2019, doi: 10.1109/JBHI.2018.2874033.