This project has four main objectives.
1. To perform segmentation with deep learning on lung CT images to distinguish tumor areas from surrounding tissues and to identify tumor boundaries.
2. To develop a deep learning model for segmentation called Lung Tumor Deepedit Segmentation (LTDeS) by selecting the U-Net architecture from deep learning methods.
3. To develop a 3D Simulation-based Thoracic Biopsy and Surgical Assistance application (3D PulmoSIM) for preoperative pulmonary biopsy planning, preoperative safe pulmonary surgery planning (minimally invasive/non-invasive assistance in lung cancer surgery), thoracic surgery assistant training, and doctor-patient communication, based on the segmented tumor's location and size.
4. To determine the opinions and attitudes of thoracic surgeons regarding the use of the 3D PulmoSIM application.
The goal of this project is to select appropriate data for efficient and accurate segmentation of lung tumors in CT images. The data has been carefully selected, considering different types, sizes, and locations, to enable the model to segment with a higher success rate on new data during training.
In this project, publicly available datasets from The Cancer Imaging Archive (TCIA) were used. The Cancer Imaging Archive (TCIA) is an organization that has been supporting cancer researchers since 2011 by providing researcher-focused, de-identified, and highly curated radiology and histopathology imaging resources under the National Cancer Institute (NCI) and National Institutes of Health (NIH).
The TCIA datasets used in this project are NLST, LCTSC, NSCLC-Radiomics, and RIDER Lung CT datasets, all obtained from lung cancer patients.
For training the dataset, the data was divided into validation and test sets. A U-Net architecture-based model was developed using appropriate loss functions (e.g., Dice loss) and optimization algorithms (e.g., Adam) to fine-tune model weights and achieve optimal segmentation results. The Lung Tumor Deepedit Segmentation (LTDeS) deep learning model performs lung and tumor segmentation on CT images. Grid search experiments were conducted to optimize hyperparameters such as learning rates, batch sizes, and U-Net layers. Cross-validation techniques were used to improve accuracy and prevent overfitting.
The DeepEdit architecture combines both interactive and non-interactive segmentation methods into a single deep learning model. It uses an active learning strategy to rank unlabeled volumes from most to least uncertain. For tumor annotation, the 3D Slicer software was used, and the segmentation was verified by a radiology expert before model training.
Trial-and-error experiments determined the optimal metrics and parameters, with an input image size of 192x192x192 as the best fit for the available GPU capacity. Standard metrics like Dice score, accuracy, precision, and recall were used to evaluate the model's performance. Post-processing techniques like thresholding and connected component analysis were applied to refine segmentation masks.
Results from training with 86 epochs showed a validation mean Dice score of 0.795, with tumor and lung Dice scores of 0.816 and 0.971, respectively. These metrics indicate the model’s effectiveness in segmenting lung tumors, which is critical for early diagnosis and treatment planning in lung cancer.
The LTDeS model was implemented using the MONAILABEL extension in 3D Slicer, with performance metrics calculated through standard binary cross-entropy and Dice loss, commonly used in biomedical imaging evaluation.
The Tumor and Lund Dice Score Graph
Overall Loss Graph
Lung cancer surgery and biopsy operations are quite challenging due to the complex anatomical structure of the lung (bronchi, pulmonary arteries, veins). The 3D Simulated Thoracic Biopsy and Pulmonary Surgery Assistance Application (3D-PulmoSIM) was developed after receiving feedback from doctors in the surgical department at Süleyman Demirel University (SDU) medical faculty. It was determined that there is a need for a 3D thoracic biopsy and pulmonary surgery simulation environment for preoperative planning/preparation, medical student training, and patient education.
The goal of the 3D-PulmoSIM application is to assist doctors in planning operations by creating 3D visualizations of nodules (tumors) segmented using deep learning algorithms from CT images of lung cancer patients. Additionally, the 3D-PulmoSIM application is designed to be used in thoracic surgery education and for informing patients. It was designed using Unity3D.
Main Menu
Biopsy Interface
Surgery Interface
A demonstration meeting was held at the Department of Thoracic Surgery at Süleyman Demirel University Medical Faculty, where surgeons were introduced to the 3D-PulmoSIM application and given the opportunity to test it. Feedback was collected regarding the application. The doctors' attitudes towards the "Ease of Use," "Usability and Efficiency," and "Future Use" of the 3D-PulmoSIM application were assessed.
The analysis of responses to semi-structured open-ended interview questions and participant experience surveys revealed that thoracic surgeons found the 3D-PulmoSIM application helpful for preoperative preparation, assistant training, and patient education. Furthermore, based on the participant experience survey findings, it was concluded that the participants had positive attitudes towards using the 3D-PulmoSIM application and found it efficient and easy to use. The application was found to be easy to install and suitable for clinical applications.