Future Work
1. Rebuild the model with 3D deconvolutional layers to capture information on all axes.
1. Rebuild the model with 3D deconvolutional layers to capture information on all axes.
- We are currently using 2D deconvolutional layers in the decoder of our model and believe that this is causing us to miss important information on the z-axis, since the the x- and y-axes and z-axis are not symmetric under this architecture. We could consider to redesign our model by using 3D deconvolutional layers so as to preserve symmetric among all axes.
2. Train the model by using augmentation (i.e. different X-ray projections such as rotation, scale, translation, source-to-isocenter distance, distortion etc.) for each patient.
2. Train the model by using augmentation (i.e. different X-ray projections such as rotation, scale, translation, source-to-isocenter distance, distortion etc.) for each patient.
- We have trained our model using only one X-ray for each patient. We plan to use different combinations of X-rays for each patient based on various projections to make our model more accurate in terms of predicting 3D volumes.
3. Develop a custom loss function.
3. Develop a custom loss function.
- In our model, we are currently using mean squared error – which is not context-aware. We believe that using a custom loss function that gives more importance to clinically-relevant parts of the chest will lead to better predictions.
4. Build an ensemble model trained to predict different aspects (i.e. bone, air, and soft tissue).
4. Build an ensemble model trained to predict different aspects (i.e. bone, air, and soft tissue).
- We plan to construct an ensemble model that consists of different parts trained to predict different tissues/features.
5. Narrow focus of model to clinically relevant aspect.
5. Narrow focus of model to clinically relevant aspect.
- Now that we have a proof-of-concept that a basic volume can be recovered from a chest x-ray, it would be worthwhile to discuss clinically relevant possibilities of such a model with physicians. Not all features of the 3D volume are equally relevant for certain tasks.
- For example, we could focus on resolving the 3D structure of the airways. This could guide bronchoscopy in areas without access to CT scanners.
- Alternatively, 3D reconstruction of tubes and lines relative to the lung space could help nurses confirm correct placement of a line/tube without referral to a radiologist.
6. Add another view as input to the model (lateral chest x-ray).
6. Add another view as input to the model (lateral chest x-ray).
- Increasing the number of views input into the model ought increase the accuracy of the model. From our CT dataset, we could generate an arbitrary number synthetic x-rays as input. While adding many extra views might seem enticing, we want to keep our model input as close as possible to real clinical imaging.
- The only other standard view taken with an AP or PA chest x-ray is a lateral view. We would not expect a chest x-ray series depicting anything but the coronal (AP/PA) or sagittal (lateral) plane. A model that expected input from more than those two vantage points would undoubtedly achieve greater accuracy, but at the same time lose any clinical applicability.