PRELIMINARY Results

Patient 0540 in the LIDC-IDRI Dataset

Synthetic X-Ray Input

Predicted CT-Scan

Actual CT-Scan

Histogram of the Hounsfield intensities of the actual patient CT after thresholding the lower bound at -1000 HU.

Histogram of the Hounsfield intensities of the predicted patient CT after thresholding the lower bound at -1000 HU. Notice a smoother distribution.

Actual CT-scan of Patient 0540

Predicted CT-scan of Patient 0540

Coronal slices from the actual CT-scan of Patient 0540

Coronal slices from the predicted CT-scan of Patient 0540

The mediastinum (heart + other structures mid-chest) and vertebral column (bright white) especially are visible in the predicted scan.
The breasts are present on our predicted scan even though it is scarcely detectable on the chest X-ray.
The lung shape and size is very similar.
Pearson correlation coefficient: -0.1279 (p = 0.0413)

Patient 0010 in the LIDC-IDRI Dataset

Synthetic X-ray Input

Predicted CT-scan

Actual CT-scan

Actual CT-scan of Patient 0010

Predicted CT-scan of Patient 0010

Coronal slices from the actual CT-scan of Patient 0010

Coronal slices from the predicted CT-scan of Patient 0010

The shape of the lung in the predicted scan are consistent with the overall shape of the actual lung.
The apex of the lung in the predicted output is skewed posteriorly.
The overall body structure was not well-preserved in this example.
Pearson correlation coefficient: -0.05 (p = 0.4261)

Histogram of the Hounsfield intensities of the actual patient CT after thresholding the lower bound at -1000 HU.

Histogram of the Hounsfield intensities of the predicted patient CT after thresholding the lower bound at -1000 HU. Notice a smoother distribution.

Summary

The predicted CT volume are able to reproduce several unique characteristics of the input volume, especially the lung.
The quality of the CT volume are far from ideal but is able to consistently predict the size and shape of lungs.
It appears as though the model mainly focused on resolving air vs not air. Bone was not well preserved in the 3D transformation, despite being easily viewable on the chest X-ray. This was contrary to our expectations. In hindsight, we think our loss function penalized false bone intensities more (as it lies at 1000+ hounsfield units, whereas air and tissue are around -1000 and 0 HU, respectively), leading our model to avoid predicting bone intensities in all but the safest cases (vertebral column). We suspect that training a model that resolves the skeleton separately would be a way to combating this.
We could not determine a valid quantitative metric by which to judge the similarity of the reconstructed 3D volumes and the target CT scans.
- Initially we were planning on testing if lung nodules were correctly resolved, however our results were too coarse for this to be a relevant metric.
- The Henzler et al. paper used L2 (Euclidean Loss) as a metric, however as noted by the authors, L2 "is often not well-correlated with the perceived quality of a re-construction that is dominated for 'what’s in” for the final image.'" We also found this to be the case. The L2 loss gives little-to-no information as to whether the holistic structure is preserved in a human-interpretable manner.
- In lieu of quantitative metrics, we relied on qualitative judgement from medically trained individuals (including group member Kendall, a medical student) to determine if relevant features were preserved.
Our work was primarily aimed at figuring out if 3D reconstruction of a single X-ray via deep learning is even possible. To our best knowledge, no research groups have attempted this before. We believe our results show that deep learning has the potential to "learn" normal human anatomy and successfully extrapolate 3D features from the 2D image.
Note: Because we intend to continue this project in the summer, we have limited our exploration of the test set results to preserve experimental integrity in our future work.

Google Sites

Report abuse