Abstract

The past few years have witnessed the great success of applying deep neural networks (DNNs) in computer-aided diagnosis. However, little attention has been paid to provide pathological evidence in the existing DNNs for medical diagnosis. In fact, feature visualization in DNNs is able to help understanding how the computer make decisions, and thus it shows promise on finding pathological evidence from computer-aided diagnosis. In this paper, we propose a novel pathology-aware visualization approach for DNN-based glaucoma classification, which is used to locate the pathological evidence from fundus images for glaucoma. Besides, we apply the visualization framework to the glaucoma images synthesis task, through which specific pathological areas of synthesized images can be enhanced. Finally, experimental results show that the visualization heat maps can pinpoint different glaucoma pathologies with high accuracy, and that the generated glaucoma images are more pathophysiologically clear in rim loss (RL) and retinal neural fiber layer damage (RNFLD), which is verified by the ophthalmologist.

Method

Pathology-aware feature visualization

To visualize the pathological evidence of the glaucoma classification network, we propose a novel pathology-aware feature visualization approach. Our approach consists of two main procedures: pathology-aware feature selection and gradient-based combination. The pathology-aware feature selection includes two subprocess: (1) The features of a pre-trained classification network are selected by their activation values; (2) the features are further screened and divided into two groups by their centroid-centric moment of inertia (CMI) values. Finally, the two groups of features are combined separately in a gradient-based weight and then output the visualization heat maps. The framework is shown in Figure. 1 (Upper).

Visualization-based fundus image synthesis

The Patho-GAN is achieved by combining a basic GAN and our visualization system. The framework of Patho-GAN is shown in Figure 1. The overall structure of Patho-GAN consists of three subnets: generator net, discriminator net and pathology-aware visualization net. Given the vessel segmentation image (generated by the method proposed in [13]) and a noise code as input, the generator tries to synthesize images, while the discriminator net tries to tell apart the synthesized images from the real ones. The visualization net enforces the synthesized image to have the similar visualization results to the reference image.

Result

Publication

1. Xiaofei Wang, Mai Xu, Liu li and Zulin Wang, "Pathology-aware deep network visualization and its application in glaucoma image synthesis" in Medical Image Computing and Computer Assisted Interventions, 2019.