AI4 Healthcare 2019

Medical image data science for real time insights and diagnostic support

Image data science (IDS) is a fast growing field within data analytics that focuses on insights obtained directly from pixels. It offers a blend of practices from image analysis, computer vision, machine learning and data science into a single framework facilitating human to data space interactions. Interactions are in the form of human-readable queries on the image information content either from the raw image space or associated feature spaces. Queries enforce constraints on an underlying image representation data structure and return sets of pixels representing meaningful objects in the image. The latter are attributed using shape, size and radiometric features to obtain a unique 'feature signature'. Feature signatures allow the chaining of queries with conventional data science workflows.

Image data science can be used to (i) carry out exploratory image data analysis prior to the training/deployment of machine learning algorithms, (ii) to compute high precision user driven image segmentation in the absence of training data/models, and (iii) to deliver data insight reports on the image information content.

In the medical image analytics front, statistical methods are often limited by the lack of sufficient training images and the very high cost of professional annotation services by radiologists. Image segmentation found at the center of most automated diagnostic methods, is confronted by the zero tolerance to pixel value imputation. The latter is a side effect of deep learning-based instance segmentation algorithms that rely on the convolution operator. Feature spaces generated by the application of convolutions account in part for elements of the background that appear to be filling the 'blanks'. This is either due to noise artifacts overlapping with the anatomical features of interest, the inability of the operator to capture fine, pixel-sized detail discriminating foreground from background, or the lack of sufficient training examples. The condition is referred to as foreground contamination and is a side-effect of many image analysis operators including active contours and surfaces. Moreover, extensions of instance segmentation algorithms in 3D are ill defined and can be computationally very expensive.

Introducing image data science frameworks in medical image analytics addresses all of the aforementioned limitations and enables for fast, interactive and precise interactions with the image information content. Training data may be used in the form of injected knowledge but is not a strict requirement to run supported workflows. Segmentation of anatomical features is computed in realtime by manual feature selection and tuning, interactive visualization and adaptation of empirical knowledge in the form of location, intensity and shape deformation markers. Segment classification can be computed using pixel data and/or feature signatures along with the classifier of choice. Findings are used for diagnostics, pre-operation planning and insights reports.

This lecture will discuss an end-to-end image data science framework and associated technologies and demonstrate it in 2D and 3D medical image segmentation.

example of IDS for precise liver tumor segmentation