Machine Learning for
High-Throughput
Microscopic Imaging

  • Dynamic Sparse Sampling for High-Throughput Raman Spectroscopy (IEEE TCI 2018, Analytical Chemistry 2018)

Fig. 1. Confocal Raman Imaging for Polymorphic Characterization of Active Pharmaceutical Ingredients. Top: SLADS, Bottom: Random Sampling. Each shows the measured locations and corresponding reconstruction. For reference, we also show the reconstruction error from fully-measured ground truth image. SLADS mainly finds the informative samples along the edge generating high-fidelity reconstruction compared with random sampling.

Raman spectroscopy is a powerful tool to provide detailed chemical information in multiple dimensions (spatial and spectral). Imaging based on Raman spectroscopy requires relatively long integration times to obtain sufficient signal to noise, which in turn limits the applications of Raman imaging. Dynamic sparse sampling methods have the potential to reduce the number of measurements needed for high-fidelity interpolation by finding the most informative measurement location based on previous measurements compared with a standard raster scheme. However, existing methods are computationally expensive due to the stochastic process (e.g., the Monte Carlo method) for calculating conditional expectations given measurements. We propose a Supervised Learning Approach for Dynamic Sparse Sampling (SLADS), which is efficient enough for practical application. SLADS finds a regression function that approximately relates in interpolation error with a new measurement to image features given measurements in an off-line training database. By applying the regression function to given measurements in the unknown testing image, SLADS then very rapidly estimates the conditional expectation, allowing for fast and accurate selection of the next measurement location. In Fig. 1, SLADS-enabled Raman imaging achieved a six-fold reduction in measurement time relative to full field of view rastering with negligible loss (< 0.5%) in image quality.

  • Deep Learning Sparse Sampling for High-throughput Mass Spectrometry Imaging (Nature 2019, Electronic Imaging 2021, NIH UG3HL145593, Code)

Fig. 2. Binary measurement masks and reconstructions of a mouse uterine sample with nano-DESI MSI. Top: SLADS, Bottom: DLADS. DLADS outperforms SLADS allowing for effective reconstructions at sampling densities as low as 10%..

Mass Spectrometry Imaging (MSI) is an emerging tool that enables label-free spatial mapping of different classes of biomolecules in biological systems. However, the throughput of ambient MSI is typically limited by the inherently slow microprobe-type sampling from surfaces, which is a characteristic shortcoming of many chemical imaging modalities. To address this challenge, we propose an enhanced SLADS method, called Deep Learning Approach for Dynamic Sampling (DLADS). DLADS removes the need for a simple regression based on gradient features, by employing deep convolutional networks to find the most informative samples leveraging inter-pixel spatial relationships. DLADS demonstrates 25% reductions in acquisition times over SLADS for high-resolution visualization of mouse uterine tissues with Nanospray Desorption Electro-Spray Ionization (nano-DESI) MSI. (See Fig. 2)