Advanced optimization methods for inverse problems & applications to image microscopy

Title: 3D super-resolution by TIRF-MA reconstruction and molecule fluctuations

Abstract: Fluorescence microscopy is limited in resolution by the light diffraction barrier. Several super-resolution techniques have been proposed. Among them an emerging one leverages natural random molecule fluctuations, introducing diversity which is exploited to reconstruct super-resolved image in the 2D lateral plane (orthogonal to the objective axis). Such super-resolved setup does not need specific material (microscopes or specific fluorophores), is not harmful for the biological sample and preserves temporal resolution. Based on the acquisition of a sequence of images at high rate, typically 100 images per second, the reconstruction method exploit first and second order moments in a variational approach. The support of the sample together with the noise variance are estimated from the covariance, then the intensity within the support together with the background are estimated from the mean. All hyper-parameters are estimated giving a fully automatic procedure.

To the best of our knowledge, this approach is the only one which estimates the intensity level in the super-resolved image. This allows us to combine this super-resolution procedure in the 2D lateral plane with a Multi-Angle TIRF (Total internal refection fluorescence) microscopy which is able to access super-resolution in the axial direction, then producing a fully 3D super-resolved image in a thin layer up the coverslip.

We will conclude by presenting a new way to solve the inverse problem for the support reconstruction by using a GAN (Generative Adversarial Network) approach that takes into account the random distribution of observed image pixels in the data term.

Title: Position Estimation of 3D Spherical Beads in Confocal Microscopy via Poisson Denoising using Bregman Iterations

Abstract: Particle estimation is a classical problem arising in many science fields, such as biophysics, fluid mechanics and biomedical imaging. Many interesting applications in these areas involve 3D imaging data: such data may be corrupted by statistical noise and by a blurring linear operator. In this work we apply a Bregman iteration technique to each slice of the 3D scanned volume: the advantage of this technique is that it does not require a fine tuning of the regularization parameter and possesses a remarkable contrast enhancement property. The successive step consists in coupling the Total Variation functional and a regularized weighted Least Squares fit for the reconstruction of the particles' profile in each slice. Then, the 2D information is used to retrieve the 3D coordinates using geometrical properties. The experiments provide evidence that image denoising has a large impact on the performance of the particle tracking procedures, since they strongly depend on the quality of the initial acquisition. This work shows that the choice of tailored image denoising technique for Poisson noise leads to a better estimation of the particle positions.

Title: Three-Dimensional Inverse Scattering with Lippmann-Schwinger Model

Abstract: A broad class of imaging modalities involve the resolution of an inverse-scattering problem. While classical reconstruction methods were relying on linear approximation of the forward model, recent works have shown the benefit of considering nonlinear models that adhere more closely to the physic of the acquisition. In this talk, I will present an accurate and efficient implementation of the forward model that relies on the exact (nonlinear) Lippmann-Schwinger equation. It addresses several crucial issues such as the discretization of the Green function, the computation of the far field, as well as the estimation of the incident field. ​​I will then deploy this model in a regularized variational-reconstruction framework and show on both simulated and real data its ability to provide high quality reconstructions for difficult configurations (high contrasts, few illuminations). Finally, I will show how to exploit this inverse scattering method to recover the refractive index (RI) of biological samples from fluorescent acquisitions only (i.e., without phase measurements). This not only provides structural information about the sample (the RI), but also allows the correction of sample-induced aberrations in the fluorescent signal (functional information).

Title: Nearly exact discrepancy principle for low-count Poisson image restoration

Abstract: Effectiveness of variational methods for restoring images corrupted by Poisson noise strongly depends on a suitable selection of the regularization parameter balancing the effect of the regulation term(s) and the generalized Kullback-Liebler divergence data term. One of the approaches still most used today for choosing the parameter is the discrepancy principle proposed in (Zanella et al. 2009). It relies on imposing a value of the data term equal to its expected value and works well for mid- and high-count Poisson noise corruptions. However, the series truncation approximation used in the theoretical derivation of the expected value leads to poor performances for low-count Poisson noise. In this talk, we highlight the theoretical limits of the approach and then propose a nearly exact version of it based on Montecarlo simulation and weighted least-square fitting. Several numerical experiments are presented proving that in the low-count Poisson regime the proposed modified, nearly exact discrepancy principle performs far better than the original approximated one, with the two principles working similarly well in the mid- and high-count regimes.

Title: New advances on multiphoton microscopy image restoration. Application to 3D imaging of muscle ultrastructure.

Abstract: Multiphoton microscopy (MPM) is a contactless method devoted to sample imaging of various objects, from living kingdom to materials. MPM allows the in-vivo observation of 3D images of targets with a sub-micrometric resolution up to several hundreds of micrometers under the sample surface without sample slicing nor dyeing procedure. However, deep 3D MPM suffers from wavefront distorsions, spherical abberation and scattering phenomena resulting in a space varying blur and significant noise level. In this talk, I will introduce a new computational pipeline named FAMOUS: Fast algorithm for 3D multiphoton microscopy of biomedical structures (Lefort et al., 2021). This solution integrates a Gaussian fitting method for quantitative evaluation of the variations in depth of the 3D PSF (Chouzenoux et al., 2019), and a fast 3D image deblurring approach accounting for a model for depth-variant PSF (Chalvidal et al, 2020). The practical performance of FAMOUS are demonstrated on the 3D imaging of muscle ultrastructure.

This is joint work with Claire Lefort, (XLIM, CNRS), and Jean-Christophe Pesquet, (CentraleSupélec, Univ. Paris Saclay).

Title: Learning low-dimensional models of microscopes

Abstract: Linear operators appear in microscopy as a model for the spatially varying blurs an acquisition device impairs images with. In real-world applications, the operators are very large objects thus computationally intensive to deal with. To bypass this burden, state-of-the-art methods rely on the coarse approximation of the operators by convolutions. In this talk we will first investigate some tools to efficiently represent and estimate these operators. Then, we will present a method to learn a low dimensional subset of operators from the observation of the action of a few of them on micro-beads images. This computationally efficient procedure allows to accurately calibrate microscopes whose physics of degradation relies on a small number of parameters (even though the physics might seem complex at first). This approach is an essential step towards the effective resolution of blind (or semi-) blind inverse problems.

Title: Optimization methods for inverse problems in imaging: new results and challenges.

Abstract: The recent literature shows that one of the most reliable ways for facing imaging problems is based on their variational formulation. The more and more sophisticated models which have been developed for specific imaging applications stimulated the design of numerical optimization methods able handle their intrinsic difficulties. This talk discusses some recent theoretical and implementation advances in the framework of first order optimization methods, highligting also the more challenging possible future developments.

Title: Scaled, adaptive and generalized FISTA algorithm for sparse image super-resolution problems

Abstract: In this talk we propose SAGE-FISTA, a Scaled Adaptive GEneralized FISTA-type algorithm for the minimization of the sum of two (possibly strongly) convex functions. The proposed algorithm employs an adaptive backtracking procedure, which allows for the non-monotone adjustment of the steplength along the iterations. Further acceleration is provided by introducing a variable scaling matrix in the computation of the proximal-gradient step. If one or both functions are strongly convex, we prove that SAGE-FISTA converges linearly in the function values, with a convergence factor depending on both the strong convexity moduli of the two functions and the upper and lower bounds on the eigenvalues of the scaling matrices. Otherwise, we show that SAGE-FISTA retains the same O(1/k^2) convergence rate available for FISTA. Furthermore, we apply our algorithm to an image super-resolution problem where a sparsity-promoting regularization function is coupled with a weighted least squares term. Our numerical experiments show that SAGE-FISTA boosts the practical convergence speed with respect to standard implementations of FISTA, especially when used as inner solver of iteratively reweighted ell_1 schemes.

This is joint work with Luca Calatroni (Laboratoire I3S, CNRS, UCA), Marta Lazzaretti and Claudio Estatico (University of Genova).

Title: DeepBlur: a deep learning framework for blind deblurring and super-resolution problems

Abstract: In this talk, I plan to present a recent work on the resolution of blind deblurring problems using convolutional neural networks. Assuming that the unknown blurring operator lives in a known low-dimensional manifold of operators, we propose a two step approach :

• train a neural network to identify the operator from the blurry image,

• train a second neural network to deblur the image using the estimated operator.

This requires the design of specific sampling strategies to cover the manifold of operators as densely as possible. This procedure provides excellent results compared to more traditional variational approaches.

This is a joint work with Valentin Debarnot.

Title: Fluorescence Laser-Scanning Microscopy with Single-Photon Detector Array: A New Class of Multi-Dimensional Inverse Problems

Abstract: The last twenty years have seen a blooming of new approaches to improve the spatial resolution of fluorescence microscopy, thus opening to new exciting observations of biomolecules at work in living cells. These so-called super-resolution techniques achieve sub-diffraction resolution by combining new photonics technologies, image reconstruction methods, and fluorophore’s photophysical properties. In this scenario, we have recently introduced a new single-photon detector array for laser-scanning-microscopy, which provides access to spatiotemporal dimensions typically discarded by conventional detectors (Buttafava et al., 2020). Based on this information-rich multi-dimensional dataset, we implemented a series of computational methods to improve the resolution of many laser-scanning microscopy techniques, e.g., confocal, two-photon and stimulated-emission depletion microscopy (Castello et al., 2019, Koho et al., 2020, Tortarolo et al., 2020). In this talk, we will present the above methods from the inverse problem viewpoint. We will describe the general forward model linking the distribution/concentration of biomolecules inside the sample to the multi-dimensional intensity dataset -- obtained by the new detector. We will derive a series of inverse problems for 2D (or 3D) super-resolved imaging, and we will show a solution for only a few of them while keeping open a discussion for the others. We will finally consider the single-photon nature of the detector, thus its ability to generate true photon-couniting (i.e., Poissionian distributed) measurements.
This is joint work with Giorgio Tortarolo and Alessandro Zunino.