Atmospheric Turbulence Mitigation

The Problem

Long range terrestrial observations using visible and infrared imaging systems are susceptible to degradations from atmospheric turbulence.   The degradations occur due to random fluctuations in the index of refraction along the optical path length, which are caused by variations in temperature.  Under anisoplanatic imaging conditions, the degradation is modeled by a spatially varying point spread function (PSF) in each frame, which generally results in a blurred and warped image.  By using short exposure frames, the spatially varying atmospheric PSF can be approximately “frozen” in time within each frame.   Multiframe image restoration techniques may be applied to these frames to produce restored imagery.   Keys to successful algorithms are accurate image registration and accurate modeling of the atmospheric PSF [1].

Working with researchers in industry at L-3 Cincinnati Electronics, Dr. Hardie led the algorithm development of a new real-time turbulence mitigation system [3-7].  This system is capable of performing turbulence mitigation and super-resolution (TMSR) jointly.  It also handles complex imaging conditions such as a moving camera platform and local scene motion.  

Dr. Hardie has also recently developed a new software tool for simulating extended scenes imaged through atmospheric optical turbulence under anisoplanatic conditions [2].    This tool allows researchers to simulate a variety of imaging conditions and imaging systems.  Most importantly, the simulation allows turbulence mitigation algorithms to be tested, tuned, and evaluated quantitatively.  This is because with the simulation, the true scene image is known exactly [1].

Turbulence mitigation example with real imagery from [1].  A single turbulence corrupted frame is shown on top, and the corresponding restored frame is shown on the bottom. 

Another turbulence mitigation example with real imagery.

Atmospheric optical turbulence simulation output frames [2].  Real turbulence is shown on the left, and simulated turbulence is shown on the right

Atmospheric optical turbulence simulation showing how each point in the scene passes through a slightly different portion of the atmosphere (modeled with phase screens).  This results in a different point spread function for each pixel from the scene [3]. 

Dr. Hardie gave an invited presentation at CVPR 2022 on June 20, 2022: "Atmospheric Optical Turbulence: Modeling, Simulation, and Mitigation" by Russell Hardie and Michael Rucci. Here is a link to the video presentation.

Selected References

1. Michael A. Rucci, Russell C. Hardie, Richard K. Martin, and Szymon Gladysz, "Atmospheric optical turbulence mitigation using iterative image registration and least squares lucky look fusion," Appl. Opt. 61, 8233-8247 (2022) 

2. Michael A. Rucci, Russell C. Hardie, Richard K. Martin, and Szymon Gladysz, "Atmospheric optical turbulence mitigation using iterative image registration and least squares lucky look fusion," Appl. Opt. 61, 8233-8247 (2022) 

3. Russell C. Hardie, Michael A. Rucci, Santasri R. Bose-Pillai, Richard Van Hook, Barry K. Karch, "Modeling and simulation of multispectral imaging through anisoplanatic atmospheric optical turbulence," Opt. Eng. 61(9) 093102 (10 September 2022) https://doi.org/10.1117/1.OE.61.9.093102

4. Richard L. Van Hook and Russell C. Hardie, "Scene motion detection in imagery with anisoplanatic optical turbulence using a tilt-variance-based Gaussian mixture model," Appl. Opt. 60, G91-G106 (2021) 

5. Matthew A. Hoffmire, Russell C. Hardie, Michael A. Rucci, Richard Van Hook, Barry K. Karch, "Deep learning for anisoplanatic optical turbulence mitigation in long-range imaging," Opt. Eng. 60(3) 033103 (23 March 2021) https://doi.org/10.1117/1.OE.60.3.033103

6. Russell C. Hardie, Michael A. Rucci, Santasri Bose-Pillai, and Richard Van Hook, "Application of tilt correlation statistics to anisoplanatic optical turbulence modeling and mitigation," Appl. Opt. 60, G181-G198 (2021) 

7. Michael A. Rucci, Russell C. Hardie, and Richard K. Martin, "Simulation of anisoplanatic lucky look imaging and statistics through optical turbulence using numerical wave propagation," Appl. Opt. 60, G19-G29 (2021) 

8. R. C. Hardie, M. Rucci, B. K. Karch, A. J. Dapore, D. R. Droege, and J. C. French, "Fusion of interpolated frames superresolution in the presence of atmospheric optical turbulence," Opt. Eng. 58(8) 083103 (22 August 2019) .

9. Russell C. Hardie, Michael A. Rucci, Alexander J. Dapore, Barry K. Karch, "Block matching and Wiener filtering approach to optical turbulence mitigation and its application to simulated and real imagery with quantitative error analysis," Opt. Eng. 56(7), 071503 (2017), doi: 10.1117/1.OE.56.7.071503.

10. Russell C. Hardie, Jonathan D. Power, Daniel A. LeMaster, Douglas R. Droege, Szymon Gladysz, Santasri Bose-Pillai, "Simulation of anisoplanatic imaging through optical turbulence using numerical wave propagation with new validation analysis," Opt. Eng. 56(7), 071502 (2017), doi: 10.1117/1.OE.56.7.071502.

11. D. R. Droege, R. C. Hardie, B. S. Allen, A. J. Dapore, and J. C. Blevins, “A Real-Time Atmospheric Turbulence Mitigation and Super-Resolution Solution for Infrared Imaging Systems,” Proceedings of SPIE Defense Security and Sensing, Baltimore, MD, USA, April 23-27, 2012.

12. R. C. Hardie, D. R. Droege, B. S. Allen, A. J. Dapore, J. C. Blevins, and K. M. Hardin “Real-Time Video Processing for Simultaneous Atmospheric Turbulence Mitigation and Super-Resolution and its Application to Terrestrial and Airborne Infrared Imaging,” MSS Passive Sensors, Pasadena, CA, USA, March, 2012.

13. R. C. Hardie, D. R. Droege, and K. M. Hardin, “Real-Time Atmospheric Turbulence Correction with Moving Objects,” Proceedings of MSS Passive Sensors, Orlando, FL, February, 2011.

14. R. C. Hardie, D. R. Droege, and K. M. Hardin, “Real-Time Atmospheric Turbulence Correction for Complex Imaging Conditions,” Proceedings of MSS Passive Sensors, Orlando, FL, February, 2010.

15. R. C. Hardie  and D. Droege, “Atmospheric Turbulence Correction for Infrared Video ,” Proceedings of MSS Passive Sensors, Orlando, FL, 23-26 February 2009.

Video showing estimated optical flow of geometric warping due to atmospheric turbulence.