June 2020 -  Dec 2022

The problem of single image super-resolution (SISR) using convolutional neural networks has been extensively explored in the computer vision field. Current architectures perform well in both the qualitative and quantitative output evaluation of optical images, but applications to experimental data in the physical sciences have not been well explored. This data is often gathered using other modalities, producing images that differ both in appearance and encoded information. One example of this is electron backscatter diffraction (EBSD), a widely used materials characterization technique that produces maps describing the crystal arrangement of solid materials. These maps are critically important to the understanding of materials behavior in a wide array of applications ranging from biomedical to aerospace. The ability to generate a physically accurate, high-resolution EBSD map from a captured low-resolution map would significantly impact both science and industry, allowing for the capture of three-dimensional EBSD experimental data in a high-throughput manner and with high spatial resolution. However, as EBSD maps are built by indexing electron diffraction patterns that follow crystallographic symmetry conventions, we cannot directly apply state-of-the-art SISR networks to generate physically accurate high-resolution images. For the neural network to have an appropriate understanding of the relevant domain knowledge, physics-based information must be incorporated into the learning process. Therefore, we introduce a novel physics-based loss for network training that accounts for both orientation representation and crystallographic symmetry. We apply this approach to EBSD data of a commercially relevant titanium alloy, and prove both quantitatively and qualitatively that the generated results using physics-based loss are significantly better than those achieved using commonly used L_1 loss approaches or traditional upsampling algorithms such as bilinear, bicubic, and nearest neighbor. Additionally, we present a new inference pipeline to convert network output into established visualization formats for EBSD maps.

[Paper] [Code] [Bisque Module] [Poster]

June 2021 -  Sept  2021 (Patent in progress)

Images captured from a smartphone are processed through lens, sensor, and different processing blocks before being saved as jpeg images in the storage. First, the captured light-rays from the lens go through the Bayer sensor and forms Bayer Raw images. Then, the Bayer Raw images are processed through an image signal processing (ISP) block and finally saved as RGB jpeg images. The ISP pipeline contains different processing modules such as white balance, de-mosaicking, de-noising, color space transform, tone reproduction, etc. Some of the processing modules in the ISP pipeline, such as de-mosaicking and de-noise, introduce blurs in the images. The blurs due to de-mosaicking and de-noise are called in-processing blurs. The other blur, which comes from the optical system, such as the lens of mobile cameras, is called lens blur. The in-processing blurs and lens blur worsens in mobile captured images during non-ideal conditions such as high-noise and low-light conditions. The current exiting non-AI-based methods cannot remove these blurs and sharpen images, and AI-based methods have shown better performance in other image restoration tasks such as de-blurring, de-noising, super-resolution, etc. Therefore, it becomes crucial to design an AI-based network to remove in-processing and lens blurs and sharpen images. Due to the non-availability of publicly available in-processed and lens blur datasets, we have designed our data generation pipeline to simulate both in-processing and lens blur in real mobile camera images and synthetic images generated from the dead leaves model. Our designed deep neural network architectures are trained on these simulated blurred datasets. We have shown that our networks remove in-processing and lens blurs and sharpen mobile images with reasonable computational complexity.  

[This work was done during summer'21 internship at Samsung Research America , and submitted for patent. ]

May 2014 -  July  2014

In this project, we consider a Maximum Likelihood (ML) detection approach for signals which are corrupted by Middlenton Class A noise. These signals are transmitted from source to destination through N number of Amplitude and Forward relays. There already exists a rich literature on copoperative diversity but these results are mainly restricted to the conventional assumption of additive white gaussian noise (AWGN). There are few papers which consider impulsive nature of noise. We know, Maximum Ratio Combining (MRC) is optimal detector in presence of AWGN but it is not true in case of impulsive noise. Most of the papers talk about MRC detection scheme in A-F relays system in prsence of implusive noise but not ML detection scheme. We performed the simulation for both detection schemes (ML and MRC). We found that performance of the ML detection scheme is much better than MRC detection scheme.

[Report]

[This work was done during summer internship at Jacobs University, Bremen as DAAD-WISE Scholar]

[Image Source: BITS Wordpress]