Tutorial at the 8th National Conference on Computer Vision, Pattern Recognition, Image Processing, and Graphics (NCVPRIG) 2023
Ashish Tiwari and Shanmuganathan Raman
Computer Vision, Imaging, and Graphics Lab, IIT Gandhinagar
Abstract
Passive depth sensing in Computer Vision is an active research area as it has benefits over active depth sensors such as less cost, less power consumption, portability, and more depth range. These benefits, however, come at the expense of increased computation. While modern geometric passive techniques based on multi-view stereo and structure from motion have achieved faithful 3D shape reconstruction of objects and large-scale scenes, they fail to model the underlying surface reflectance, making the reconstruction “3D incomplete”. Photometric methods have excelled at recovering the thinnest geometric variation with superior quality and inferring the complete 3D information by jointly estimating the shape, material, and reflectance of the surface. Unlike the geometric methods which perform well only for textured opaque surfaces, photometric methods apply even to transparent, translucent, and textureless objects. Photometric Stereo spans applications such as industrial inspection, medical imaging, cultural heritage preservation, material characterization, robotics, and entertainment.
Photometric Stereo is a method to recover the underlying 3D surface from a series of images captured with a fixed camera, but with varying illumination. This tutorial will introduce and elaborate on a thread of deep learning-based methods for photometric stereo, with emphasis on uncalibrated and un/self-supervised settings. It will also highlight some key applications through recent advances and discuss interesting future research directions.
Topics
Session 1 : An Introduction to Photometric Stereo for 3D Reconstruction 22nd July 2023, 10:00 - 11:15 AM
1.1 Geometric Vs Photometric Methods: A Comparison
1.2 Basic concepts on Photometric Image Formation
1.3 Mathematical Principles : Shape from Shading
1.4 Conventional Lambertian least-squares Photometric Stereo
1.5 Applications of Photometric Stereo
Session 2 : Photometric Stereo meets Deep Learning 22nd July 2023, 11:30 - 12:50 PM
2.1 Benchmark datasets and evaluation
2.2 Per-pixel vs all-pixel vs Hybrid methods
2.3 Calibrated and Uncalibrated Photometric Stereo
2.4 Supervised and Self-supervised Photometric Stereo
2.5 Lighting Calibration for Weakly Calibrated Photometric Stereo
2.6 Relighting for Photometric Stereo
2.7 Multi-supervised Photometric Stereo
Session 3 : Conclusion, Open Challenges, and Question & Answers 22nd July 2023, 12:50 - 13:00 PM
The material covering the tutorial will be made available after NCVPRIG
Speakers
He is currently a PhD student in Electrical Engineering at IIT Gandhinagar. He obtained his MTech degree in Electrical Engineering from IIT Gandhinagar in 2020. He is currently the recipient of the Prime Minister's Research Fellowship. His research interest lies at the intersection of Computer Vision, Computer Graphics, and Deep Learning with primary focus on inferring 3D world from image(s) through photometric methods such as Photometric Stereo, Shape from Polarization, and Photo-polarimetric Stereo. Recently, he has also won the Qualcomm Innovation Fellowship (QIF) 2023 for his project proposal on Photometric Stereo for Refractive Objects. He has been a part of the core organizing team in ICVGIP 2022 held at IIT Gandhinagar in Dec 2022. He has delivered a plenty of invited talks on his research on Photometric Stereo at the venues such as Vision India 2022 during the ICVGIP conference, ACM ARCS 2023, Smart Manufacturing Research Conclave (SMRC) 2023, and the PMRF National Symposium 2023, to name a few. He has published his research across different venues such as ECCV, WACV, ICASSP, and ICPR and has served as reviewer for BMVC, ICCV, WACV, and NeurIPS. He was also the part of the Google Research Week 2023 with Google Research India owing to his contributions in Computer Vision research.
He is currently serving as an Associate Professor in Electrical Engineering jointly with Computer Science and Engineering at IIT Gandhinagar. He obtained his MTech (Biomedical Engineering) and PhD (Electrical Engineering) degrees from IIT Bombay. His research interests include Computer Vision, Deep Learning, and Computer Graphics. Shanmuga was a recipient of the Microsoft Research India PhD Fellowship for the year 2007. He obtained an Excellence in PhD Thesis Work Award at IIT Bombay in 2012. He obtained the Excellence in Research Work Award at IIT Gandhinagar in the year 2022. He was awarded the Jibaben Patel Chair in Artificial Intelligence at IIT Gandhinagar in the year 2020. He has published more than 100 peer-reviewed articles in journals and conferences of repute in the areas of Computer Vision and Signal Processing. He is a recipient of several sponsored research projects including SERB Core Research Grant, SERB IMPRINT-2, ISRO RESPOND, and SERB MATRICS and has worked with several industries. He was the organizing Chair and a Program Chair of ICVGIP 2022 held at IIT Gandhinagar.