EE6130 - Computational Imaging & Artificial Intelligence
Based 3D Displays
Jan-May 2022
Instructor: Dr. Mansi Sharma
Lectures: Mondays Tuesdays Fridays, Slot C, Online Mode
Problem Session: Fridays, Online Mode
Embedded Files
Course Description
New 3D display technology combines computational imaging, visual neuroscience, optics, visual computing, machine learning, multidimensional signal and data processing, haptic feedback, and sound for true immersion and interaction. This significantly broadens the field of applications ranging from healthcare, defence, industrial engineering, entertainment, vocational and educational domains.
Current VR/AR/MR/3D displays fall far short of truly recreating visual reality. This can never be achieved by portraying the real world on a flat surface or screen, but require full-parallax displays which can recreate the complete light fields traveling in every direction through every point in space. This course aims at providing a thorough insight and hands-on experience in understanding the working principles, theoretical background, technical aspects and challenges of full-parallax light field imaging techniques for Glasses-free 3D Displays and Superstereoscopic VR/AR/MR Head Mounted Displays. In addition, we will incorporate special topics on Holography for Near-Eye Displays, Visuo-Haptic Interaction with Mixed Reality or Autostereoscopic 3D Displays, Volumetric Haptic Display, Holo-Haptics See-Through 3D Display, and Free-viewpoint 6DoF VR video applications. This new type of media expands the user experience beyond what is offered by traditional 2D media, that is, reconstruct both 'Depth-of-field' and 'Field-of-view' for displays and offers haptics feedback with full-parallax 3D. In this offering, we focus on 3D vision technology that employed computational imaging and artificial intelligence based algorithms to reconstruct light fields on variety of display types.
Course is open for final-year B.Tech., Dual Degree, M.Tech., M.S., Ph.D. students. Nonetheless, the course is open to project staffs, startup personal, postdoc researchers as well. It will be useful for EE, ECE, CSE, MATHS, ME, Engineering Design, Aerospace, Biomedical, etc. background students in their research and career.
Helpful Background
We encourage students from diverse background to attend this course. Instructor's consent will be primarily required. There is no specific prerequisite. Students can register after discussion with me.
The basic knowledge of Linear Algebra, Calculus, Optimization, Computer Vision, Image Processing, Machine Learning and/or Computer Graphics would be very useful.
Computer Vision lecture from past courses, particularly the Projective Geometry, Camera Models, N-view Geometry. If students have not attended that class, there will be a concise repetition. The course material will be provided to study the necessary basics in order to work for the project and understanding basic papers. Previous knowledge of Computer Vision, Image Processing and Machine Learning will be supportive for projects, but not crucial. Students will be given full flexibility to define their own ideas/projects according to their background and experience.
Courses that will be helpful, but are not prerequisites:
CS6350 Computer Vision
CS6360 Computer Graphics
EE5175 Image Signal Processing
Related online courses useful for creating background for the study:
1. Multiple View Geometry - YouTube Lecture Series by Prof. Daniel Cremers (Technical University of Munich)
Multiple View Geometry - Lecture Slides by Prof. Daniel Cremers
2. Computer Vision Lectures - Prof. Marc Pollefeys (ETH Zurich)
3. Photogrammetry Lectures
Prof. Cyrill Stachniss (University of Bonn)
4. Fundamentals of Computer Graphics - Prof. Ravi Ramamoorthi
YouTube Video Available - BerkeleyX: CS184.1x Foundations of Computer Graphics
Note: This course is absolutely dedicated to Light Field Imaging & Display Technologies. A few of the course topics may overlap with other related courses.
Grading Policy
Your final grade will be made up from
(1) Three assignments: 40% total
(2) Midterm exam: 20%
(3) End semester exam: 20% (viva voce exam)
(4) Term project: 20%
This includes a project proposal, conference paper style report, detail technical report, source code/demo videos/poster presentation.
Textbooks and Other Relevant Reading Materials
References of some important books can be found here on 3D Displays, Computational Imaging & Light fields, and VR/AR/MR
In addition, students are expected to read research papers/technical literature as discussed in class. Course/project relevant material will be provided
in class. Check out material outlined in the "additional readings" section of the syllabus.
Important Class Information & Announcements
TA contacts:
(1) Joshitha R ee19D401@smail.iitm.ac.in
(2) Sally Khaidem ee20d041@smail.iitm.ac.in
(3) Sumit Sharma ee20d042@smail.iitm.ac.in
(4) Rohit Choudhary ee20s002@smail.iitm.ac.in
Class Assignments/Projects
To Be Announced in Class
Acknowledgements
Some of the materials used in class build on that from other tutors and researchers. In particular, we will use some materials from Nestor Matthews (Denison University), Jennifer M. Groh (Duke Univ.), Byoungho Lee (Seoul National University), Yasuhiro Takaki (TUAT Japan), Masayuki Tanimoto (Nagoya University), Matthew Hirsch, Gordon Wetzstein, Ramesh Raskar (MIT USA), Marc Pollefeys (ETH Zurich), Harlyn Bayer, Jen-Hao Rick Chang, B. V. K. Vijaya Kumar, Aswin C. Sankaranarayanan (CMU, USA), Daniel Cremers (TUM Germany), Daniel Minoli (DVI Communications), Gordon Wetzstein (Stanford University), Phil Surman (De Montfort University/LUMINOUS! Centre of Excellence for Semiconductor Lighting and Displays NTU), Cyrill Stachniss (University of Bonn), Steven M. LaValle (University of Oulu). Feel free to use these slides/materials only for academic or research purposes, but should maintain all acknowledgments.
Page updated
Google Sites
Report abuse