Image and Video Processing

What is image and video processing? Images and videos are everywhere, from those we take with our mobile devices and share with our friends to those that we receive from Mars and the ones we see in the movie theatre, without forgetting the whole ensemble of images of our bodies that are taken in hospital visits. Image and video processing is the art of working with such images and movies, from making it possible to store and transmit them to making those dark and blurry images look nice, as well as interpreting and analyzing the medical data and recognizing our friends’ faces in social pictures. This discipline is also fascinating because it uses tools from many areas of applied mathematics. In this class you will look behind the scenes of image and video processing, from the basic and classical tools to the most modern and advanced algorithms.

The course will start with an introduction to the basics of image formation and the fundamental concepts that translate a physical scene into a digital image. We will then describe the underlying concepts of image compression, the enabling technology that makes it possible for images to be sent from Mars and videos to be stored in our mobile phones. We will cover the most fundamental tools in image enhancement, showing how simple tools can significantly improve images. Both geometric and non-geometric tools as well as spatial and non-spatial operations will be presented. Details on image segmentation will be provided, one of the most fundamental and useful problems in image processing.

The above topics will be extended to color images and video. Once we have covered the fundamentals, which both provide the basis for modern image and video processing and serve many important applications until today, we will move into recent progress in image and video compression.

Image and video analysis can be approached from numerous areas of mathematics, from linear algebra to geometry, optimization, and differential equations. We plan to make all the lectures as self-contained as possible, but basic background in linear algebra and digital signal processing will be helpful. The background is covered in the course "Mathematical and formal fundamentals for computer sciences".

20h lecturer, 30h exercise, 15h homework

The final grade will be determined based on regular homeworks, one midterm exam, and a Semester Project:

Homeworks: 20%

Midterm Exam: 30%

Semester Project: 50%

Homework Policy: You may use any material you want to solve homework problems. Teaming up with other students to work out homework problems is acceptable. However, you will maximize your learning experience by working out homework problems on your own. In any event, you are required to write out your solutions independently.

The lectures will follow, in part, "Digital Image Processing, 3rd edition" by Gonzalez and Woods. The more advanced material will be based on material the instructor will make available. Some interesting books for the advanced material include:

• Dao Nam Anh, "Phân tích và xử lý ảnh", Image Processing and Analysis, NXB BKHN, Code: 569–2015/CXBIPH/07–18/BKHN, ISBN 978-604-938-525-4, 2015.
• Gonzalez, Woods, Eddins. Digital Image Processing Using MATLAB. Prentice Hall, 2004
• Bahadir K. Gunturk. Image Analysis lectures, LSU, 2006
• Christopher M. Bishop. Pattern Recognition and Machine Learning. Springer, New York, 2006
• Forsyth/Ponce, Computer Vision: A Modern Approach, Prentice-Hall.
• Chris Solomon, Toby Breckon. Fundamentals of Digital Image Processing: A Practical Approach with Examples in Matlab, John Wiley & Sons, 2011
• Guillermo Sapiro, "Image and video processing: From Mars to Hollywood with a stop at the hospital"

Research Labs

• Computer Graphics Group at Stanford University
• Computer Graphics Lab at MIT
• Computer Graphics Research at Caltech
• Computer Graphics Group at UC Berkeley
• Computer Graphics Group at Cornell University
• Graphics and Imaging Lab at University of Washington
• Computer Graphics Group at Brown University
• Computer Graphics Lab at Carnegie Mellon
• Graphics Group at Princeton
• Computer Graphics and Visualization Group at Utah
• Computer Graphics at Microsoft Research MSRA
• MIRALab

Major IP and CV Journals

• Siggraph
• Eurographics
• ACM Transactions on Graphics
• IEEE Transactions on Visualization and Computer Graphics
  • IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI)
  • Computer Vision and Image Understanding (CVIU)
  • International Journal of Computer Vision (IJCV)
  • Image and Vision Computing (IVC)
  • IEEE Transactions on Image Processing (TIP)
  • Pattern Recognition (PR)
  • Machine Vision and Applications (MVA)

Major IP and CV Conferences

• IEEE International Conference on Computer Vision (ICCV)
• IEEE International Conference of Image Processing (ICIP)
• IEEE Computer Vision and Pattern Recognition (CVPR)
• International Conference of Pattern Recognition (ICPR)

Codes

Here there are links to useful code:

• The Steerable Pyramid
• Representation and Synthesis of Visual Texture, Portilla & Simoncelli
• Berkeley Segmentation
• Pb
• Superpixels
• Structure from Motion for Unordered Image Collections
• Peter Kovesi's Functions for Computer Vision
• SIFT implementation by Andrea Vedaldi
• Affine Covariant Features
• A simple object detector with boosting

Other useful code:

• Code for downloading Flickr images
  • Matlab Image Processing Toolbox on-line documentations
  • Matlab image processing toolbox tutorial
• A tutorial on sampling theorem at Wikipedia
  • A tutorial on C.I.E. Chromaticity Digram
  • Data-compression.com, a web site with useful information about data compression.
• Arithmetic coding resources
• Run-Length coding resources
• JPEG and MPEG image and video coding resources
• Resources for Morphological Image Processing
  • Dital image processing tutorials and interactive applets: a collection of informative and interactive Java applets explaining basic digital image processing concepts.
  • Computational geometric and graphics codes
  • Code and free software maintained by Frédo.
  • Computational Geometry Algorithm Library (CGAL)
  • GNU Triangulated Surface Library (GTS)
  • Computer Vision Software
  • OpenGL at dmoz
  • Graphics Gems
  • MIT Software Resources
• Open Source Repository
• Graphics Source Codes
• OpenGL and OpenInventor

Useful Mathematics, Statistics, and Geometry resources

• Math Cheat Sheet (lots of useful formulas)
• Numerical Recipes in C
• The Geometry Center Home Page
• Geometry in Action
  • Computational Geometry Pages
  • Applied Statistics

Formats and Viewers

• Image Formats and Viewers
• Image formats
• Xv is an interactive image display program for the X window system that is useful for displaying and editing images in a variety of formats.
• Irfanview (freeware graphic viewer for Windows)
• GISLook (PGM support on Mac)

Software

• CVIPtools: a GUI-based computer vision and image processing tools, ANSI-C source code and librariesfor Windows95/NT and UNIX, extended computer imaging TCL shell. Also contains an extended Tcl shell with all the computer imaging functions. ANSI-C source code and libraries for image analysis, image compression, image enhancement, image restoration, and many imaging utilities.
• Intel Computer Vision Library (OpenCV): image processing and computer vision algorithms optimized to run on Intel microprocessors.
• KHOROS
• Matlab: a numeric computation and visualization environment. The image processing and signal processing toolboxes are especially useful. See also: Matlib Tutorial (Univ Utah), Matlab Basics (RPI), Matlab Primer (200K postscript; 25 pages).
• More software .... (Good stuff !!)

Debugging

• Debuggers (presentation slides)
• ddd: GNU debugger

Other

• FAQ Image Processing
• Glossary
  • A Survey of Compressed Domain Processing Techniques
  • ResearchIndex (a scientific literature digital library - find papers easy!)