Computer Vision

Lecture slides and readings

Course introduction and overview

• Date: Sept 4
• Lecture 1
• Lecture slides: keynote, pdf
• Readings
  • Richard Szeliski book, Chapter 1 (RS 1)
  • The speed of processing in the human visual system, Thorpe et al., Letters to Nature, 1996

Radiometry

• Date: Sept 6
• Lecture 2
• Lecture slides: keynote, pdf
• Readings
  • RS 2
  • Surface reflectance estimation and natural illumination statistics, R.O. Dror, E.H. Adelson, and A.S. Willsky, Workshop on Statistical and Computational Theories of Vision 2001
  • Chapter 3 of Forsyth and Ponce on shape from shading.
  • Wikipedia: Photometric stereo, BRDF

Light and color

• Date: Sept 13, 18
• Lecture 3, 4
• Lecture slides: keynote, pdf
• Readings and resources
  • RS 2
  • Color matching applet from Stanford
  • B. Berlin and P. Kay, Basic Color Terms: Their Universality and Evolution (1969)
  • D.A. Forsyth, A novel algorithm for color constancy
  • Wikipedia: Trichromacy, Color constancy
  • https://www.ted.com/talks/beau_lotto_optical_illusions_show_how_we_see?language=en
  • https://en.wikipedia.org/wiki/Sergey_Prokudin-Gorsky

Image formation

• Date: Sept 20, 23
• Lecture 5, 6
• Lecture slides: keynote, pdf
  • Note the last part of the lecture slides were covered in the next lecture.
• Readings
  • RS 2, 3
  • https://en.wikipedia.org/wiki/Camera
  • https://en.wikipedia.org/wiki/History_of_the_camera
  • Light stages: http://gl.ict.usc.edu/LightStages

Modeling natural images

• Date: Sept 27
• Lecture 7
• Lecture slides: keynote, pdf
  • Note the last part of the lecture slides will be covered in the next lecture.
• Readings
  • RS 3

Image filtering

• Date: Oct 2, 4
• Lecture 8, 9
• Lecture slides: keynote, pdf
• Readings
  • Hybrid image gallery: http://cvcl.mit.edu/hybrid_gallery/gallery.html
  • RS 3

Scale-invariant features

• Date: Oct 4, 11
  • Note: no class on Oct 9 since it is a Monday schedule.
• Lecture 9, 10
• Lecture slides: keynote, pdf
• Readings
  • David Lowe's SIFT page: https://www.cs.ubc.ca/~lowe/keypoints
  • Lindberg's scale-space theory: https://www.tandfonline.com/doi/abs/10.1080/757582976
  • Other SIFT implementations: VLFeat, OpenCV
  • RS 4

Alignment and model fitting

• Date: Oct 16, 18
• Lecture 11, 12
• Lecture slides: keynote, pdf
• Readings
  • RS 6 and 9. Chapter 9 goes into much more detail on how images can be stitched together to form panoramas.
  • http://www.robots.ox.ac.uk/~vgg/research/affine/index.html

Applications of feature matching and alignment

• Date: Oct 23, 25
• Lecture 13, 14 (wrap up)
• Lecture slides: keynote, pdf
• Readings and references:
  • Photo tourism
  • Photogrammetry
  • Oxford building search
  • David Nister's vocabulary tree for instance search [paper]

Optical flow

• Date: Oct 25
• Lecture 14
• Lecture slides: keynote, pdf
• Readings and references
  • Lucas Kanade optical flow
  • Motion magnification:
    • http://people.csail.mit.edu/celiu/motionmag/motionmag.html (covered in class)
    • http://people.csail.mit.edu/mrub/vidmag/

Recognition overview

• Date: Oct 30, Nov 1, Nov 6
• Lecture 15, 16
• Lecture slides: keynote, pdf
• Readings and references:
  • https://en.wikipedia.org/wiki/Bag-of-words_model_in_computer_vision
  • https://en.wikipedia.org/wiki/Histogram_of_oriented_gradients

Linear models

• Date: Nov 8
• Lecture 17
• Lecture slides: keynote, pdf
• Readings and references:
  • https://en.wikipedia.org/wiki/Perceptron
  • http://vision.stanford.edu/teaching/cs231n-demos/linear-classify/

Neural networks

• Date: Nov 13, 15, 27
• Lecture 18, 19, 20
• Lecture slides: keynote, pdf
  • https://cs.nyu.edu/~fergus/papers/zeilerECCV2014.pdf
  • http://yosinski.com/deepvis
  • http://vis-www.cs.umass.edu/texture/
  • https://www.robots.ox.ac.uk/~vedaldi/assets/pubs/mahendran16visualizing.pdf

Learning and transfer with neural networks

• Date: Nov 29, Dec 4
• Lecture 21, 22
• Lecture slides: keynote, pdf
  • http://vis-www.cs.umass.edu/bcnn/
  • CNN Features off-the-shelf: an Astounding Baseline for Recognition

Graphics and vision

• Date: Dec 6
• Lecture 23
• Lecture slides: keynote, pdf
  • http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture12.pdf
  • http://www.robots.ox.ac.uk/~vedaldi//research/visualization/visualization.html
  • http://vis-www.cs.umass.edu/texture/
  • https://dmitryulyanov.github.io/deep_image_prior
  • Generative adversarial networks (Goodfellow et al. paper)
  • "Can computers create art?" by Aaron Hertzmann https://arxiv.org/abs/1801.04486

Adversarial attacks against ML systems

• Date: Dec 11
• Lecture 24
• Lecture slides: keynote, pdf
  • https://www.kaggle.com/c/nips-2017-defense-against-adversarial-attack