Computer Vision Systems

CAP 6411

Fall 2018, Tuesday and Thursday 3:00 pm -4:15 pm, BA rm 207

Instructor: Dr. Abhijit Mahalanobis, email:

Office: HEC 244.

Course Description

The purpose of this course is to discuss how computer vision is used in different applications, and enable students to design their own computer vision systems. The course will build on foundational knowledge of computer vision algorithms, and focus on requirements analysis for vision systems, algorithm and processing architectures, and interfaces for automated systems as well as for human operators. Examples of applications include (but are not limited to) event/activity recognition for surveillance and security, biometric systems that require face and fingerprint recognition, and automated target detection and recognition systems. Students are required to present and lead discussions on current research papers on selected topics, and provide a critical review of the papers presented in class including their strengths and weaknesses.

Text Book – There is no required textbook for this course. Lecture notes will be handed out in class as needed.

A suggested reference book is Handbook of Machine and Computer Vision: The Guide for Developers and Users, Second Edition, Wiley.

Term Project

Students will be required to work (individually or in a group) on a term project that will defined early in the semester. The project must emphasize the system level requirements, architecture design, algorithm strategy, software modules, output definitions and display, and interface to automated system and/or users. Students can select one of the application topics discussed in the class, or propose their own application for the project. The project must be well defined so that it can be completed during the semester, make use of state of the art algorithms, and demonstrate all aspects of an end-to-end computer vision system.


Class Participation – 15%

Summaries of papers presented in class – 15%

Research Paper Presentation and Discussions – 30%

Term Project – 40%

Course Outline





Overview of Systems, Requirements and Specifications



Important factors in ensuring favorable lighting conditions


Cameras Basics

Discuss various Imaging Sensors, calibration, data acquisition


Special Lecture 1

Infra-red imaging sensors


System Architecture 1

Single Platform Systems, Paper Presentation 1.


System Architecture 2

Distributed Systems, Paper Presentation 2.


Algorithm Architectures 1

Deep Learning Algorithms for Images, Paper Presentation 3


Special Lecture 2

Computational Imaging Systems for Object Detection


Algorithm Architecture 2

Deep Learn Algorithms for video, Paper Presentation 4



Processor Considerations, Paper Presentation 5


Case Study 1

ATR systems 1, Paper presentation 6


Case Study 2,

ATR systems 2, Paper presentation 7


Case Study 3

surveillance system, Paper Presentation 8


Case Study 4

Biometrics, Paper Presentation 9


Final Projects Presentation1


Final Project Presentations2

Project Description:

  • Project: Face Detection and Recognition System

In this project, you will develop a face detection and recognition system for identity verification purposes. You are allowed to use any existing data base (if needed) and face detection/recognition software. Project can be done individually or in teams of at most 2 people. Before starting work on the project, you should submit a brief project plan (no more than 2-3 pages).

Here are the guidelines and key steps to follow-

◦ Describe your system. What is the field of view (FOV) of the camera? What are the estimated optical resolution and the iFOV ? What are the image size and frame rate of the camera. Describe your algorithm architecture, training approach, and test methodology. Based on your algorithm complexity, what is the processing time and the output frame rate?

◦ The minimum output frame rate will be 1Hz, but the goal is 10Hz. What architectural and algorithmic considerations did you need to make to meet the processing requirements?

◦ Characterize performance in terms of timing, stability (i.e. how sensitive is it to nuisance parameters), and accuracy.

◦Enroll at least three people, including yourself. For each person, a maximum of three images can be acquired for this purpose.

◦Verify the identities of persons who are enrolled. Demonstrate that the system rejects unknown persons.

◦Design and operate a minimal GUI interface for accepting user commands and displaying images (or videos) with annotation

◦ Develop user guidelines for necessary lighting conditions, minimum and maximum distances, and head tilt for your system to work

◦Is your system robust enough to be operated by others during class presentation?


Your system will be evaluated by you and your classmates. The performance criteria is the number of times the system correctly verifies an enrolled use and rejects an imposter. Perform 10 trials and accumulate the statistics. A minimum of 90% correct verification, and no more than 10% imposter acceptance rate is required. Recall that the minimum output frame rate is 1Hz with a goal of 10Hz.

In addition to the verification and rejection score, the system that achieves the performance metrics with the fewest number of resolved pixels on the subjects face wins! You can use any camera, and use any additional lighting at your disposal.