Course Description: An introductory course on machine vision and related machine learning used in automation, autopilots, security and inspection systems. Topics covered include theory of computer and machine vision and related algorithms for image capture and processing, filtering, thresholds, edge detection, shape analysis, shape detection, salient object detection, pattern matching, digital image stabilization, stereo ranging, and methods of sensor and information fusion. Machine vision sensors covered include visible to long-wave infrared including passive EO/IR (Electro-Optical/Infrared) as well as active methods such as RGB depth mapping, RADAR and LIDAR. Embedded and automation topics covered include implementation of these algorithms with SoC multi-core and GP-GPU embedded real-time vision systems for autopilots (intelligent transportation and AV/ADAS), general machine vision automation and security including methods for detection, classification, recognition of targets, and applications including inspection, surveillance, search and rescue, and machine vision navigation.  Other topics covered include use of FPGA co-processors (compared to GP-GPU), camera and sensor hardware and interfaces, and multi-modal and multi-spectral sensor fusion.

EXERCISES and ASSIGNMENTS: On CU Canvas - Please consult Canvas for all assignments and DUE dates! - CU Canvas

ESE Academic Honesty Policy - Complete Agreement

Course Prerequisites: Strong C/C++ programming skills, familiarity with operating systems, computer architecture and hardware/software interface and debug skills are required. Advanced skills and knowledge of embedded SoC CPU multi-core and GP-GPUs are helpful, but not required. Before taking ECEE 5763, students should have completed ECEE 5623 (or Coursera ECEA 5615 and 5616), or ECEN 5803 or ECEN 5813. Real-time Machine Vision and Learning concepts will be taught using embedded Linux and students will be expected to work with camera interfaces and OpenCV to build an interactive or autonomous embedded automation application. Some MATLAB examples will be used for machine learning demonstrations and exercises. Students will have an industry understanding of embedded vision systems consistent with practice (e.g. Embedded Vision Alliance) upon completion of this course.  

All students must use an NVIDIA Jetson Nano (2g or 4g or Orin) or R-Pi 4 and Logitech C270 or C615 or check out and borrow from the "E-Store" (The E-Store is located in ECEE 1B10. Hours vary by semester and are posted outside the E-Store door. The phone number is 303-492-7453).

Please flash your SD card with JetPack 4.6.x for the Nano 2g or 4g - (release notes) or Jetson Orin Nano with 5.x or 6.x from JetPack Archive

Due to long-lead times for ordering NVIDIA hardware, you can use R-Pi 4 or Virtual-Box Linux to get started.  See - Linux-Dev, Linux-Help

Lab Description:The course makes use of OpenCV, MATLAB, CUDA or OpenCL as software tools and requires the student to install Ubuntu Linux (ideally native) on their own PC or laptop. If a native install is not possible, Oracle Virtual-Box installation will work, but only with pre-recorded video. For work with live video please note recommended cameras and make sure you have access to native Ubuntu Linux. Note that a Quartus-II installation with an OpenCL license is needed for the FPGA approach using the DE1-SoC and CUDA used with a NVIDIA Jetson TK1 or Nano for the GP-GPU approach for completion of the final scaffolded assignment. 

Remote Lab Note: NVIDIA Jetson Nano - NVIDIA Nano running Embedded Linux. The ESE Program has a limited number of Jetson Nano systems to loan out as well for students in Colorado who can come to campus, so please contact an SA if you would like to check one out for Term-D.