Class time: Wednesday, 11:30 AM - 1:35 PM

Classroom: Baker Systems 130

Course website: https://sites.google.com/view/osu-cse-5539-au23-chao 

Instructor: Prof. Wei-Lun (Harry) Chao  https://sites.google.com/view/wei-lun-harry-chao 

Email: chao.209@osu.edu 

Office hour: Wednesday, 2:30 PM - 3:30 PM, in person (DL 587)

Course abstract:  This research-driven course will cover various advanced machine learning (ML) and computer vision (CV) topics and their applications to, for example, perception for autonomous driving. Specifically, we will study different neural network models, different training strategies (e.g., supervised, unsupervised, and semi-supervised learning), fundamental CV tasks (e.g., image classification, object detection, and semantic/instance segmentation), and generative models (e.g., VAE, GAN, and diffusion models). We will also study practical aspects, such as domain adaptation, imbalanced learning, etc. We will also study data, dataset design, experimental design, evaluation, and analysis. The format of the class will be a mix of lectures and research paper presentations. Students who participate in this class are expected to be self-motivated graduate students or senior undergraduate students.

Pre-requisites:  

• Students are expected to have a decent degree of mathematical sophistication and to be familiar with linear algebra (Math 2568), multivariate calculus, probability, and statistics. Students are also expected to have knowledge of algorithm design and data structures. Students are expected to be able to code in Python.

• Students are expected to (self) learn deep learning software (e.g., Tensorflow and Pytorch). 

• Students are expected to have a strong interest in machine learning, computer vision, and their applications in autonomous driving

• Students are expected to have taken courses in artificial intelligence/machine learning (3521/6521, 5523, or 5526) and/or computer vision (5524)

• Review materials can be found: linear algebra, probability, Python-1, Python-2, Python-3 

Syllabus: Click

Grading (tentative):

• Participation: 5%

Participation includes attendance, asking questions, discussion in the lectures

• Paper presentation & survey (1-2 people): 40% (25%: presentation; 15%: survey)

The presentation needs to be in a job-talk/lecture style. It is graded based on 1) the quality of the papers you pick and 2) efforts and clearness in presenting the ideas of the papers.
The survey report is graded based on efforts, clearness, and how well you organize the topics/papers that you read. The report should include sections like introduction, background, approach, experiments, etc.

• Final project (1-2 people): 55% (10%: proposal; 20%: final presentation; 25%: report & results)

The proposal includes: 5% oral presentation, 5% written report (2 pages)

Required Textbook: No required textbook 

Suggested references: 

• Kevin P. Murphy. Machine Learning: A Probabilistic Perspective. The MIT Press, 2012

• Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. The MIT Press, 2016

• Joel Janai, Fatma G¨uney, Aseem Behl, Andreas Geiger. Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Art. Foundations and Trends in Computer Graphics and Vision, 2020

• Claudine Badue et al. Self-driving cars: A survey. Expert Systems With Applications, 2021

• Tutorials & workshop talks in CVPR, ICCV, ECCV, ICML, ICLR, and NeurIPS 

Useful reference:

• Kaare Brandt Petersen and Michael Syskind Pedersen. The Matrix Cookbook

Announcements and communications: 

• We will make announcements using the Carmen Canvas. Announcements of urgent matters will be mailed to your name.#@osu.edu address. If you do not regularly check those accounts, make sure you forward them to somewhere that you do.

• We will use Piazza for discussions. If you have questions about the course materials or policy, please also post them on these platforms. I will also monitor these discussions and answer as appropriate, but students should feel free to use the forums to have group discussions as well.

• Please only use email to contact the instructor for urgent or personal issues. Any e-mails sent to the instructor should include the tag "[OSU-CSE-5539]" in the subject line. (This ensures we can filter and prioritize your messages.) We reserve the right to forward any questions (and their answers) to the entire class if they should prove relevant. Please indicate if you wish to be anonymized (i.e. have your name removed) in this case. 

Survey report:

• NeurIPS 2023 Latex style: https://neurips.cc/Conferences/2023/PaperInformation/StyleFiles 

• Extend your paper presentation from 2-3 papers to >10 papers

• At least 10 pages, excluding reference (Introduction, an overview of the background, descriptions of some key algorithms and their concepts, important experimental results and findings, potential further improvements)

• Due day: Outline due 1 week after your presentation; the final version is due on 12/6/2023. Late reports will lead to an immediate 30% deduction on your report scores. If your report is late by a week, the deduction will become 60%. If your report is late by two weeks, you will get 0 points.

• The survey report is graded based on efforts, clearness, and how well you organize the topic/papers that you read. The report should include sections like introduction, background, approach, experiments, etc.

• Example:

https://papers.nips.cc/paper/8420-failing-loudly-an-empirical-study-of-methods-for-detecting-dataset-shift.pdf 

https://link.springer.com/article/10.1007/s10994-019-05855-6 

Final project (Competition & challenges or research project: 1-2 people):

• Competition & challenges: A team (or the whole class) selects a competition & challenge held in top conferences and participates in it. 

• Research project: A team (or the whole class) selects a research topic. The research topic MUST be relevant to the topics studied in this course. The expectation is to be ready to submit to a conference (e.g., ICML 2024 or ECCV 2024).

• Either way, the final report is expected to be submittable to arXiv (with high quality).

Example challenges:

• Learning from Imperfect Data (LID)  

• https://sites.google.com/view/learning-with-limited-data 

• Other suggestions from NeurIPS, ICML, ICLR, ICCV, ECCV, CVPR workshops 

 Week 1: PDF, PPT

Week 2: PDF, PPT

Week 3: PDF, PPT

Week 4: PDF, PPT

Week 5: PDF, PPT

Week 6: 3D detection (PDF, PPT), Domain adaptation (PDF, PPT)   

Week 7: Domain adaptation (PDF, PPT), Class-imbalance learning (PDF, PPT)

Week 8: Imbalanced learning (PDF-1, PPT-1) (PDF-2, PPT-2), Self-supervised learning (PDF, PPT)

Week 9 (class is online via Zoom): Self-supervised learning (PDF, PPT), Federated learning (PDF, PPT)

Week 10: Federated learning (PDF, PPT), Continual learning (PDF, PPT)

Week 11: Continual learning (PDF-1, PPT-1) (PDF-2, PPT-2), Efficient learning & inference (PDF, PPT)

Week 12: Efficient learning & inference (PDF, PPT), Generative models (PDF, PPT)

Week 13: Generative models (PDF)

Week 14: NeRF (PDF, PPT)