CPSC 8810: Machine Learning-based Image Synthesis
(2023 Fall)
(2023 Fall)
Location: Olin Hall 203
Time: MW 4:00-5:15 pm
Credits: 3
Instructor: Siyu Huang, Assistant Professor at Clemson University
Email: siyuh@clemson.edu
Office hours: Wed 2:00-3:00 pm
Course overview
This course offers a comprehensive exploration of machine learning techniques for visual data (e.g., images or videos) synthesis and editing. The course will cover a range of topics from classical algorithms (e.g., image filtering) and deep learning methods (e.g., fundamental deep learning models including CNNs and Transformers, as well as deep generative models including VAEs, GANs, and Diffusion Models). This course consists of instructor-led lectures and student-led presentations. Participants will learn to implement image synthesis algorithms, to understand cutting-edge image synthesis techniques, and to explore intriguing research questions. This course will be of particular interest to students seeking to delve into fields of computer vision, machine learning, and AIGC.
Prerequisites
This course requires students to have basic knowledge in machine learning, Python programming, and computer vision (recommended but not necessary).
Assignments
Three homework assignments during the first half of the course
Two paper reviews during the second half of the course
Presenting once in class (as a group of 1 people)
A final project (as a group of 1-2 people).
Grading
Three homework assignments (30% = 10%*3)
VAE
DCGAN
Diffusion-based model
Two paper reviews (10% = 5%*2)
Paper presentation (20%)
Final project (30%)
Class participation (10%)
*Students will be allowed a total of five late days per course. Each additional late day will incur a 10% penalty.
Note: Assignment out, Deadline
Date Lecture Presenter
8/23 Introduction [pdf] Siyu Huang
8/28 Image filtering [pdf] Siyu Huang
8/30 Image transformations [pdf] Siyu Huang
9/4 Labor Day Holiday -
9/6 Deep leraning fundamentals [pdf] Siyu Huang
Paper presentation/review list out
9/11 Pytorch tutorial, Palmetto [pdf] Siyu Huang
Assignment #0 out - The usage of Palmetto
9/13 Deep generative models, Autoregressive models [pdf] Siyu Huang
Paper presentation topics submitted
9/18 Variational Autoencoder [pdf] Siyu Huang
Assignment #1 out - VAE
Paper presentation schedule confirmed by lecturer
9/20 Generative Adversarial Networks (Part I) [pdf] Siyu Huang
Assignment #2 out - GANs
9/25 Generative Adversarial Networks (Part II) [pdf] Siyu Huang
9/27 Student Lectures (Cutting-Edge GANs) [pdf] Ashish Bastola (Student)
10/2 Diffusion-based models (Part I) [pdf] Siyu Huang
Assignment #3 out - Diffusion model
10/4 Diffusion-based models (Part II) [pdf] Siyu Huang
Assignment #1 due - VAE
10/9 Diffusion-based models - applications (Part III) [pdf] Siyu Huang
10/11 Student Lectures (Cutting-Edge Diffusion Models) [pdf] Prasanna Gupta (Student)
Assignment #2 due - GANs
10/16 Fall break
10/18 Image to Image Translation [pdf] Siyu Huang
Assignment #3 due - Diffusion model
10/23 Student Lectures (Text-to-Image Synthesis) [pdf] Rayid Mohammed (Student)
10/25 Unpaired Image Translation [pdf] Siyu Huang
10/30 Student Lectures (Video Synthesis) [pdf] Swapnil Srivastava (Student)
Final project proposal due
11/1 Student Lectures (Image Editing) [pdf] Utkal Sirikonda (Student)
11/6 Student Lectures (Image Restoration) [pdf] Joshua Lumpkin (Student)
11/8 Guest Lecture [pdf] Jie An (Guest)
11/13 Student Lectures (3D-aware Synthesis-1) [pdf] Dehao Qin (Student)
11/15 Student Lectures (Data Augmentation) [pdf] Michael Harris (Student)
Two paper reviews due
11/20 Student Lectures (Interpretable Generative Models) [pdf] Hao Wang (Student)
11/22 Thanksgiving Holidays
11/27 Final project preparation Siyu Huang
11/29 Student Lectures (3D-aware Synthesis-2) Joy Yang (Student)
12/4 Final project presentation: Ashish Bastola, Prasanna Gupta, Rayid Mohammed, Swapnil Srivastava, Utkal Sirikonda
12/6 Final project presentation: Joshua Lumpkin, Dehao Qin and Joy Yang, Michael Harris, Hao Wang
12/8 Final project paper due
12/18 Grades delivered