CPSC 8810: Machine Learning-based Image Synthesis
(2024 Fall)
(2024 Fall)
Location: Lehotsky Hall 134
Time: Tuesday/Thursday 3:30-4:45 pm
Credits: 3
Instructor: Siyu Huang, Assistant Professor at Clemson University
Email: siyuh@clemson.edu
Office hours: Tuesday 2:00-3:00 pm
Course overview
This course offers a comprehensive exploration of machine learning techniques for visual data (e.g., images or videos) synthesis and editing. The course will cover a range of topics from classical algorithms (e.g., image filtering) and deep learning methods (e.g., fundamental deep learning models including CNNs and Transformers, as well as deep generative models including VAEs, GANs, and Diffusion Models). This course consists of instructor-led lectures and student-led presentations. Participants will learn to implement image synthesis algorithms, to understand cutting-edge image synthesis techniques, and to explore intriguing research questions. This course will be of particular interest to students seeking to delve into fields of computer vision, machine learning, and AIGC.
Prerequisites
This course requires students to have basic knowledge in machine learning, Python programming, and computer vision (recommended but not necessary).
Assignments
Three homework assignments during the first half of the course
Two paper reviews during the second half of the course
Presenting once in class (as a group of 2 people)
A final project (as a group of 1-2 people).
Grading
Three homework assignments (24% = 8%*3)
VAE
DCGAN
Diffusion-based model
Two paper reviews (10% = 5%*2)
Paper presentation (20%)
Final project (40%)
Class participation (6%)
*Students will be allowed a total of five late days per course. Each additional late day will incur a 10% penalty.
Note: Assignment out, Deadline, Holiday
Date Lecture Presenter
8/22 Introduction [pdf] Siyu Huang
8/27 Image filtering [pdf] Siyu Huang
8/29 Image transformations [pdf] Siyu Huang
9/3 Deep learning fundamentals [pdf] Siyu Huang
9/5 Pytorch tutorial, Palmetto [pdf] Siyu Huang
Assignment #0 out - The usage of Palmetto
9/10 Deep generative models, Autoregressive models [pdf] Siyu Huang
Paper presentation topics submitted
9/12 Variational Autoencoder [pdf] Siyu Huang
Assignment #1 out - VAE
Paper presentation schedule confirmed by lecturer
9/17 Generative Adversarial Networks (Part I) [pdf] Siyu Huang
Assignment #2 out - GANs
9/19 Generative Adversarial Networks (Part II) [pdf] Siyu Huang
Assignment #1 due - VAE
9/24 Diffusion-based models (Part I) [pdf] Siyu Huang
Assignment #3 out - Diffusion model
Assignment #2 due - GANs
9/26 Diffusion-based models (Part II)[pdf] Siyu Huang
10/1 Diffusion-based models - applications (Part III) [pdf] Siyu Huang
10/3 Flow-based models [pdf] Siyu Huang
10/6 Assignment #3 due - Diffusion model
10/8 Final project proposal presentation-I
10/10 Final project proposal presentation-II
10/15 Fall break
10/17 Student Lectures
Cutting-Edge GANs Connor McKiernan
Fairness in Generative Models Yucong Dai, Xusheng Ai
10/22 Student Lectures
Other Generative Models Jingjing Wang
Text-to-Image Synthesis Xiangyu Jiang, Reek Majumder
10/24 Student Lectures
Image to Image Translation Stephen Becker
Image Editing Chaoyi Zhou, Uma Meleti
10/29 Student Lectures
Image Style Transfer Benjamin Formby
Video Synthesis Prakhar Gupta, Mayuresh Bhosale
10/31 Student Lectures
3D-aware Synthesis Xi Liu
2.5D Map In-painting and Geometry Aware Synthesis Vasudev Purohit, Benhamin
Two paper reviews due
11/5 Election Day
11/7 Student Lectures
Image Restoration Abyad Enan
Data Augmentation Angel Thu, Do Chenxi Zhao
11/12 Student Lectures
Face and Pose Modeling Austin Hartley, Matthew Yang
Image Forensics Krishna Panthi
11/14 Student Lectures
Interpretable Generative Models Chris Williams, Luo li
11/19 Student Lectures
Cutting-Edge Diffusion Models Jie ji, Kaiyuan Deng
11/21 Final project presentation-I
11/26 Final project presentation-II
11/28 Thanksgiving Holidays
12/3 Final project presentation-III
12/5 Final project website and paper preparation
12/6 Final project website and paper due
12/15 Grades delivered