Class time: Wednesday, 1PM - 2:45PM

Classroom: Dulles Hall 020

Course website: https://sites.google.com/view/osu-cse-5539-sp23-zhu

Course Description

Course abstract: In the past decade, deep learning has demonstrated unprecedented performance across many different domains in engineering and science, ranging from traditional AI fields such as computer vision, natural language processing, and gaming, to health care, finance, and so on. However, we still have a limited understanding of this success and deep neural networks (DNNs) are often designed by trial and error and then trained and used as black boxes. This course will introduce the basic ingredients of DNNs, overview recent work on the models and theory of deep learning, sample important applications, and discuss some open questions. The format of the class will be a mix of lectures and research paper presentations. Students who participate in this class are expected to be self-motivated graduate or senior undergraduate students.

Students are expected to have a background in linear algebra, multivariate calculus, probability, statistics, and python. Students are also expected to have taken courses in artificial intelligence/machine learning (3521/6521, 5523, or 5526).

Course credits: 2 units

Pre-requisites:

• Required background:

§ Linear algebra: Math 2568, 2174, 4568, or 5520H

§ Artificial intelligence: 3521, 5521, or 5243

§ Statistics and probability: 5522, Stat 3460, or 3470

§ Machine learning: 5523 or Neural Networks: 5526

• Students in the class are expected to have a decent degree of mathematical sophistication and to be familiar with linear algebra, multivariate calculus, probability, and statistics. Students are also expected to have knowledge of programming, algorithm design, and data structures.

• Programming: students are expected to know or self-learn deep learning software (e.g., Tensorflow and Pytorch).

• Review materials can be found: linear algebra, probability, Python-1, Python-2, Python-3

Textbook (optional)

• Kevin P. Murphy, Machine Learning: A Probabilistic Perspective. The MIT press, 2012. (Free online version, Amazon)

• Hastie, Tibshirani, and Friedman, The Elements of Statistical Learning, 2009. (Free online version, Amazon)

• Hsuan-Tien Lin, Malik Magdon-Ismail, and Yaser Abu-Mostafa, Learning From Data, AMLBook, 2012. (Amazon)

• Ian Goodfellow, Yoshua Bengio, and Aaron Courville, Deep learning. The MIT press, 2016. (Amazon)

Useful reference:

• Kaare Brandt Petersen and Michael Syskind Pedersen, The Matrix Cookbook

Grading (tentative):

• Participation: 10%

Includes attendance, asking questions, discussion

• Paper presentation (1-2 people): 35%

It will be graded based on efforts and clearness in presenting the ideas of the papers. See the syllabus for detailed rubrics.

• Final project (1-2 people): 55%

Proposal (10%): 5% presentation, 5% report

Final (45%): 22.5% presentation, 22.5% report

Announcements, communications, and discussions:

• We will make normal announcements using the Carmen Canvas.

• We will use Piazza for discussions. If you have questions about the course materials or policy, please post them on Piazza. The TA and I will also monitor these discussions and answer as appropriate, but students should be active and feel free to use the forums to have group discussions as well.

• Please only use email to contact the instructor for urgent or personal issues. Any e-mails sent to the instructor should include the tag "[OSU-CSE-5539]" in the subject line. (This ensures we can filter and prioritize your messages.) We reserve the right to forward any questions (and their answers) to the entire class if they should prove relevant. Please indicate if you wish to be anonymized (i.e. have your name removed) in this case.

Policy (see the syllabus for details)

Paper presentation (1-2 people):

• Pick at least 2 papers from one research topic it is best for each group.

Final project (comprehensive survey with experiments, individual project, or challenge: 1-2 people):

• Comprehensive topic studies: A team selects a topic and performs a comprehensive survey of the techniques, datasets, and evaluation metrics. Then, the team is going to re-implement those techniques and re-evaluate on all the datasets with all the metrics. Students are encouraged to propose new techniques and experimental setups.

• Competition & challenges: A team selects a competition & challenge held in top conferences and participates in it.

• Self-picked research topic: A team selects a research topic. The expectation is to be ready to submit to a conference (e.g., NeurIPS 2023).

Example topics for paper presentation (keep updating)

Schedule (tentative)