Course Description This is a seminar-style course for graduate students. Topics in the intersection of deep learning and graph algorithms shall be covered, which naturally include two threads -- 1) deep learning for graphs, where we shall see how a flurry of modern deep neural networks have been devised to better the performance of various classical graph tasks, such as node classification, link prediction, graph matching, and so on, and 2) graph for deep learning, where we shall scrutinize why by knowing the inherent topological structures underlying given input data beforehand can lead to a more efficient training of deep learning models. Weekly in-person meetings are expected, unless otherwise be changed per university policies. Facial masks are required inside the classroom per the university policy (link: www.odu.edu/news/2021/8/message_from_preside) due to COVID-19 concerns.
Course Objectives Students completing this course should be able to:
Demonstrate knowledge of recent advances in graph-orientated and graph-based deep neural networks models;
Understand the intuitions behind the corresponding neural architecture and loss function designs;
Be familiar with the common mathematical tools and basic theoretical proof skills;
Push the research frontline, distill the learned knowledge so as to come up with new graph neural network designs.
Instructor: Yi He
Office: E&CS 3108 Email: yihe@cs.odu.edu
Office Hour: 1PM -- 2PM Monday or by appointment Classroom: DRGS 1117 Meeting Time: 7:10PM -- 8:25PM Tuesday
Reading & Research & Write-ups: 7:10PM -- 8:25PM Thursday
Final grade is decomposed by the following weights:
Attendance: 10%
Attendance is required. Each absence incurs a deduction of 1% on attendance until all points are deducted in this aspect. If more than 11 absences are observed, the student automatically gets an F.
**Accommodations**: Exceptions will be made only in situations of unusual and unforeseeable circumstances beyond the student’s control, and such arrangements must be made promptly, prior to the due date in any situations where the conflict is foreseeable. Students are encouraged to self-disclose disabilities that have been verified by the Office of Educational Accessibility by providing Accommodation Letters to their instructors early in the semester in order to start receiving accommodations. Accommodations will not be made until the Accommodation Letters are provided to instructors each semester.
Paper Presentation: 35% = 20% (peer grade) + 15% (instructor grade)
Each student is expected to make one paper presentation. The paper should be selected from the literatures listed below. Please email me within the first two weeks your preferred presentation date and the selected paper, avoiding time conflict and overlapped selection with other students. All students other than the presenter shall be asked to grade the presentation, which is called a "peer grade", based on the following aspects: Content, Organization, Clarity, Delivery, Visual Aids, Time Management, and Audience Retention. (A detailed rubric will be given at each presentation meeting). Comments from the instructor will also be given for each presentation.
Project: 55% = 10% (proposal) + 20% (midterm) + 25% (final)
Each student is expected to complete one course project, which shall be tracked and evaluated based on three milestones: the proposal, the midterm report, and the final essay. The proposal and the midterm report shall NOT be longer than one and two pages, respectively. The final essay has no page limit, however is expected to be self-contained within eight pages. Each student needs to make an appointment with the instructor to discuss the submitted final essay during 04/26/2022 -- 04/27/2022. Good essays will be encouraged to be compiled into a conference paper submission, under the instructor's help.
No textbook is required in this course in general. Below are some recommended materials that could consolidate your background knowledge, so as to facilitate your understanding of what shall be covered in this course.
Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. "Deep learning". MIT press, 2016. (mitpress.mit.edu/books/deep-learning)
Yao Ma and Jiliang Tang. "Deep Learning on Graphs". Cambridge University Press, 2021. (web.njit.edu/~ym329/dlg_book/)
*** You are recommended to go through the papers shown below, from which you can choose one that you like the best as your presentation topic. If none of these papers fall into your interest scope, you could also choose a paper from top-tier conferences including, but not limited to, NeurIPS, KDD, ICML, ICLR, ICDM, WWW, AAAI, IJCAI, etc. The selected paper must be published after 2015.
What is deemed as a top-tier conference?
In computer science, we usually use the CORE ranking system (http://portal.core.edu.au/conf-ranks/?search=NeurIPS&by=all&source=CORE2021&sort=atitle&page=1), in which a conference ranked as A* is deemed as a top conference.
Grover, Aditya, and Jure Leskovec. "node2vec: Scalable feature learning for networks." KDD 2016.
Veličković, Petar, et al. "Graph attention networks." ICLR 2018.
Ying, Rex, et al. "Graph convolutional neural networks for web-scale recommender systems." KDD2018.
Ying, Rex, et al. "Hierarchical graph representation learning with differentiable pooling." NeurIPS 2018.
Wang, Xiaoyang, et al. "Traffic flow prediction via spatial-temporal graph neural network." WWW 2020.
Guo, Shengnan, et al. "Attention-based spatial-temporal graph convolutional networks for traffic flow forecasting." AAAI 2019.
Rossi, Emanuele, et al. "Temporal graph networks for deep learning on dynamic graphs." ICLR 2020.
Ying, Chengxuan, et al. "Do Transformers Really Perform Bad for Graph Representation?." ICLR 2021.
Shi, M., Huang, Y., Zhu, X., Tang, Y., Zhuang, Y., & Liu, J. "GAEN: Graph Attention Evolving Networks," IJCAI 2021.
Deng, Songgaojun, Huzefa Rangwala, and Yue Ning. "Learning dynamic context graphs for predicting social events." KDD 2019.
01/11/2022: First class meeting.
03/07/2022 -- 03/12/2022: Spring holiday (no meeting).
04/21/2022: Last class meeting.
04/25/2022: Last day to withdraw.
04/26/2022 -- 04/27/2022: Individual meeting and discussion with Instructor.
Academic Honesty
Everything turned in for grading in this course must be your own work. You are expected to conform to academic standards in avoiding plagiarism and fabrication. Please refer to the IEEE ethical requirements for more details (conferences.ieeeauthorcenter.ieee.org/author-ethics/ethical-requirements/). The instructor reserves the right to question a student orally or in writing and to use his evaluation of the student’s understanding of the presentations and essay submissions as evidence of cheating. Violations will be reported to the Office of Student Conduct and Academic Integrity.
**Citations**:
In write-ups, you may borrow ideas and solutions and draw insights from those published in conference proceedings/journals or posted on the internet; Please provide that you acknowledge your sources appropriately.
1) If you use someone else’s thoughts, proofs, or arguments, you must cite your source appropriately, in a fashion that allows me to verify it.
2) If you use someone else’s wording, you must enclose that wording in quotation marks and cite your source.
Discrimination and Harassment
All people have equal access to programs, facilities, admittance, and employment at the institution. The university has a policy of not harassing or discriminating against anyone because of their age, race, color, ancestry, national origin, religion, creed, veteran status, sex, sexual orientation, marital or family status, pregnancy, pregnancy-related conditions, physical or mental disability, gender, perceived gender, gender identity, genetic information, or service in the uniformed services (as defined by state and federal law). Individual dignity is violated by discriminatory conduct and harassment, as well as sexual misconduct and relationship violence, which obstructs the university's educational goal and will not be tolerated. Gender-based sexual harassment, including sexual violence, is a kind of gender discrimination in which an individual's ability to participate in or benefit from University programs or activities is denied or limited. These policies shall not be construed to restrict academic freedom at the university, nor to restrict constitutionally protected expression. The policy is coded in University Policy #1005.
General University Policies
The ODU Catalog (https://catalog.odu.edu/) lays out the University policies that are binding upon both students and faculty. All students are required to abide by these.
Please check the course website periodically for updates. Substantial changes will also be announced via emails.