------ Announcements ------ 

Welcome to LLMs! The first four weeks of classes will be in person in EB 103. See you then!

Course Information

Instructor: Ameeta Agrawal, PhD, ameeta@pdx.edu

TA: Yufei Tao, yutao@pdx.edu

Lectures: Tue/Thu, 8am - 9:50am

Class Location: Please note that this is a 'hybrid' course with some in-person and some online classes.

>> In-person days: EB 103 (no synchronous Zoom session)

>> Online/remote days: Zoom 

Instructor office hours: Mon 12pm - 1pm on Zoom or by appointment

TA office hours: Wed 10:30 am - 11:30 am in person in the Fishbowl or virtually on Zoom.

Slack channel: Please join #llm_f24 for collaborative learning and communication.

Course Description

Unlock the secrets of cutting-edge Artificial Intelligence (AI) with our Large Language Models course! Dive deep into the fascinating world of transformative models like GPT and beyond. We will learn about the underlying mechanisms driving these large-scale models and leverage their computational capabilities for practical, real-world implementations.

Course Prerequisites

• Proficiency in Python: We'll use Python extensively; make sure you're comfortable with it.

• Basic Probability, Statistics, and Linear Algebra: These fundamentals are essential for understanding Large Language Models.

• (Recommended) Prerequisites: It would help to have completed one of the following courses - Natural Language Processing, Adventures in Natural Language Processing, Machine Learning, Deep Learning, or Foundations of Emerging Technologies.

Reference Texts: All our material will be from resources that are available online. See this list of our weekly readings: https://docs.google.com/document/d/1AFXP9k02ZjBM0oYlE9kkqaGAp2ggv3h0/edit?usp=sharing&ouid=109257511019160107186&rtpof=true&sd=true

Coursework and Grading

• 3 Assignments (45%): You'll complete 3 assignments (each worth 15 points) containing written questions and programming parts. You may form study groups but the assignments are to be completed and submitted individually through Canvas. In other words, your work must be your own.

• 2 Research discussions (20%): You will participate in a 'panel-style' research discussion 2 times during the term, each participation will be worth 10 points. The research paper and role will be assigned by the instructor. 

• 1 Course project (35%): In this project, students will work on exploring real-world applications of LLMs (like GPT-4, BERT, etc.) to address one of the United Nations' 17 Sustainable Development Goals (SDGs) (https://sdgs.un.org/goals). This coursework will focus on practical, impactful uses of LLMs in domains such as education, healthcare, poverty alleviation, climate action, and more. Students will select one SDG and identify ways in which LLMs can contribute to achieving the goal. The project will involve designing a solution using an LLM to address a real-world issue related to the chosen SDG. The project deliverables are as follows:

• • Project discussion (5%): your team will discuss an initial idea with the instructor (5-10 minutes).

    • • Evaluation: How well the project addresses the chosen SDG and its core objectives?

  • Project presentation (15%): a demo/presentation (10 minutes).

    • • Evaluation: Clear articulation of the problem, solution, and potential impact.

  • Final report (15%): a final report including 

    • (i) your project goal focusing on a real-world application where technology can help, 

    • (ii) your methodology/solution that incorporates LLMs to address a specific issue within the chosen SDG, 

    • (iii) a brief analysis of how the solution can positively impact the target group or community, considering factors like scalability, affordability, and sustainability, and 

    • (iv) ethical concerns, such as bias in LLMs or privacy risks, suggesting mitigation strategies for the proposed application (4-6 pages, not including references).

      • • Evaluation: Creativity in applying LLM technology to solve real-world problems, practicality of implementing the proposed solution, and understanding of potential risks, including biases, fairness, and data privacy, and strategies to mitigate them

Research Discussion Roles

How to prepare for RD? 

1. All students in this course read the RD assigned paper. This is important so you can understand the panel discussion because we will not be presenting the paper in class. More importantly, you can/should ask questions during the panel for extra credit.

2. Students in RD1 group read the paper 2-3 times. Then read the paper one more time from their role's perspective to formulate their synthesis/arguments. Self-select a role in this Google Sheet: https://docs.google.com/spreadsheets/d/14z2nwIQXKfxGgFPc1QOyglr-GuIDdsLb-V-qmxDq7TQ/edit#gid=0

3. Students in RD1 group prepare 2-3 minutes of analysis to discuss during the panel. If you think having a slide or two will help (e.g., a visual to support your point), you may use that to illustrate.

4. The Prof. will be facilitating the panel discussion and asking questions.

The roles:

• Archaeologist: You’re an archaeologist who must determine where this paper sits in the context of previous work. Find one older paper cited within the current paper that substantially influenced the current paper and be prepared to discuss what is new. Trace aspects of the paper (e.g., model, training, data) back to prior work and evaluate what innovations have been made.

• Researcher: You are on the test-of-time award committee and are trying to assess the impact of this work after its publication. Find newer papers that cite this work and were substantially influenced by the current paper, or if the current paper is relatively new, think of possible new directions that the current paper could inspire.

• PhD student: Act as a PhD student and propose a research project that would build on the findings or limitations of the paper. Consider improvements in methods, or new questions arising from this work. 

• Salesperson: You are an author of the paper. Convince others why this paper should win a best paper award. Highlight the key contributions in an engaging way.

• Bug Hunter: You are reviewer #2, who wants to dig into the details of the paper. Your job is to uncover any potential weaknesses such as issues with reproducibility, experimental design, or correctness. Provide constructive criticism and suggest corrections or improvements.

• Social Impact Assessor: You are an auditor of the societal impact of the paper. Identify how this paper self-assesses its real-world impact (both positive and negative). Have any social impacts (especially negative ones) been left out?

Rubric for evaluating the roles:

• Depth of analysis (2 pt) - deep and thorough or basic and minimal?

• Use of evidence (2pt) - uses specific data/examples from the paper or lacks evidence to support arguments?

• Critical thinking and response to questions (2pt) - original and thoughtful or struggles to respond meaningfully to questions?

• Presentation/communication (2pt) - clear, concise and highly engaging or unclear, disorganized, difficult to follow?

• Role engagement (2pt) - embodies the assigned role offering deep insights or role-playing is unclear/unfocused?

Course Policies

Late policy: You are allowed a total of 3 late days over the entire term to use for the assignments (no exceptions for project deliverables), please use them wisely. Each late day gives you an extra 24 hours. Submissions received after late days have been exhausted will receive at most 50% of the points.

Canvas/Slack: Please check Canvas regularly for updates. All course materials and assignments will be posted on Canvas. Please use Slack for all questions as this can also potentially help other students who may have similar questions.

Honor code: Academic integrity is a vital part of the educational experience at PSU. Please see the PSU Student Code of Conduct for the university’s policy on academic dishonesty. A confirmed violation of that Code in this course may result in failure of the course. Copying or paraphrasing someone’s work (including code) is not allowed and will result in an automatic grade of 0 for the assignment/project in which copying/paraphrasing was done. If you believe you are going to have trouble completing an assignment, please talk to the instructor or TA well in advance of the due date.

Student Services

Disability Access Statement

If you have, or think you may have, a disability that may affect your work in this class and feel you need accommodations, contact the Disability Resource Center to schedule an appointment and initiate a conversation about reasonable accommodations. The DRC is located in 116 Smith Memorial Student Union, 503-725-4150, drc@pdx.edu, https://www.pdx.edu/disability-resource-center/.

Safe Campus Statement

Portland State University desires to create a safe campus for our students. As part of that mission, PSU requires all students to take the learning module entitled Creating a Safe Campus: Preventing Gender Discrimination, Sexual Harassment, Sexual Misconduct and Sexual Assault. If you or someone you know has been harassed or assaulted, you can find the appropriate resources on PSU’s Enrollment Management & Student Affairs: Sexual Prevention & Response website at http://www.pdx.edu/sexual-assault.