Advanced Topics in Machine Learning

Topic: Large Language Models for Reasoning

Large Language Models (LLMs) are an exciting new development in machine learning and artificial intelligence.  This course explores research directions in using LLMs as a strong base model to enable reasoning tasks.

The goal in this course is for students to be able to:

• Learn about range of reasoning tasks. (Lectures)

• Gain operational understanding of how to use LLMs within applications. (Colab Notebooks)

• Learn to deconstruct research papers. (Paper presentation)

• Formulate challenges in applying LLMs to new reasoning tasks.  (Project Proposal)

• Explore aforementioned challenges as a research project. (Final Project)

Those interested in learning more about the architecture of LLMs and other Generative AI models should take EE 148. 

Prerequisites:

• CS 155 (Machine Learning & Data Mining) -- Hard Prerequisite

• EE 148 (Large Language and Vision Models) -- Soft Prerequisite, encouraged to take concurrently (especially your final project is about fine-tuning LLMs).

Instructors and Teaching Assistants

• Yisong Yue (yyue@caltech.edu) -- Head Instructor

• Ziniu Hu (acbull@caltech.edu) -- Guest Instructor

• Hao Liu (hliu3@caltech.edu) -- Teaching Assistant

• Ivan Jimenez Rodriguez (ivan.jimenez@caltech.edu) -- Teaching Assistant

Course Structure

• Weeks 1 & 2: Introductory Lectures

• Weeks 3-10: Student Presentations & Guest Lectures

• Week 5: Project Proposal

• Week 10: Project Presentation and/or Poster Session

Grading

90% of the grade is on the final project.  10% of the grade is on the presentation.

Student Presentation

Student presentations will revolve around a specific paper.  Each paper will have four presenters taking on the roles of: Champion, Critic, Pioneer, Entrepreneur.

Final Project

The final project should be done in groups (recommended group size is 2-3, max is 4), and one final project document should be submitted per group. The deliverables are: a project proposal due on April 30th, and the final project report due at the last day of class (May 31st).

Tentative: Students will also have an opportunity to present their project in a poster session on May 30th.

LLM APIs

The course requires API access to LLMs (e.g., GPT API).  This course will reimburse $50 per student for API usage.