Schedules: MWF 10:00 AM - 10:50 AM, Spieth Hall | Room 2200
February 16th: Midterm
Schedule and Paper List
List of weekly research papers will be updated during the first week of the classes.
Week 1 (Jan 8) : NLP Tasks and Neural Models (Transformers, BERT, T5, GPT Families)
Week 1-1: [Slides] Introduction
Week 1-2: [Slides] Language Models
Week 1-3: [Slides] Transformers
Week 2 (Jan 15): NLP Core Techniques (Fine-tuning, Pre-training, RLHF)
No class on Monday, Jan. 15th (national holiday) Homework 1 released
Week 2-1: [Slides] Pretrained Models
Week 2-2: [Slides] Instruction Tuning and In Context Learning
Week 3 (Jan 22): LLM Security/Limitation
Week 3-1 (Yue): Is LLM Smart, Safe, Fair?
Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs Through a Global Prompt Hacking Competition (EMNLP 2023 best paper)
Universal and Transferable Adversarial Attacks on Aligned Language Models
A Watermark for Large Language Models (ICML 23 outstanding paper)
Additional papers:
Lost in the Middle: How Language Models Use Long Contexts
Large Language Models Can Be Easily Distracted by Irrelevant Context
Week 4 (Jan 29): LLM Analysis/Evaluations
Week 4-1 (Yue): Final project overview
Is ChatGPT a General-Purpose Natural Language Processing Task Solver?
Are Emergent Abilities of Large Language Models a Mirage? (NeurIPS outstanding paper)
Additional papers:
The Curse of Recursion: Training on Generated Data Makes Models Forget
Week 5 (Feb 5): LLM Reasoning/Grounding
W5 Monday Zoom (weather) Advanced prompt engineering
Chain-of-thought prompting elicits reasoning in large language models
Tree of thoughts: Deliberate problem solving with large language models
Self-Consistency Improves Chain of Thought Reasoning in Language Models
Wednesday, Feb 7th, 11:59pm PST: homework due
Additional papers:
Week 6 (Feb 12): Multi-Modal Models
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
Friday, Feb. 16th: Midterm
Additional papers:
Week 7 (Feb 19): Project Proposal Week
Proposal Presentation Instruction (Zoom office hour Tuesday Feb 20th 10-11am)
No class on Monday, Feb 19th (national holiday)
Wednesday, Feb 21st: project proposal 1
Friday, Feb 23rd: project proposal 2
Week 8: RAG/LLM agents/LLM for Coding
Survey: Retrieval-Augmented Generation for Large Language Models: A Survey
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
Toolformer: Language Models Can Teach Themselves to Use Tools
Additional papers:
Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents
ReAct: Synergizing Reasoning and Acting in Language Models
Week 9: Bias and Efficiency
From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models (ACL 23 best paper)
LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention
DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models (NeurIPS outstanding paper)
Week 10: LLM Safety & Interpretability & Defense
Week 10-1: DPO and Interpretability
Week 10-2: Interpretability and Attacks
Final project due: March 24th, 11:59pm PST. [Final project description]
Papers of interests:
Textbook is all you need
Large Language Model Unlearning
Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attacks
Mamba: Linear-Time Sequence Modeling with Selective State Spaces