Week 1: Multimedia Semantic Forensics
Watch Those Words: Video Falsification Detection Using Word-Conditioned Facial Motion
Detecting Deep-Fake Videos From Aural and Oral Dynamics
Detecting Deep-Fake Videos From Phoneme-Viseme Mismatches
NewsCLIPpings: Automatic Generation of Out-of-Context Multimodal Media
Week 2: Self-Supervision Revisited
Masked Autoencoders Are Scalable Vision Learners
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Week 3: Language and 3D Vision / Graphics Part 1
Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddings
Looking Outside the Box to Ground Language in 3D Scenes
Week 4: Language and 3D Vision / Graphics Part 2
Zero-Shot Text-Guided Object Generation with Dream Fields
Text2Mesh: Text-Driven Neural Stylization for Meshes
Language2Pose: Natural Language Grounded Pose Forecasting
Week 5: No Class
Week 6: Explainable and Advisable AI
Editing a Classifier by Rewriting its Prediction Rules
Natural Language Descriptions of Deep Visual Features
Week 7: No Class
Week 8: Structured prediction tasks (VRD, HOI, Scene Graphs)
Graphical Contrastive Losses for Scene Graph Parsing
Unbiased Scene Graph Generation from Biased Training
Week 9: No Class
Week 10: Compositionality
Compositionality as Lexical Symmetry
Task-Driven Modular Networks for Zero-Shot Compositional Learning
Week 11: Multimodal Dialog
VD-BERT: A Unified Vision and Dialog Transformer with BERT
History for Visual Dialog: Do we really need it?
Week 12: Language and Robotics
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
Concept2Robot: Learning Manipulation Concepts from Instructions and Human Demonstrations
Week 13: HowTo: Narrated Instructional Video
VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding
MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound
Week 14: Visuolanguage Generative Models
High-Resolution Image Synthesis with Latent Diffusion Models
Hierarchical Text-Conditional Image Generation with CLIP Latents
Week 15: Dead week
Week 16: (Finals week:) Project presentations