IFT 6765 - H2025 - Lecture Schedule

IFT 6765 - Links between Computer Vision and Language

Course Lectures

Lecture 1 : Introduction to the course

Lecturer: Aishwarya Agrawal
Slides (pdf, keynote)

Lecture 2 : Vision-Language landscape before Transformer + Pre-training

Lecturer: Aishwarya Agrawal
Slides (pdf, keynote)

Lecture 3: Vision-Language landscape during Transformer + Pre-training 

Lecturer: Aishwarya Agrawal
Slides (pdf, keynote)

Lecture 4: Shortcomings of Vision-Language models and Open Challenges

Lecturer: Aishwarya Agrawal
Slides (pdf, keynote)

Lecture 5: Image captioning

Review Paper

Paper presentation 1: Image captioning Slides

Paper presentation 2: Image captioning Slides

Project presentation 1 : Masked Visual Pre-Training for Mitigating Language Priors in Vision-Language Models Slides
Project presentation 2 : RAG for VQA Slides

Lecture 6: Visual Question Answering: Datasets

Review Paper

Paper presentation 1: Visual Question Answering - Part 1 Slides

Paper presentation 2: Visual Question Answering - Part 2 Slides

Project presentation 1 : Visual Question Answering for Bird Biodiversity and Species Discovery Slides

Project presentation 2 : Effects Of Using Counterfactual Images In Fine Tuning For VQA Slides

Lecture 7: Visual Dialog: Datasets and Models

Review Paper

Paper presentation 1: Visual Dialog - Part 1 Slides

Paper presentation 2: Visual Dialog - Part 2 Slides

Project presentation 1 : Improving VLA with World Model Slides

Project presentation 2 : QA on Doc VQA Slides

Lecture 8: Interpretability and Explainability

Review Paper

Paper presentation 1: Interpretability and Explainability - Part 1 Slides

Paper presentation 2: Interpretability and Explainability - Part 2 Slides

Project presentation 1 : In Context learning with fine-grained VQA Slides

Project presentation 2 : Escher: Reasoning for faithful Image Editing Slides

Lecture 9: Finetuning based VLP models

Review Paper

Paper presentation 1: Finetuning based VLP models - Part 1 Slides

Paper presentation 2: Finetuning based VLP models - Part 2 Slides

Project presentation 1 : Quantifying aleaotric uncertainity in Visual Autoregressive Image Generation Slides

Project presentation 2 : Improving CLIP in zero-shot classification Slides

Lecture 10: LLM based vision-language models

Review Paper

Paper presentation 1: LLM based vision-language models - Part 1 Slides

Paper presentation 2: LLM based vision-language models - Part 2 Slides

Project presentation 1 : Distilling OOD Robustness from VLMs Slides

Project presentation 2 : Enhancing medical classification with multi-modal inputs Slides

Lecture 11: VLP models for vision: classification, segmentation

Review Paper

Paper presentation 1: VLP models for vision: classification, segmentation - Part 1 Slides

Paper presentation 2: VLP models for vision: classification, segmentation - Part 2 Slides

Project presentation 1 : Improving compositional reasoning of MLLMs Slides

Project presentation 2 : Compositional generalization in T2I models Slides

Lecture 12: Efficient VLMs

Review Paper

Paper presentation 1: Efficient VLMs - Part 1 Slides

Paper presentation 2: Efficient VLMs - Part 2 Slides

Project presentation 1 : Interpretability and Reliability of MoE in VLMs Slides

Project presentation 2 : Efficient Test Time Adaptation for VLMs Slides

Lecture 13: Shortcomings of Vision-Language models

Review Paper

Paper presentation 1: Shortcomings of VLMs - Part 1 Slides

Paper presentation 2: Shortcomings of VLMs - Part 2 Slides

Project presentation 1 : Masked pre-training for reducing language priors in VLMs Slides

Project presentation 2 : Retrival Augmentation in VQA to reduce Hallucination Slides

Lecture 14: Beyond statistical learning in vision-language

Review Paper

Paper presentation 1: Beyond statistical learning in vision-language - Part 1 Slides

Paper presentation 2: Beyond statistical learning in vision-language - Part 2 Slides

Project presentation 1 : Effects Of Using Counterfactual Images In Fine Tuning For VQA Slides

Project presentation 2 : Visual Question Answering for Bird Biodiversity and Species Discovery Slides

Lecture 15: Image Generation

Review Paper

Paper presentation 1: Image Generation - Part 1 Slides

Paper presentation 2: Image Generation - Part 2 Slides

Project presentation 1 : Improving VLA with World Model Slides

Project presentation 2 : QA on Doc VQA Slides

Lecture 16: Video understanding and generation

Review Paper

Paper presentation 1: Video understanding and generation - Part 1 Slides

Paper presentation 2: Video understanding and generation - Part 2 Slides

Project presentation 1 : In Context learning with fine-grained VQA Slides

Project presentation 2 : Escher: Reasoning for faithful Image Editing Slides

Lecture 17: Embodied AI

Review Paper

Paper presentation 1: Embodied AI - Part 1 Slides

Paper presentation 2: Embodied AI - Part 2 Slides

Project presentation 1 : Quanitfying uncertainity in visual autoregrssive generation Slides

Project presentation 2 : Improving CLIP for zero-shot classification Slides

Lecture 18: 3D understanding and generation

Review Paper

Paper presentation 1: 3D understanding and generation - Part 1 Slides

Paper presentation 2: 3D understanding and generation - Part 2 Slides

Project presentation 1 : Role of language in robust KD Slides

Project presentation 2 : Enchancing medical classification with MLLMs Slides

Lecture 19: Applications of VLMs

Review Paper

Paper presentation 1: Applications of VLMs - Part 1 Slides

Paper presentation 2: Applications of VLMs - Part 2 Slides

Project presentation 1 : Improving Compositional Reasoning of MLLMs Slides

Project presentation 2 : Compositional Generalization in Text-to-Image Models Slides

IFT 6765 - Links between Computer Vision and Language

Course Lectures

Lecture 1 : Introduction to the course

Lecture 2 : Vision-Language landscape before Transformer + Pre-training

Lecture 3: Vision-Language landscape during Transformer + Pre-training

Lecture 4: Shortcomings of Vision-Language models and Open Challenges

Lecture 5: Image captioning

Lecture 6: Visual Question Answering: Datasets

Lecture 7: Visual Dialog: Datasets and Models

Lecture 8: Interpretability and Explainability

Lecture 9: Finetuning based VLP models

Lecture 10: LLM based vision-language models

Lecture 11: VLP models for vision: classification, segmentation

Lecture 12: Efficient VLMs

Lecture 13: Shortcomings of Vision-Language models

Lecture 14: Beyond statistical learning in vision-language

Lecture 15: Image Generation

Lecture 16: Video understanding and generation

Lecture 17: Embodied AI

Lecture 18: 3D understanding and generation

Lecture 19: Applications of VLMs

Lecture 20: VLM Safety

Lecture 3: Vision-Language landscape during Transformer + Pre-training 