Pattern Recognition and Machine Learning
Jan to May 2019
Course Information
Space: CRC 102
Time: E Slot. (Tue: 11AM, Wed: 10AM, Thu: 8AM, Friday: 4:50 PM)
Teaching Assistants: TBA
Learning Outcomes
At the end of the course, the student should be able:
- To understand the use cases and limitations of machine learning.
- To recognise the type of learning problem suitable for a practical task at hand.
- To identify the data required for solving a given type of learning problem.
- To identify the right learning algorithms for solving a given learning problem. [KEY]
- To analyse (several) learning algorithms and identify the role of the various critical knobs in the algorithms. [KEY]
- To efficiently use various software packages for solving learning problems.
- To implement various learning algorithms from first principles. [KEY]
Grading
- Mid-term: 20%
- Final exam: 30%
- Min-Quizzes: 15% (Best 3 out of 4)
- Programming Assignments: 20 (10 for each assignment)
- Programming Contest: 15%
Important Dates
Mini-Quiz 1: Feb 1
Mini-Quiz 2: Feb 22
Midsem: Mar 8
Mini-Quiz 3:Mar 22
Mini-Quiz 4: Apr 12
Assignment 1: Available on Feb 8. Due on March 1.
Assignment 2: Available on Mar 15. Due on April 12.
Data contest: Due April 26.
Reference Books:
- [DHS] Richard O. Duda, Peter E. Hart and David G. Stork. Pattern Classification. John Wiley, 2001.
- [CB] Christopher M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006.
- Download PDF from here : https://www.microsoft.com/en-us/research/people/cmbishop/#!prml-book
- [SB] Shai Shalev-Shwartz and Shai Ben-David. Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press, 2014.
- Download PDF from here: http://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/copy.html
- [JW] Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani. An Introduction to Statistical Learning with Applications in R, Springer, 2014.
- Download PDF from here: http://www-bcf.usc.edu/~gareth/ISL
- Jupyter notebooks for ISLR: https://github.com/JWarmenhoven/ISLR-python
Related Resources on the Web:
- Google crash course on machine learning: https://developers.google.com/machine-learning/crash-course/ml-intro
- Intel course on Machine learning: https://software.intel.com/en-us/ai-academy/students/kits/machine-learning-501
- Nptel course on Machine learning: https://nptel.ac.in/downloads/117108048/
- Hal Daume III. A course in machine learning. http://ciml.info/
Reference material for Pre-requisites:
1. Undergraduate multivariate calculus.
1b. https://ocw.mit.edu/courses/mathematics/18-02sc-multivariable-calculus-fall-2010/2.-partial-derivatives/ (Part A and B)
2. Linear algebra.
2a. https://youtu.be/kjBOesZCoqc (Essence of linear algebra, youtube series)
2b. https://youtu.be/ZK3O402wf1c (Gilbert Strang course on Linear algebra.)
Books:
Gilbert Strang. Linear Algebra and its Applications.
3. Probability.
Books:
Bertsekas and Tsitsiklis. Introduction to Probability.
4. Basic Python/Numpy programming.
https://developers.google.com/edu/python/
http://www.astro.up.pt/~sousasag/Python_For_Astronomers/Python_qr.pdf
http://cs231n.github.io/python-numpy-tutorial/
https://www.tutorialspoint.com/numpy
Announcements:
- Mini-quiz 1 to be held on Feb 1
- Programming Assignment 1 released on Feb 12 in Moodle. Part A due on March 1 11:55 PM.
- New worksheet released on Feb 17.
- Programming assignment 1 Part A, deadline extended to Sunday March 3, 11:55 PM.
- Mid-sem exam on March 8 Friday.
- Programming Assignment 2 released. Due March 29.
- Data contest released. Due April 26.
- Programming Assignment 3 released. Due April 26.
Supporting Material:
Class Notes/Slides:
Course instructional handout : pdf
- Introduction slides : Jan 16 : pdf
- Calculus/Geometry notes : Jan 17 : pdf
- Probability/Linear algebra/optimisation notes: Jan 18-25 : pdf
- Bayes classifier, learning from data : Jan 25-31 : pdf
- Regression : Feb5-13 : pdf
- Logistic Regression : Feb 14-19 : pdf
- Constrained Optimisation : Feb 19 -21 : pdf
- SVM : Feb 27-Mar 1 : pdf
- Decision Trees and Boosting: Mar 6 - 13 : pdf
- Ensemble methods: Bagging, Random Forests and Boosting : Mar 7-13 : Slides pdf
- Classification wrap up: kNN, ANN, Multiclass, Surrogate loss functions, Structured prediction : Mar 19-26 : pdf
- PCA : pdf
- Clustering: slides
- Collaborative filtering: pdf
- EM algorithm : Bishop Chapter 9
- Regularisation: TBA
- Learning theory : pdf
Worksheets:
- Probability basics : Available on Moodle
- Mega-worksheet - Prerequisite/Regression/classification : pdf
- Linear algebra/least squares regression comprehensive worksheet: Gilbert Strang (4th Ed) book exercises from Sections 3.1, 3.2 and 3.3.
- Classification worksheet: Gather from "Exercises" in the notes.
- Multiclass, clustering, PCA, collaborative filtering: pdf
- EM algorithm : pdf
Quiz papers/Solutions:
- Mini-quiz 1: Qn paper : Solutions
- Mini-quiz 2: Qn paper : Solutions
- Mid-sem : Qn paper : Solutions
- Mini-quiz 3: Qn paper : Solutions
- Mini-quiz 4: Qn paper : Solutions
Jupyter Notebooks:
- Multivariate-Normal illustration : notebook
- 1d-Regression illustration : notebook
- Programming assignment 1: Released on Moodle.
- SVM illustration: notebook
- Programming Assignment 2: Released on Moodle.
- Programming Assignment 3: Released on Moodle
- Lasso Regression illustration: notebook