Pattern Recognition and Machine Learning
Aug to Dec 2020
Course Information
Space: CS 25
Time: F Slot. (Tue: 5PM, Wed: 11AM, Thu: 9AM, Friday: 8 AM)
Teaching Assistants: TBA
Learning Outcomes
At the end of the course, the student should be able:
To understand the use cases and limitations of machine learning.
To recognise the type of learning problem suitable for a practical task at hand.
To identify the data required for solving a given type of learning problem.
To identify the right learning algorithms for solving a given learning problem. [KEY]
To analyse (several) learning algorithms and identify the role of the various critical knobs in the algorithms. [KEY]
To efficiently use various software packages for solving learning problems.
To implement various learning algorithms from first principles. [KEY]
Grading
Final exam: 50%
Programming Assignments: 30%
Data Contest: 20%
Links
Google group link : https://groups.google.com/g/prml-cs5691-iitm-aug20
Prof. Arun's section link : https://sites.google.com/view/arun-rajkumar/teaching/pattern-recognition-and-machine-learning-cs5691
Reference Books:
[DHS] Richard O. Duda, Peter E. Hart and David G. Stork. Pattern Classification. John Wiley, 2001.
[CB] Christopher M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006.
Download PDF from here : https://www.microsoft.com/en-us/research/people/cmbishop/#!prml-book
[SB] Shai Shalev-Shwartz and Shai Ben-David. Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press, 2014.
Download PDF from here: http://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/copy.html
[MRT] Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of Machine Learning. MIT Press. 2018.
Download PDF from here :https://cs.nyu.edu/~mohri/mlbook/
[DFO] Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong. Mathematics for Machine Learning. Cambridge University Press, 2020.
Book site here : https://mml-book.github.io/
[JW] Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani. An Introduction to Statistical Learning with Applications in R, Springer, 2014.
Download PDF from here: http://www-bcf.usc.edu/~gareth/ISL
Jupyter notebooks for ISLR: https://github.com/JWarmenhoven/ISLR-python
Related Resources on the Web:
Google crash course on machine learning: https://developers.google.com/machine-learning/crash-course/ml-intro
Intel course on Machine learning: https://software.intel.com/en-us/ai-academy/students/kits/machine-learning-501
Nptel course on Machine learning: https://nptel.ac.in/downloads/117108048/
Hal Daume III. A course in machine learning. http://ciml.info/
Reference material for Pre-requisites:
1. Undergraduate multivariate calculus.
1b. https://ocw.mit.edu/courses/mathematics/18-02sc-multivariable-calculus-fall-2010/2.-partial-derivatives/ (Part A and B)
2. Linear algebra.
2a. https://youtu.be/kjBOesZCoqc (Essence of linear algebra, youtube series)
2b. https://youtu.be/ZK3O402wf1c (Gilbert Strang course on Linear algebra.)
Books:
Gilbert Strang. Linear Algebra and its Applications.
3. Probability.
Books:
Bertsekas and Tsitsiklis. Introduction to Probability.
Grimmett and Stirzaker. Probability and Random Processes.
Bruce Hajek. Probability course notes. Link
4. Basic Python/Numpy programming.
https://developers.google.com/edu/python/
http://www.astro.up.pt/~sousasag/Python_For_Astronomers/Python_qr.pdf
http://cs231n.github.io/python-numpy-tutorial/
https://www.tutorialspoint.com/numpy
Announcements:
Supporting Material:
Class Notes/Slides:
Course instructional handout : pdf
Introduction slides : pdf
Calculus/Optimisation : pdf
Linear algebra : pdf
Probability : pdf
Bayes classifier : pdf
Regression : pdf
Logistic Regression : pdf
Constrained optimization : pdf
SVM : pdf
Decision Trees and Boosting : pdf
Ensemble methods slides : pdf
Supervised learning wrap-up : Surrogate losses, Neural nets, multiclass methods, kNN, Precision/Recall : pdf
PCA : pdf
Clustering : pdf
GMM/EM : pdf
Collaborative filtering : pdf. Link to Paper on Neighborhood models : Link
Learning Theory : pdf
LASSO : pdf
External Links:
Decision trees : link
Clustering visualization : link
Neighborhood models in collaborative filtering : link
Jupyter Notebooks:
SVM examples. Link
PCA example : Link
Spectral Clustering illustration : Link
GMM EM. Link
LASSO Regression : Link
Worksheets:
Probability basics : pdf (V1)
Mega-worksheet - Prerequisite/Regression/classification : pdf (V1)
SVMs/Classification : pdf
Trees/Boosting : pdf
Multiclass methods/Clustering/Collaborative filtering : pdf
EM/GMM/HMM : pdf
Sample/Previous question papers :