Machine Learning
Objective
The objective of this course is to introduce beginner to intermediate level concepts of machine learning. Initially in this course, basic notion of data pre-processing is discussed as this step is absolutely necessary before applying any machine learning algorithm to extract pattern from the data. Most of this course will focus on supervised machine learning algorithms and will train students so that they can understand pros and cons of different algorithms and can select best algorithm for a given dataset / problem. To understand different concepts discussed in this course, students are expected to have strong familiarity with concepts of linear algebra, probability theory, analytical geometry and multivariate calculus.
Course Contents
Week
Module
Topics
Reading / Reference Material
Lecture Notes / Video Recording
1
Introduction to ML
AI vs ML
What is ML? Why it is used?
Taxonomy of ML Algorithms
Chapter 1 & 5: Pattern Recognition, Konstantinos Koutroumbas and Sergios Theodoridi, Academic Press, 4th or latest edition.
Chapter 1: Combining Pattern Classifiers: Methods and Algorithms, Ludmila I. Kuncheva, Wiley-Interscience.
2 - 3
Supervised ML problem setup and Data Preprocessing
Supervised ML problem setup
Objective Function
Data Pre-processing
Outliers
Null values
Scaling
Model Evaluation
Chapter 2: Data Mining, Practical Machine Learning Tools & Techniques, Witten and Franck, Elsevier Books, 2nd Edition.
Chapter 2: Data Mining & Analysis : Fundamental Concepts & Algorithms, Zaki and Meira, Cambridge University Press 2014.
3 - 4
Lazy Learner
Discussion on how to reduce dimensionality
Chapter 8: Machine Learning, Tom Mitchell, McGraw Hill, latest edition.
Chapter 2: Pattern Recognition, Konstantinos Koutroumbas and Sergios Theodoridi, Academic Press, 4th or latest edition.
5
Perceptron
Linear classification
Perceptron learning rule
Geometric Intuition
Proof of convergence
Chapter 4: Machine Learning, Tom Mitchell, McGraw Hill.
Chapter 5: Pattern Classification, R. Duda et al., Wiley Interscience
Chapter 3: Pattern Recognition, Konstantinos Koutroumbas and Sergios Theodoridi, Academic Press, 4th or latest edition.
New Yorker 1958 article on perceptron
6
Perceptron and KNN hands-on programming
Impact of varying values of “k” on accuracy (KNN)
Perceptron learning algorithm
Perceptron visualization
Chapter 1 & 2: Introduction to Machine Learning with Python, Muller et al., O’Reilly.
Course Lecture Notes / Exercise handbook.
7
Kernel Methods
Widest margin theory
Support Vector Machine (SVM)
SVM dual optimization
Different Kernels
Chapter 3: Pattern Recognition, Konstantinos Koutroumbas and Sergios Theodoridi, Academic Press, 4th or latest edition.
Chapter 6 & 7: Pattern Recognition and Machine Learning, Christopher M. Bishop, Springer Books, latest edition.
Book Support Vector Machines Succinctly, Alexandre K, 2017.
8 Mid Term Exam Week
9
Decision Trees
Tree Intuition / Representation
ID3 Algorithm
Data splitting measure / Best Attribute selection
Tree induction example
Chapter 3: Machine Learning, Tom MITCHELL, McGraw Hill, latest edition.
Chapter 9: Introduction to Machine Learning, Ethem ALPAYDIN, The MIT Press, latest edition.
Microsoft Research Technical Report TR-2011-114: A. Criminisi et al. Decision Forests for Classification, Regression, Density Estimation, Manifold Learning and Semi-Supervised Learning. Microsoft Research 2011.
10 - 11
Regression
Linear Regression
Cost function
Gradient Descent
Polynomial Regression
OLS method for Regression
Chapter 1 & 3: Pattern Recognition and Machine Learning, Christopher M. Bishop, Springer Books, latest edition.
Chapter 8: Machine Learning, Tom MITCHELL, McGraw Hill, latest edition.
11
Hands-on programming
Varying Depth of Tree
Overfitting
Under fitting
Gradient Descent implementation
Chapter 1 & 2: Introduction to Machine Learning with Python, Muller et al., O’Reilly.
Course Lecture Notes / Exercise handbook.
12 - 13
Model Debugging and Ensemble Learning methods
Understanding Error
Bias - Variance tradeoff
Dataset split techniques
Dealing with high bias and high variance
Ensemble Learning
Bagging
Boosting
Random Forest
Chapter 3 & 4: Pattern Recognition, Theodoridis et al., Academic Press
Chapter 7: Combining Pattern Classifiers: Methods and Algorithms, Ludmila I. Kuncheva, Wiley-Interscience.
Chapter 14: Pattern Recognition and Machine Learning, Christopher M. Bishop, Springer.
Chapter 8 & 15: The Elements of Statistical Learning, Hastie et al., Springer Books.
Microsoft Research Technical Report TR-2011-114: A. Criminisi et al. Decision Forests for Classification, Regression, Density Estimation, Manifold Learning and Semi-Supervised Learning, 2011.
Lecture
14
Unsupervised Learning
Clustering Intuition
K-means clustering
Finding right number of clusters
Hierarchical clustering
Chapter 9: Pattern Recognition and Machine Learning, Christopher M. Bishop, Springer Books, latest edition.
Chapter 14: Principles of Data Mining, Max Bramer, Springer Books.
Chapter 8: Introduction to Data Mining Kumar et al., 2nd Edition, Pearson Education.
15-16
Biologically inspired (Artificial Neural) Network
Logistic Regression
Logistic / Sigmoid function
Log loss calculation
GD / backpropagation
Artificial Neural Network
Basic intuition
Learning weights
Activation functions
Example solution
Chapter : 4 Pattern Recognition, Theodoridis et al., Academic Press, 4th Edition.
Chapter : 5 Pattern Recognition and Machine Learning, Christopher M. Bishop, Springer Books, latest edition.
Chapter : 4 Machine Learning, Tom Mitchell, McGraw Hill, latest edition.
17 Project Week / Review
18 Final Exam Week
Reference Books
Machine Learning, Tom Mitchell, McGraw Hill.
https://www.amazon.com/exec/obidos/ISBN=0070428077/4660-3450753-250555
Pattern Recognition and Machine Learning, Christopher M. Bishop, Springer Books.
https://www.microsoft.com/en-us/research/people/cmbishop/prml-book/
Pattern Recognition, Konstantinos Koutroumbas and Sergios Theodoridi, Academic Press.
https://www.amazon.com/Pattern-Recognition-Sergios-Theodoridis/dp/1597492728
The Elements of Statistical Learning, Trevor Hastie, Robert Tibshirani and Jerome Friedman, Springer Books.
https://web.stanford.edu/~hastie/ElemStatLearn/
LaTeX Guide
Students are encouraged to write course project report using LaTeX. If you are unfamiliar with LaTex, then you may refer to concise guide that will help you getting started with it. [LaTeX getting started]