courses‎ > ‎

CS 651 Machine Learning Jan 2018

               Machine learning is about designing programs that can learn without being explicitly programmed. It is a branch of Artificial Intelligence in which we learn concepts/patterns/hypotheses from Data by using heuristic based algorithms. Accordingly, this field is about study and implementation of two main category of algorithms: Supervised and Unsupervised. Supervised learning algorithms make use of data with known classification, aka labeled examples whereas Unsupervised learning algorithms use data with unknown classification, aka unlabeled examples. This field has become so popular that one can find machine leaning applications in virtually all domains ranging from identifying emails as spam or legitimate to automated vehicle guided system to game playing to credit card fraud detection. As this form is unlikely to become exact science, a learning method/algorithm needs to be evaluated and estimated for its performance on unseen data or the population.

        Course Contents                                                                                                                                                                 [Project Page]

Learning Problem, Designing a Learning System, Supervised Learning - Linear and Logistic regression, Decision Tree Learning, Instance-Based Learning, kNN and CBR, Bayesian Learning, Naive Bayes Classifier, Artificial Neural Network (ANN), Unsupervised Learning- K-Means Clustering, Association Rule Mining, Formulating and Evaluating Hypotheses, Computational Learning Theory, Issues and practical advice in Machine Learning,

Ref Book (TB): Tom Mitchell. Machine Learning, Mc Graw Hill, 1997.


        Course Slides and other Reading material  [A companion ML programming resources page]

 Topics Slides Readings
1. Learning Problem
 Lecture #1
 TB: Chapter 1
Linear Regression*
2. Supervised Learning, Linear Regression
3. LR of One Variable using Gradient Descent Algo
4. LR of Multiple Variables using Gradient Descent Algo
 Lecture #2
 Lecture #3
 Lecture #4
 TB: 4.4.3 Gradient Descent and Delta Rule (Page 89) Visualizing the hypothesis space Derivation of the gradient descent rule Stochastic Approximation to Gradient Descent
 4.5.1 A differentiable threshold unit  (This is for logistic regression)

Ex- Linear Regression
Ex- Multivariate LR

6. Logistic Regression, Hypothesis representaiton,
7. Logistic Regression, Cost Function, Gradient descent
 Lecture #6
 Lecture #7

8. Regularization
 Lecture #8

9. Performance Measures
10 Programming for ML (Python)
11. Neural Networks
12. Representation
13. Cost Function
14. Back propagation and gradient Descent
 Lecture #13
 Lecture #14
 Lecture #15
 Lecture #16
15. Rule Based Learning
16. Decision Tree Learning - I
17. Decision Tree Learning - II
18. Ensemble Learning Bagging and Boosting

 Lecture #17
 Lecture #18
 Lecture #19
 Lecture #20,21

19. Error in Classification, Confidence Interval
20. Hypothesis Testing, type I and Type II Error, p-value
 Lecture #22
 Lecture #23

Instance Based Learning
21. K-Nearest Neighbors
22. Case Based Reasoning and Systems
 Lecture #24
 Lecture #25
 8.2 k-Nearest Neighbor Learning  8.2.1 Distance Weighted NN Algorithm
 8.2.2 Remarks on K-NN,  8.5 CBR
 8.6 Remarks on CBR

Unsupervised Learning
23. K-means algorithm, DBSCAN,
mixture models and EM
Dimentionality Reduction
  Lecture #26
 Lecture #27
 Lecture #34
 Lecture #35
 Ch. 6 - 6.12
 Lecture slides include topics covered by Mr Punit Kumar

24. Support vector machines Lecture #28
 Lecture #29

Slides contributed by Vikas Malviya
Bayesian Learning
Computing Probability, Bays Theorem, Naive Bays Classifier

 Lecture #30,#31
 TB: 6.2 Bayes theorem,  Table 6.1 Summary of basic probability formulas
 6.4 Maximum Likelihood and Least-squared error hypotheses
Maximum Likelihood Hypotheses for Predicting  Probabilities
 6.9 Naive Bayes classifier


Association Rule Mining/Learning, Apriori algorithm, frequent itemsets
 Lecture #32,33
 Section 6.1, 6.2, 6.3

Bias and Variance Tradeoff
 Lecture #
Understanding the Bias-Variance Tradeoff  

       Additional (Suggested) Readings

         1.    A Few Useful Things to Know about Machine Learning, Communication of the ACM, 2012 (another version)
         2.    Machine Learning & Artificial Intelligence: Main Developments in 2016 and Key Trends in 2017 @KDnuggets
Machine Learning and Cyber Security Resources @KDnuggets

References for Project Ideas/ Datasets for Machine Learning Projects/Competitions

1. UCI Machine learning repository - Perhaps the biggest repository for ML applications/projects
2. Competitions@Kaggle and Datasets@Kaggle
3. Sample ML Projects -  A big list can be found (Student ML projects at Stanford) - 2016, 2015, 2014 2013, 2012, 2011, 2010, 2009, 2008, 2007, 2006, 2005, 2004
Carlos Guestrin's class at CMU.
5. Goeff Gordon's class at CMU
6. Ray Mooney's class, UT Austin
7. Andreas Krause's class, Caltech
8. Data Science Student Projects at Radboud University.

Practical Tips and Advice

Programming Environment and Tools

3. R Programming
4. Octav
5. Machine learning in Java
6. Weka
7. Google cloud machine leaning services

Evaluation Criteria

Mid Sem 20%
Quizzes 20%
End Sem 40%
Course Project 20% + 10% (Extra)

        Some useful links

1. Machine Learning Course at Stanford (also at Coursera)
2. Machine Learning Course at Caltech
5. Machine Learning Group at Saarland University
6. Notes on various topics in Machine Learning
7. Machine Learning Resources list

Latest Happenings

         1. @KDnuggets for recent trends/articles in Machine Learning/Data Science