courses‎ > ‎

### CS 651 Machine Learning Jan 2018

Preamble

Machine learning is about designing programs that can learn without being explicitly programmed. It is a branch of Artificial Intelligence in which we learn concepts/patterns/hypotheses from Data by using heuristic based algorithms. Accordingly, this field is about study and implementation of two main category of algorithms: Supervised and Unsupervised. Supervised learning algorithms make use of data with known classification, aka labeled examples whereas Unsupervised learning algorithms use data with unknown classification, aka unlabeled examples. This field has become so popular that one can find machine leaning applications in virtually all domains ranging from identifying emails as spam or legitimate to automated vehicle guided system to game playing to credit card fraud detection. As this form is unlikely to become exact science, a learning method/algorithm needs to be evaluated and estimated for its performance on unseen data or the population.

Course Contents                                                                                                                                                                 [Project Page]

Learning Problem, Designing a Learning System, Supervised Learning - Linear and Logistic regression, Decision Tree Learning, Instance-Based Learning, kNN and CBR, Bayesian Learning, Naive Bayes Classifier, Artificial Neural Network (ANN), Unsupervised Learning- K-Means Clustering, Association Rule Mining, Formulating and Evaluating Hypotheses, Computational Learning Theory, Issues and practical advice in Machine Learning,

Ref Book (TB): Tom Mitchell. Machine Learning, Mc Graw Hill, 1997.

References

Course Slides and other Reading material  [A companion ML programming resources page]

 Topics Slides Readings Assignments/Exercises 1. Learning Problem Lecture #1 TB: Chapter 1 Linear Regression*2. Supervised Learning, Linear Regression3. LR of One Variable using Gradient Descent Algo4. LR of Multiple Variables using Gradient Descent Algo Lecture #2 Lecture #3 Lecture #4 TB: 4.4.3 Gradient Descent and Delta Rule (Page 89) 4.4.3.1 Visualizing the hypothesis space 4.4.3.2 Derivation of the gradient descent rule 4.4.3.3 Stochastic Approximation to Gradient Descent 4.5.1 A differentiable threshold unit  (This is for logistic regression) Assignment-Ex- Linear RegressionEx- Multivariate LR 6. Logistic Regression, Hypothesis representaiton,7. Logistic Regression, Cost Function, Gradient descent Lecture #6 Lecture #7 8. Regularization Lecture #8 9. Performance Measures Lecture#9 10 Programming for ML (Python) Lecture#10-11-12 11. Neural Networks12. Representation13. Cost Function 14. Back propagation and gradient Descent Lecture #13 Lecture #14 Lecture #15 Lecture #16 15. Rule Based Learning16. Decision Tree Learning - I17. Decision Tree Learning - II18. Ensemble Learning Bagging and Boosting Lecture #17 Lecture #18 Lecture #19 Lecture #20,21 19. Error in Classification, Confidence Interval20. Hypothesis Testing, type I and Type II Error, p-value Lecture #22 Lecture #23 Instance Based Learning 21. K-Nearest Neighbors22. Case Based Reasoning and Systems Lecture #24 Lecture #25 8.2 k-Nearest Neighbor Learning  8.2.1 Distance Weighted NN Algorithm 8.2.2 Remarks on K-NN,  8.5 CBR 8.6 Remarks on CBR Unsupervised Learning 23. K-means algorithm, DBSCAN, mixture models and EMDimentionality Reduction Lecture #26 Lecture #27 Lecture #34 Lecture #35 Ch. 6 - 6.12 DBSCAN Lecture slides include topics covered by Mr Punit Kumar 24. Support vector machines Lecture #28 Lecture #29 Slides contributed by Vikas Malviya Bayesian LearningComputing Probability, Bays Theorem, Naive Bays Classifier Lecture #30,#31 TB: 6.2 Bayes theorem,  Table 6.1 Summary of basic probability formulas 6.4 Maximum Likelihood and Least-squared error hypotheses 6.5Maximum Likelihood Hypotheses for Predicting  Probabilities 6.9 Naive Bayes classifier 6.10 http://www.cs.cmu.edu/~tom/mlbook/NBayesLogReg.pdf Association Rule Mining/Learning, Apriori algorithm, frequent itemsets Lecture #32,33 http://www-users.cs.umn.edu/~kumar/dmbook/ch6.pdf Section 6.1, 6.2, 6.3 Bias and Variance Tradeoff Lecture # Understanding the Bias-Variance Tradeoff

1.    A Few Useful Things to Know about Machine Learning, Communication of the ACM, 2012 (another version)
2.    Machine Learning & Artificial Intelligence: Main Developments in 2016 and Key Trends in 2017 @KDnuggets
3.
Machine Learning and Cyber Security Resources @KDnuggets

References for Project Ideas/ Datasets for Machine Learning Projects/Competitions

1. UCI Machine learning repository - Perhaps the biggest repository for ML applications/projects
2. Competitions@Kaggle and Datasets@Kaggle
3. Sample ML Projects -  A big list can be found (Student ML projects at Stanford) - 2016, 2015, 2014 2013, 2012, 2011, 2010, 2009, 2008, 2007, 2006, 2005, 2004
4.
Carlos Guestrin's class at CMU.
5. Goeff Gordon's class at CMU
6. Ray Mooney's class, UT Austin
7. Andreas Krause's class, Caltech
8. Data Science Student Projects at Radboud University.

Programming Environment and Tools

2. MATLAB
3. R Programming
4. Octav
5. Machine learning in Java
6. Weka
7. Google cloud machine leaning services

Evaluation Criteria

Mid Sem 20%
Quizzes 20%
End Sem 40%
Course Project 20% + 10% (Extra)

1. Machine Learning Course at Stanford (also at Coursera)
2. Machine Learning Course at Caltech
5. Machine Learning Group at Saarland University
6. Notes on various topics in Machine Learning
7. Machine Learning Resources list

Latest Happenings

for recent trends/articles in Machine Learning/Data Science
2.

Subpages (1):