Course Description

Objective

The main goal of Machine Learning (ML) is the development of systems that are able to autonomously change their behavior based on experience. ML offers some of the more effective techniques for knowledge discovery in large data sets. ML has played a fundamental role in areas such as bioinformatics, information retrieval, business intelligence and autonomous vehicle development.
The main goal of this course is to study the computational, mathematical and statistical foundations of ML, which are essential for the theoretical analysis of existing learning algorithms, the development of new algorithms and the well-founded application of ML to solve real-world problems.

Methodology

  • Professor's lectures on fundamental topics
  • Practical assignments and exercises to be solved by students 
  • Technical papers' review and presentation by students
  • Final project
  • Written and practical tests. Students must show a good grasp of concepts and skills covered in the course.

Contents


Topic Material Assignments Presentations
1. Introduction
Brief Introduction to ML
[Mit97] Cap 1
[Alp10] Cap 1,2
[DHS00] A.1, A.2
Assignment 1 Videos:
Machine Learning: A Love Story

Winning The DARPA Grand Challenge
Review:
Linear Algebra and Probability Review (part 1 Linear Algebra, part 2 Probability)

2. Bayesian decision theory
2.1 A review of probability theory
2.2 Classification
2.3 Loss and risk
2.6 Maximum likelihood estimation
2.7 Bayesian estimation
2.8 Parametric Classification
[Alp10] Chap 3, Chap 4,
Chap 5  
[DHS00] Chap 3
[Tenenbaum06]

3. Graphical models
3.1 Conditional independence
3.2 Naive Bayes classifier
3.3 Hidden Markov
2.5 Bayesian Networks
2.6 Belief propagation
2.7 Markov Random Fields
[Alp10] Chap 16
Markov Random Fields
Assignment 3 Video:
Embracing uncertainty: the new machine intelligence
Presentation:
Eduardo Santos [Bishop07]
  
Marla Contanza Barrera [Ratnaparkhi97] 
3. Kernel methods
3.1 The kernel trick
3.2 Kernel ridge regression
3.3 Kernel functions
3.4 Other kernel Algorithms
3.5 Kernels in complex structured data
[SC04] Chap 2
[Alp10] Chap 13
Introd. to kernel methods


Presentation:
Carlos Rodríguez 
[Kwok04]
Rafael Niquefa [Quadrianto10]
John Alexánder Garzón [Grauman05 
4. Support vector learning
4.1 Support vector machines
4.2 Regularization and model complexity
4.3 Risk and empirical risk
4.4 SVM variations
[Alp10] Chap 13
An introduction to ML, Smola
Support Vector Machine Tutorial, Weston
Assignment 4
May 9th
Presentation:
Angélica Atehortua [Chalimourda04
David Alberto Montaño [Laskov06] 
5. Performance evaluation
5.1 Performance evaluation in supervised learning
5.2 Performance evaluation in unsupervised learning
5.3 Hypothesis testing

[Alp10] Chap 19
[TSK05] Chap 8 (Sect. 8.5)

Presentation:

6. Unsupervised learning
6.1 Mixture densities
6.2 Expectation maximization
6.3 Mixture of latent variables models
6.4 Latent semantic analysis
6.5 Non-negative matrix factorization
[Alp10] Chap 7
Latent Semantic Indexing, Prasad
Generative Learning for BOF,  Lazebnik
NMF for Multimodal Image Retrieval, González

Presentation:
Jesús David Yanquén [Ding08]
Pedro Luis Monroy [Dhillon04]  
7. Learning on complex-structured and non-structured data
7.1 Sructured output prediction
7.2 Structured SVM


Presentation:
Fabián Trinidad Ropero M. [Joachims09]

8. Large Scale Machine Learning     Presentation:
Hugo Camilo Salomón [Lin12]
Sebastián Otálora [Vevaldi12] 
Final Exam:

May 11th
Project:

Proposal: May 4th
Final report and
presentation: Jun 20th

Grading

  • Assignments 40%
  • Exams 30%
  • Presentation 15%
  • Final project 15%

References

  • [Alp10] Alpaydin, E. 2010 Introduction to Machine Learning, 2Ed. The MIT Press.
  • [Bis06] Bishop, C. 2006 Pattern Recognition and Machine Learning. Springer-Verlag.
  • [Mit97] Mitchell, T. M. 1997 Machine Learning. 1st. McGraw-Hill Higher Education.
  • [DHS00] Duda, R. O., Hart, P. E., and Stork, D. G. 2000 Pattern Classification (2nd Edition). Wiley-Interscience.
  • [HTF01] Hastie, T. and Tibshirani, R. and Friedman. 2001 The elements of statistical learning: data mining, inference, and prediction. Springer.
  • [SC04] Shawe-Taylor, J. and Cristianini, N. 2004 Kernel Methods for Pattern Analysis. Cambridge University Press.
  • [TSK05] Pang-Ning Tan, Michael Steinbach, Vipin Kumar,  2005, Introduction to Data Mining, Addison-Wesley.
  • [CST00] Cristianini, N. and Shawe-Taylor, J., 2000, An introduction to support Vector Machines: and other kernel-based learning methods,, Cambridge Univ Press.
  • [SS02] Scholkopf, B. and Smola, A.J., 2002, Learning with kernels, MIT Press.
  • [Bak07] Bakir, G. (Ed), 2007, Predicting Structured Data, MIT Press.
  • [OCW-ML] 6.867 Machine Learning, Fall 2006,  MIT OpenCourseWare.
  • [STANFD-ML] Andrew Ng, CS229 Machine Learning, Stanford University