Objective
The main goal of Machine
Learning
(ML) is the development of systems that are able to autonomously change
their behavior based on experience. ML offers some of the more
effective techniques for knowledge discovery in large data sets. ML has
played a fundamental role in areas such as bioinformatics, information
retrieval, business intelligence and autonomous vehicle development.
The main goal of this course is to study the computational,
mathematical and statistical foundations of ML, which are essential for
the theoretical analysis of existing learning algorithms, the
development of new algorithms and the wellfounded application of ML to
solve realworld problems.
Methodology
 Professor's lectures on fundamental topics
 Practical assignments and exercises to be solved
by students
 Technical papers' review and presentation by students
 Final project
 Written and practical tests. Students must show a
good grasp of concepts and skills covered in the course.
Contents
Topic 
Material 
Assignments 
Presentations 
1. Introduction

Brief Introduction to ML
[Mit97] Cap 1
[Alp10] Cap 1,2
[DHS00] A.1, A.2

Assignment 1 
Videos:
Machine Learning: A Love Story
Winning The DARPA Grand Challenge
Review:
Linear
Algebra and Probability Review (part 1 Linear Algebra,
part 2 Probability)

2. Bayesian decision theory
2.1 A review of probability theory
2.2 Classification
2.3 Loss and risk
2.6 Maximum likelihood estimation
2.7 Bayesian estimation
2.8 Parametric Classification

[Alp10] Chap
3, Chap
4,
Chap
5
[DHS00] Chap 3
Bias and Variance notebook 


3. Graphical models
3.1 Conditional independence
3.2 Naive Bayes classifier
3.3 Hidden Markov
2.5 Bayesian Networks
2.6 Belief propagation
2.7 Markov Random Fields

[Alp10] Chap 16
Markov Random Fields

Assignment 2
(data.txt) 
Video:
Embracing
uncertainty: the new machine intelligence

3. Kernel methods
3.1 The kernel trick
3.2 Kernel ridge regression
3.3 Kernel functions
3.4 Other kernel Algorithms
3.5 Kernels in complex structured data

[SC04] Chap 2
[Alp10] Chap 13
Introd. to kernel methods

Assignment 3 

4. Support vector learning
4.1 Support vector machines
4.2 Regularization and model complexity
4.3 Risk and empirical risk
4.4 SVM variations

[Alp10] Chap 13
An
introduction to ML, Smola
Support
Vector Machine Tutorial, Weston


Video:
Introduction to kernel methods (part 2) Presentations: [Lin02] Oct 22 Fabian Gómez, Erick Sánchez [Joachims09] Oct 22 Soleyda Manrique, Juan Pablo Contreras

5. Linear Models 5.1 Generalized linear models 5.2 The perceptron 5.3 Logistic regression 5.4 Batch and online learning 5.5 Stochastic gradient descent
 [Alp10] Chap 10 [Bis06] Chap 4   
6. Performance evaluation 6.1 Performance evaluation in supervised learning 6.2 Performance evaluation in unsupervised learning 6.3 Hypothesis testing

[Alp10] Chap 19
[TSK05] Chap
8 (Sect. 8.5)


Presentations: [Fawcett06] Oct 29 Oscar Montero, Germán Ramírez [Demsar06] Oct 29 Omar Prieto, Yenny Avendaño 
7. Unsupervised learning 7.1 Mixture densities 7.2 Expectation maximization 7.3 Mixture of latent variables models 7.4 Latent semantic analysis 7.5 Nonnegative matrix factorization

[Alp10] Chap 7
Latent Semantic Indexing, Prasad
Generative
Learning for BOF, Lazebnik
NMF for Multimodal Image Retrieval,
González


Presentations: [Ding08] Nov 5 John Cerón, Hugo Castellanos [Dhillon04] Nov 5 Jairo Jiménez, Mario Carrasco

8.Deep Learning 

Assignment 4 
Presentations: [Salakhutdinov09] Nov 12 Juan David Uchuvo, Elvert Mora 
9. Large Scale Machine Learning    Presentations: [Weston10] Nov 19 Sebastián Sierra, Miguel Chitiva [Le12] Nov 19 Andrés Escobar, Carlos Rivero

Grading
 Assignments 50%
 Presentation 20%
 Final project 30%
References
 [Alp10] Alpaydin, E. 2010 Introduction to Machine Learning, 2Ed.
The MIT Press.
 [Mur12] Murphy, Kevin P. Machine learning: a probabilistic perspective. The MIT Press, 2012.
 [Barber2013] Barber, David, Bayesian Reasoning and Machine Learning, Cambridge University Press, 2013.
 [Bis06] Bishop, C. 2006 Pattern Recognition and Machine Learning.
SpringerVerlag.
 [Mit97] Mitchell, T. M. 1997 Machine Learning. 1st.
McGrawHill
Higher Education.
 [DHS00] Duda, R. O., Hart, P. E., and Stork, D. G. 2000
Pattern
Classification (2nd Edition). WileyInterscience.
 [HTF01] Hastie, T. and Tibshirani, R. and Friedman. 2001
The
elements of
statistical learning: data mining, inference, and prediction. Springer.
 [SC04] ShaweTaylor, J. and Cristianini, N. 2004 Kernel
Methods
for Pattern Analysis. Cambridge University Press.
 [TSK05] PangNing Tan, Michael Steinbach, Vipin Kumar,
2005, Introduction to Data Mining, AddisonWesley.
 [CST00]
Cristianini, N. and ShaweTaylor, J., 2000, An introduction to support
Vector Machines: and other kernelbased learning methods,, Cambridge
Univ Press.
 [SS02] Scholkopf, B. and Smola, A.J., 2002, Learning with
kernels, MIT Press.
 [Bak07] Bakir, G. (Ed), 2007, Predicting Structured Data, MIT
Press.
 [OCWML] 6.867
Machine Learning, Fall 2006, MIT OpenCourseWare.
 [STANFDML] Andrew Ng, CS229 Machine
Learning, Stanford University
Resources
 SciPy: scientific, mathematical, and engineering package for Python
 scikitlearn: machine learning Scipy addon
 Kaggle: datascience competition, many interesting data sets and different competitions with prizes.
 Final project:
 Register at Yelp Dataset Challenge
 Obtain the data
 Think of an interesting problem that may be addressed using the data
 Write a proposal

