Syllabus

CSA202-Concepts of Machine Learning (3-0-2-4)

Unit 1-Core Concepts of Machine Learning

What kind of problems can be tackled using machine learning? The ML Mindset, Introduction to Machine Learning Problem Framing(Common ML Problems, ML Use Cases, Identifying Good Problems for ML, Hard ML Problems), Machine Learning Applications(Image Recognition, Speech Recognition, Medical Diagnosis, Statistical Arbitrage, Learning Associations), Standard learning tasks(Machine Learning Pipeline, Classification, Regression, Ranking, Clustering, Dimensionality reduction or Manifold learning)

Learning Stages (Features, Labels, Hyperparameters, Validation Samples, Test Samples, Loss Function, Hypothesis Tests), Learning Scenarios( Supervised learning, Unsupervised learning, Semi- Supervised learning, Transductive inference, On-line learning, Reinforcement learning, Active learning), Generalization, Supervised Learning, Unsupervised Learning, Reinforcement learning )

Data Preparation and Feature Engineering in ML(Data and Features, Information, Knowledge, Data Types, Big Data), Data Preprocessing: An Overview(Data Quality: Why Preprocess the Data?, Major Tasks in Data Preprocessing), Data Cleaning( Missing Values, Noisy Data, Data Cleaning as a Process), Data Integration(The Entity Identification Problem, Redundancy and Correlation Analysis, Tuple Duplication, Detection and Resolution of Data Value Conflicts), Data Reduction( Overview of Data Reduction Strategies, Attribute Subset Selection, Data Reduction, Histograms, Clustering, Sampling, Data Cube Aggregation), Data Transformation and Data Discretization(Overview of Data Transformation Strategies, Data Transformation by Normalization, Discretization by Binning, Discretization by Histogram Analysis, Discretization by Cluster, Decision Tree, and Correlation Analyses, Concept Hierarchy Generation for Nominal Data)

Unit 2-Supervised Learning Algorithms - Part One

How Supervised Learning Algorithms Work ?Steps (Bias-variance trade off, Function complexity and amount of training data, Dimensionality of the input space, Noise in the output values, Algorithms, Other factors to consider (Heterogeneity of the data, Redundancy in the data, Presence of interactions and non-linearities

Linear Regression Model Representation, Linear Regression Learning the Model ( Simple Linear Regression, Ordinary Least Squares, Gradient Descent), Regularization / Shrinkage Methods ( Bias-variance trade-off, Overfitting Issues, Lasso Regression, Ridge Regression), Making Predictions with Linear Regression( Cost Function, Feature Scaling, Normalization, Mean Normalization, Learning Rate, Automatic Convergence Test)

Logistic Regression, The Logistic Model ( Latent variable interpretation, Logistic function, odds, odds ratio, and logit, Definition of the logistic function, Definition of the inverse of the logistic function, Interpretation of these terms, Definition of the odds, The odds ratio, Multiple explanatory variables), Model fitting ("Rule of ten", Iteratively reweighted least squares (IRLS), Evaluating goodness of fit, Limitations of Logistic Regression), Linear discriminant analysis ( LDA for two classes, Assumptions, Discriminant functions, Discrimination rules, Eigenvalues, Effect size), Practical use and Applications ( Bankruptcy prediction, Face recognition, Marketing, Biomedical, studies), Comparison to Logistic Regression

Unit 3-Supervised Learning Algorithms - Part Two

Support Vector Machines, Linear SVM ( Hard-margin, Soft-margin), Nonlinear Classification, Computing the SVM classifier(Primal, Dual, Kernel trick), Modern methods(Sub-gradient descent, Coordinate descent), Empirical risk minimization(Risk minimization, Regularization and stability, SVM and the hinge loss, Target functions), Properties( Parameter selection, Issues)

Introduction to Artificial Neural Networks (Feed-forward Network Functions, Weight-space symmetries), Network Training ( Parameter optimization, Local quadratic approximation, Use of gradient information, Gradient descent optimization), Error Backpropagation( Evaluation of error-function derivatives, Simple examples, Efficiency of backpropagation)

Decision Tree Learning (Decision tree representation, ID3 learning algorithm, Entropy, Information gain, Overfitting and Evaluation, Overfitting, Validation Methods, Avoiding Overfitting in Decision Trees, Minimum-Description Length Methods, Noise in Data), Random Forests Algorithm ( Preliminaries: decision tree learning, Bagging, From bagging to random forests, Extra Trees, Properties, Variable importance)

Unit 4-Unsupervised Learning

Unsupervised Learning ( What is Unsupervised Learning?), Clustering Methods (Method Based on Euclidean Distance, Method Based on Probabilities, Hierarchical Clustering Methods, Method Based on Euclidean Distance )

k-means Clustering Algorithm ( Standard algorithm (naive k-means), Initialization methods), Applications (Vector quantization, Cluster analysis, Feature learning) Gaussian mixture models , Expectation-Maximization method

Principal Component Analysis for making predictive models ( First component, Further components, Covariances, Dimensionality reduction, Singular value decomposition), Properties and limitations of PCA ( Properties, Limitations), Computing PCA using the covariance method, Typical Applications

Unit 5-Parameter Estimation, Model Evaluation and Ensemble Methods

Parameter Estimation ( Point Estimation, Maximum Likelihood Estimation, Unbiased Estimation, Confidence Intervals for One Mean, Two Mean, Variances)

Model Evaluation ( ML Model Validation by Humans, Holdout Set Validation Method, Cross-Validation Method for Models, Leave-One-Out Cross-Validation, Random Subsampling Validation, Teach and Test Method, Bootstrapping ML Validation Method, Running AI Model Simulations, Overriding Mechanism Method ), The ROC Curve

Ensemble Methods ( Ensemble Theory, Ensemble Size, Voting and Averaging Based Ensemble Methods Boosting, Weightage Average, Stacking, Bagging, Boosting and Bootstrap Aggregating)

Text book/s*

1. Bishop, C. (2006). Pattern Recognition and Machine Learning. Berlin: Springer-Verlag.

2. Foundations of Machine Learning, Second Edition By Mehryar Mohri, Afshin Rostamizadeh and Ameet Talwalkar, MIT Press, Second Edition, 2018.

3. Introduction to Machine Learning, Third Edition, By Ethem Alpaydin, The MIT Pressmitpress.mit.edu › books › introduction-machine-learni...

Other References

1. Baldi, P. and Brunak, S. (2002). Bioinformatics: A Machine Learning Approach. Cambridge, MA: MIT Press.

2. Russel, S. and Norvig, P. (2003). Artifiical Intelligence: A Modern Approach. 2ndEdition. New York: Prentice-Hall.

3. Cohen, P.R. (1995) Empirical Methods in Artificial Intelligence. Cambridge, MA: MIT Press.

4. https://www.toptal.com/machine-learning/ensemble-methods-machine-learning.