Sandeep Kumar - ELL 409 Machine Intelligence and Learning

ELL 409: Machine Intelliegnce and Learning

3 credits (3-0-0)Pre-requisites: Elementary Probability, and Matrix Theory.Semester II: 2024-2025
Evaluation: Quizzes (20 %), Assignments (20%), Minor Exam (20%) and Major Exam (40 %).

Course Objective: Machine Learning (ML) is the cornerstone of modern artificial intelligence, driving innovation across diverse fields such as healthcare, finance, natural language processing, and computer vision. This course offers a comprehensive exploration of foundational principles, advanced techniques, and practical applications in ML. By integrating theoretical rigor with hands-on implementation, the course equips students with the skills to develop robust ML models, critically evaluate their performance, and understand their limitations. This comprehensive course equips students with both theoretical knowledge and practical skills to excel in the rapidly evolving field of machine learning.

Learning Outcome: By the end of the course, students will understand the core principles of supervised, unsupervised, and semi-supervised learning, gaining hands-on experience with models for regression, classification, clustering, and dimensionality reduction. They will explore advanced topics like ensemble methods, neural networks, and graph-based ML, with the ability to evaluate models, manage trade-offs, and design domain-specific applications. Students will also grasp ethical considerations, including fairness, interpretability, and scalability, essential for deploying ML solutions effectively.

Tentative Lecture Plan

Introduction to Machine Learning
- Definition, scope, and real-world applications, Historical context and evolution, and Overview of course structure and goals.
ML Setups and Fundamental Concepts
- Data types, features, labels, and datasets; Empirical Risk Minimization (ERM), regularization, and loss functions; Gradient descent and optimization basics, Model Selection and Complexity; Overfitting and underfitting; Cross-validation and hyperparameter tuning.
Regression
- Linear regression and assumptions; Regularized regression (Ridge, Lasso); and Bias-variance tradeoff and evaluation metrics.
Classification
- Logistic regression, decision boundaries, and metrics and Bias-variance tradeoff in classification.
Decision Trees and Ensemble Learning
- Decision trees, random forests, boosting, and stacking.
Generalized Linear Models and Probabilistic Models
- Extensions to GLMs, Naive Bayes, and LDA.
Support Vector Machines (SVM) and Kernel methods
- Margin maximization, kernel tricks, and applications.
Instance-Based Learning

K-Nearest Neighbors (KNN): Algorithm, Variants, and Use Cases; Curse of Dimensionality: Problems and Mitigation Techniques

Unsupervised Learning
- PCA, clustering, GMM, and dimensionality reduction.
Visualization Techniques
- MDS, t-SNE, and UMAP for high-dimensional data.
Neural Networks
- Foundations, backpropagation, and the universal approximation theorem.
Deep Learning II (Bird's eye view)
- Core architectures: CNNs, RNNs, and LSTMS
- Advanced architectures: Transformers, VAEs, GANs, and diffusion models.
Graph-Based Machine Learning
- Foundations of sem-supervised learning, Basics, embeddings (Node2Vec, DeepWalk), and GNNs.
Ethics in ML
- Privacy, fairness, and interpretability.

Textbooks:

Murphy, K. P. (2022). Probabilistic Machine Learning: An Introduction. MIT Press.
Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press.
Alpaydin, E. (2020). Introduction to Machine Learning (4th ed.). MIT Press.
Hastie, Trevor. The elements of statistical learning: data mining, inference, and prediction." (2009).
Bishop, Christopher M., and Nasser M. Nasrabadi. Pattern recognition and machine learning. Vol. 4. No. 4. New York: springer, 2006.

Additional Readings:

Gilbert Strang, Linear Algebra and Learning from Data Wellesley-Cambridge Press, (2019).
John Hopcroft and Ravi Kannan, Foundations of Data Science, Hind Book Agency, TRIM Series, (2014).
Ekman, M. (2021). Learning deep learning: Theory and practice of neural networks, computer vision, natural language processing, and transformers using TensorFlow. Addison-Wesley Professional.