Sandeep Kumar - ELL 780 MF for CTech 2024 Fall

ELL 780: Mathematical Foundations for Computer Technology

3 credits (3-0-0)Pre-requisites: Elementary Calculus and Matrix Theory.Semester I: 2024-2025
Evaluation: Quizzes (35 %), Minor Exam (25%) and Major Exam (40 %).

Course Objective: Fundamental mathematical topics like analysis, linear algebra, statistics, probability, and optimization, are essential for data science, machine learning, artificial intelligence, and allied areas. These topics are generally taught in disparate courses, making it difficult for students to hone the skills in these areas. The aim of this course is to bring the foundational topics, mainly real analysis, matrix theory, and convex optimization to the fore and deliver the information in a self-contained manner. The course will start with a comprehensive introduction to the fundamental mathematical topics, which will be followed by discussions on models and algorithms. Numerous applications from the data science domain will be covered. This course will be particularly helpful to students from an engineering background, who seek to build their expertise in the aforementioned area.

Course Sketch:

Matrix Theory: System of Linear Equations, Linear Transformation, Eigenvalues, and Eigenvectors, Singular Value Decomposition (SVD), Pseudo Inverse. Positive (Semi)Definite Matrices and Associated Properties, Quadratic Forms and Least Squares, Low-Rank Approximation, Graph Matrices, and its Spectral Properties.
Probability and Real Analysis: Real Analysis Basics, Random variables, Probability Distributions, Expectations, and Basics of Inference
Optimization Basics: Convex Set and Convex Function. Optimization Formulation, Convex Optimization, Different types of Optimization Problem Formulation (LP, QP, QCQP, SOCP). Optimality Condition, Lagrangian Duality, KKT Conditions, Introduction to First-Order Methods.

Textbooks:

Gilbert Strang, Linear Algebra and Learning from Data Wellesley-Cambridge Press, (2019.)
Axler, Sheldon. Linear algebra done right. springer, 4th Edition 2023..
Stanley H. Han, Introduction to Probability for Data Science Michigan Publication (2021).
Stephen P. Boyd and Lieven Vandenberghe, Convex Optimization, Cambridge university press, (2004).

Additional Readings:

G. H. Golub and C. F. Van-Loan, Matrix Computations, John Hopkins Studies in Computational Science, (2013).
W.Rudin, Principles of Mathematical Analysis Vol. 3. New York: McGraw-hill, 1976.
Carlos Fernandez-Granda, Probability and Statistics for Data Science 2017.
Charu C. Aggarwal, Linear Algebra and Optimization for Machine Learning, Springer, (2020).
John Hopcroft and Ravi Kannan, Foundations of Data Science, Hind Book Agency, TRIM Series, (2014).