ELL 780: Mathematical Foundations for Computer Technology

ELL 457: Special Topics in Cognitive & Intelligent System

Instructors: Prof. Sandeep Kumar (SK) and Prof. Jayadeva

Course Objective: Fundamental mathematical topics like real analysis, linear algebra, statistics, probability, and optimization, are essential for data science, machine learning, artificial intelligence, and allied areas. These topics are generally taught in disparate courses, making it difficult for students to hone the skills in these areas. The aim of this course is to bring the foundational topics, mainly real analysis, matrix theory, and convex optimization to the fore and deliver the information in a self-contained manner. The course will start with a comprehensive introduction to the fundamental mathematical topics, which will be followed by discussions on models and algorithms. Numerous applications from the data science domain will be covered. This course will be particularly helpful to students from an engineering background, who seek to build their expertise in the aforementioned area.

Course Sketch:

• Analysis: Introduction to Metric Spaces, Vector Spaces, Normed Linear Spaces, and Inner Product Spaces. Dual Norm. Continuity, Smoothness, Lipscticz property, Differentiability, Sequence, and Convergence.

• Matrix Theory: System of Linear Equations, Linear Transformation, Eigenvalues, and Eigenvectors, Singular Value Decomposition (SVD), Pseudo Inverse. Positive (Semi)Definite Matrices and Associated Properties, Quadratic Forms and Least Squares, Low-Rank Approximation, Graph Matrices, and its Spectral Properties.

• Optimization Basics: Convex Set and Convex Function. Optimization Formulation, Convex Optimization, Different types of Optimization Problem Formulation (LP, QP, QCQP, SOCP). Optimality Condition, Lagrangian Duality, KKT Conditions, Introduction to First-Order Methods.

Textbooks:

1. W.Rudin, Principles of Mathematical Analysis Vol. 3. New York: McGraw-hill, 1976.
2. Gilbert Strang, Linear Algebra and Learning from Data, Wellesley-Cambridge Press, (2019.)
3. Stephen P. Boyd, and Lieven Vandenberghe, Convex Optimization, Cambridge university press, (2004).

Additional Readings:

1. S. Chandra, Jayadeva and Aparna Mehra, Numerical Optimizations with Applications, Narosa, (2014).
2. G. H. Golub and C. F. Van-Loan, Matrix Computations, John Hopkins Studies in Computational Science, (2013).
3. E.Kreyszig, Introductory Functional Analysis with Applications, Wiley(1978).
4. Charu C. Aggarwal, Linear Algebra and Optimization for Machine Learning, Springer, (2020).
5. John Hopcroft and Ravi Kannan, Foundations of Data Science, Hind Book Agency, TRIM Series, (2014).