Advanced Probability, Linear Algebra and Optimization Techniques (FIC 504)
August-December 2025
10:00-11:00 (Monday) and 11:00-13:00 (Friday)
August-December 2025
10:00-11:00 (Monday) and 11:00-13:00 (Friday)
The objective of this course is to introduce the mathematical tools necessary for developing new algorithms in cybersecurity/ machine learning.
Textbooks:
Mathematics for Machine Learning, Marc Peter Deisenroth, A. Aldo Faisal, Cheng Soon Ong.
Sheldon Ross, A First Course in Probability, 8th Edition, Pearson, 2006.
Kolman and Hill, Elementary Linear Algebra with Applications, 9th Edition, Pearson, 2008.
Reference Books:
J. Medhi, Stochastic Processes, 3rd Edition, New Age International, 2009.
S. M. Ross, Stochastic Processes, 2nd Edition, Wiley, 1996.
Stephen H Friedberg, Arnold J Insel, Lawrence E. Spence, Linear Algebra. 4th Edition, Pearson, 2006.
Topics covered:
Lecture 1: Began with some standard set-theory notations. Introduced basic notions of probability theory (for example, random experiments, sample spaces and events, etc.) Studied and explained the basic axioms of probability theory.
Suggested reading: Sheldon Ross, sections 2.1, 2.2, 2.3, 2.4, 2.5.
Lecture notes: L-1 basic notions of probability theory.pdf
Lecture 2: Studied one of the most fundamental concepts in probability theory, which is called conditional probability. For a given event A of a random experiment with additional condition B, how should we obtain the probability P(A|B) of A given B from the prior probability P(A)?
Suggested reading: Sheldon Ross, sections 3.1, 3.2, 3.3.
Lecture notes: L-2 conditional probability.pdf
Lecture 3: Independent events, total probability laws, and Bayes' Theorem.
Suggested reading: Sheldon Ross, sections 3.3, 3.4, 3.5.
Lecture notes: L-3 independent events.pdf
Homework-1 HW-1.pdf
Lecture 4: To analyse random experiments, we usually focus on some numerical aspects of the experiments; these numerical values are called "random variables". Studied discrete and continuous random variables. Learn the probability mass functions of a random variable.
Suggested reading: Sheldon Ross, sections 4.1, 4.2.
Lecture notes: L-4 random variable.pdf
Lecture 5: Introduced the probability distributions and further studied special distributions. For example, (1) X~Bernoulli(p), X~Geometric(p) and X~Binomial(n, p).
Suggested reading: Sheldon Ross, sections 4.6
Lecture notes: L-5 special distributions.pdf
Lecture 6: Introduced the cumulative distribution function (CDF). Although the PDF is one way to describe the distribution of a discrete random variable, it can not be defined for continuous random variables. Additionally, the CDF of a random variable is another method for describing the distribution of random variables. The advantage of the CDF is that it can be defined for any random variable (discrete, continuous and mixed).
Lecture note:s L-6 cumulative distribution function.pdf
Lecture 7: Expectation of discrete variables, linearity of expectations
Suggested reading: Sheldon Ross, Sections 4.3
Lecture notes: L-7 expectations.pdf
Lecture 8: Functions of discrete random variables, Law of the Unconscious Statistician (LOTUS) for discrete random variables and variance.
Suggested reading: Sheldon Ross, Sections 4.4, 4.5
Lecture notes: L-8 LOTUS-variance of discrete random variables.pdf
Homework-2 HW-2.pdf
Lecture 9: Continuous random variables, Probability density function (PDF). Gamma function. Expectation of continuous random variables, linearity of expectations.
Suggested reading: Sheldon Ross, Sections 5.1, 5.2, 5.3.
Lecture notes: L-9 PDF of continuous random variables.pdf
Lecture 10: Functions of continuous random variables, Method of transformations
Suggested reading: Sheldon Ross, Sections 5.2, 5.3.
Lecture notes: L-10 functions of crv.pdf
Lecture 11: Uniform distribution, Exponential distribution.
Suggested reading: Sheldon Ross, Sections 5.3, 5.5.
Lecture notes: L-11 uniform and exp distributions.pdf
Homework-3 HW-3.pdf
Lecture 12: Normal distribution.
Suggested reading: Sheldon Ross, Sections 5.4.
Lecture notes: L-12 normal distributions.pdf
Lecture 13: Fixed random variables
Lecture notes: L-13 mixed rv.pdf
Homework-4 HW-4.pdf
Lecture 14: Joint probability mass functions and joint cumulative distribution functions
Suggested reading: Sheldon Ross, Sections 6.1, 6.2
Lecture notes: L-14 joint pmf of drv.pdf
Lecture 15: Joint probability density functions and their joint cumulative distribution functions
Suggested reading: Sheldon Ross, Sections 6.2, 6.3
Lecture notes: L-15 joint pdf of crv.pdf
Homework-5 HW-5.pdf
Lecture 16: Conditional and independence in two continuous random variables.
Suggested reading: Sheldon Ross, Sections 6.4
Lecture notes: L-16 conditional of joint CRV.pdf
Lecture 17: Expectation of the jointly continuous random variables X and Y. Condition LOTUS and conditional variance of X given Y.
Suggested reading: Sheldon Ross, Sections 6.7
Lecture 18: Covariance and correlations.
Suggested reading: Sheldon Ross, Sections 7.1, 7.2, 7.4
Homework-6 HW-6.pdf
Lecture 19: Bivariate normal distribution,
Homework-7 HW-7.pdf
Lecture 20: Limit Theorems and convergence of random variables
Suggested reading: Sheldon Ross, Sections 8.1, 82
Lecture 21: Central Limit Theorem
Suggested reading: Sheldon Ross, Sections 8.3, 8.4
Homework-8 HW-8.pdf
Lecture 22: Rank of matrices by using Echelon form, Gauss elimination method.
Suggested reading: Kolman and Hill, Sections 2.1, 2.2
Lecture 23: Vector spaces and linear dependence.
Suggested reading: Kolman and Hill, Sections 4.2, 4.3, 4.4, 4.5, 4.6.
Homework-9 HW-9.pdf
Lecture 24: Linear transformations and the standard matrix corresponding to a linear transformation, null spaces and rank spaces, rank-nullity theorem.
Suggested reading: Kolman and Hill, Sections 6.1, 6.2, 6.3
Homework-10 Kolman and Hill, Section 6.2: 4, 5, 10, 12. Section 6.3: 1, 2, 5.
Lecture 25: Eigenvalues and corresponding eigenvectors of a linear map. Diagonalizability of matrices.
Suggested reading: Kolman and Hill, Sections 7.1, 7.2
Homework-11 Kolman and Hill, Section 7.1: 6, 7, 9, 10, 12.
Lecture 26: Diagonalizability of linear maps. LU decomposition of matrices.
Suggested reading: Kolman and Hill, Sections 7.2, 7.3
Homework-12 Kolman and Hill, Section 7.2, problems 6 and 7.
Lecture 27: Inner product spaces, Gram–Schmidt process, and the QR decomposition of matrices.
Suggested reading: Kolman and Hill, Sections 5.3, 5.4.
Lecture 28: History of optimization problems and study of the basic notions of optimization problems.
Lecture 29: Nonlinear programming problems, Lagrange multiplier
Homework-13 HW-13.pdf,
Solution Homework HW-13 solution.pdf
Lecture 30: Linear regression