Basics on probability theory. Measurable spaces. Probability spaces. Conditional probability.
Random variables. Properties of the distribution function. Discrete, absolute continuous and singular components of distribution functions. Change of distributions and densities under change of variables. Random vectors.
Independence of σ-algebras and random variables. Independence of random variables. Independence of events and σ-algebras. Some connections among independence, σ-algebras and random variables.
Integrals of random variables on probability spaces. Integrals of multivariate random variables. Independence, uncorrelation and Gaussianity. Characteristic functions.
Conditional distribution and expectation under a given event. Conditional distributions and densities. Zero-one laws and laws of large numbers. Central Limit Theorem.
Random samples extracted from a given distribution function. Statistics as a correlation process of data with parameters. Bayesian vs Deterministic approach.
The Deterministic approach. Estimators as statistics: bias and variance. A tradeoff between bias and minimum variance.
Fisher Information Matrix and Cramer-Rao inequality. Kullback-Leibler pseudo-distance between density functions and connections with the Fisher matrix, Parameter indistinguishability and identifiability.
Maximum Likelihood (ML) estimators. ML estimators with Gaussian random samples. The invariance principle for Maximum Likelihood estimators.
Weighted least squares (WLS). Linear statistical models.
Maximum Likelihood estimators for linear statistical models with Gaussian noise.
Linear estimators (BLUE) with minimal error covariance for the linear statistical model with non-Gaussian noise. Statistical models with vector valued data.
Recursive WLS estimators.
The Bayesian approach. Cost functions and conditional expected risk.
Minimum Variance (MEV) estimators.
MEV estimators for jointly Gaussian parameter and data vectors. Linear MEV estimators of random vectors given a collection of observations. Linear MEV estimators for the linear statistical model.
Geometric characterization of the linear MEV estimator. Orthogonality in L2 spaces and the Projection Theorem.
Comparisons of Bayesian MEV and WLS estimators. Regularized WLS problems.
The recursive linear MEV estimator for linear models: Kalman filter and predictor. The recursive MEV estimator for Gaussian linear models.
Steady-state Kalman predictor and filter. Stability and asymptotic optimality.
The Kalman filter and predictor for nonlinear models: the Extended Kalman Filter.