Model Based Signal Analysis

• The objective of this course is to introduce the students to the problem of the “Parameter Estimation” specifically under known signal/data models.

• The second objective is to train the students to do performance analysis of the estimators through monte-carlo simulations

Upon completion of the course, the student should be able to

• the student should be able to formulate a statistical parameter estimation problem from a given statistical model (Unit 1)

• the student should be able to derive the Cramer-rao bound on the performance of the estimator for a given probabilistic model

• the student should be able to identify if the sufficient statistics exists and utilize them for deriving the estimator

• the student should be able to derive and implement a time-domain estimator based on any of the existing classical and the Bayesian methods and analyze its performance (Units 4 and 5)

• the student should be able to derive a spectral estimator and apply the same to extract the spectra from random signals and analyze the performance

1. Introduction (5 hrs)

   1. Mathematical estimation problem, estimator performance, Minimum variance unbiased estimation (MVUE), minimum variance criterion Existence of MVUE, Finding MVUE, Extend to Vector parameters

2. Bounds on signal estimation : ( 6 hrs)

   1. Accuracy considerations, Cramer Rao Bound (CRLB), General CRLB, Transformation of parameters, Extension to Vector parameter case, General Gaussian case, asymptotic CRB

3. Linear Models and Sufficient Statistics (5 hrs)

   1. Definition, Examples, Sufficient Statistics, finding SS, Using sufficiency for finding MVUE, Best Linear Unbiased Estimator, finding BLUE

4. Classical Approaches (10 hrs)

   1. Maximum likelihood estimation, example, finding MLE, properties, transformed parameters, Numerical methods, Asymptotic MLE

   2. Least Squares: LS Approach, Linear LS, Order recursive LS, Sequential LS, Non-linear LS.

5. Bayesian Approach (5 hrs)

   1. Introduction, Prior knowledge, Choosing the Prior, Properties of Gaussian pdf, Bayesian linear model, deterministic parameters, Minimum Mean Square Error Estimation (2 Weeks)

6. Spectral Analysis:

   1. Basics, Power spectral density, Properties of PSD, Spectral Estimation problem, Periodogram, Correlogram, Rational spectra, Covariance structure, ARMA models, AR models, Yule Walker method

Texts:

1. Fundamentals of statistical signal processing I: Estimation Theory, Steven M. Kay, Prentice Hall

2. Spectral Analysis of Signals by Peter Stoica and Randolph Moses, PHI publications.

Reference Materials

1. Statistical Signal Processing: Detection, Estimation, and Time Series Analysis, Louis Scharf, Addison Wesley.

2. Bayesian Inference in statistical analysis, Box and Tiao, Wiley

3. Model based Signal Processing, James Candy, Wiley. 704pp, 2005

Computer Assignments

Project

Scheduled Quizzes

In-class exams