The final exam will be held in NRE 2-003 on December 15th, starting at 8:30 AM (3 hours). Please bring your student ID to the exam. The exam is cumulative, but a greater emphasis will be placed on material that was not covered on quizzes 1 and 2. It will consist of long answer questions and will be not quite twice as long as the quizzes. You will be allowed two double-sided sheets of notes in the exam. I will provide relevant information and facts about probability distributions, so you do not need to memorize or transcribe their pdfs or pmfs, but you should know how to manipulate them. 

Some practice problems for the final can be found here . These questions will be updated periodically. Solutions to the practice problems will not be provided; please come talk with me in office hours if you want to work through any of the problems.

In preparation for the final I recommend that you do the practice problems (including the previous practice problems), review the simpler homework questions, review the course notes, and look over the relevant sections of the textbook. 

• Assignment 1 

• Assignment 2  

• Assignment 3 

• Assignment 4  

• Assignment 5  

• Week 1: Intro and a brief review of some prerequisites. The definition of convergence in probability and consistency of an estimator. The weak law of large numbers along with Markov's and Chebyshev's inequalities. 

• Week 2: Rules for convergence in probability of sums, products, and continuous maps. Examples of consistent estimators, such as the sample variance. The definition of convergence in distribution and what it entails relative to convergence in probability (convergence in probability implies convergence in distribution). Examples of convergence in distribution. 

• Week 3: Using moment generating functions to prove convergence in distribution. The central limit theorem. The delta method and how to use it in tandem with the CLT. Combining all of our knowledge about convergence to obtain confidence intervals that have the desired asymptotic coverage. 

• Week 4: Basics of solving optimization problems. Definition of a convex function. Consistency of the MLE. Parameterization invariance of the MLE.  

• Week 5: The score function, the Fisher information function, and properties of both. Cramer-Rao lower bound. Efficiency, asymptotic efficiency, and asymptotic relative efficiency. Minimum variance unbiased (MVUE) estimators.

• Week 6:  Examples of relative efficiency. General hypothesis testing framework, including a discussion on test functions, Type I and II errors, the power function, and asymptotic level-alpha tests. The likelihood ratio, Wald, and score tests for testing a simple, univariate null hypothesis. Proof of the chi-squared convergence of the corresponding test statistics.  

• Week 7: Sufficient statistics, the conditional distribution definition, the factorization criterion and the data emulation property. Example of using the factorization theorem. The Rao-Blackwell theorem. Complete statistics and the uniqueness of unbiased estimators that are functions of complete statistics. Statement of the Lehmann-Scheffe theorem. 

• Week 8: Example that unbiased estimators do not always exist. Definition of an exponential family of distributions and their associated natural parameter space, natural parameter, and sufficient statistic. Connections to Lehmann-Scheffe in that the sufficient statistics is complete. Behavior of exponential families under repeated sampling. How to compute the mean and variance of the sufficient statistic by differentiating the cumulant generating function. Return to some hypothesis testing theory where we recall the notions of test function, rejection region, and power. Definition of the uniformly most powerful test. Proof of the Neyman-Pearson lemma, which asserts that the UMP test for simple null and alternative hypotheses is the likelihood ratio test. 

• Week 9: Extending the Neyman-Pearson lemma, first find UMP tests for composite, one-sided, alternative hypotheses and then to find UMP tests for composite, one-sided, null and alternative hypotheses. The property of having a monotone likelihood ratio (MLR) is introduced as a condition that allows us to find UMP tests for one-sided null hypotheses.  

• Week 10: Examples of finding MLRs and how to determine the resulting UMP test for one-sided hypotheses. The duality between (collections of) hypothesis tests for point null hypotheses and confidence intervals. A brief discussion of what it means for a model to be nonparametric. The sign test for testing medians and an examination of its power function. 

• Week 11: Reading week.

• Week 12: The signed-rank test for testing medians under the assumption of symmetry of a distribution about its median. The null distribution of the signed-rank test and how to compute the quantiles by hand when the sample size n is small. The notion of a statistical functional. Empirical CDFs and the empirical distribution. The plug-in estimator associated with a statistical functional. Robustness considerations in statistics as largely the study of mitigating the effect of outliers or corruption in data. The sensitivity curve and the breakdown point of an estimator. A comparison of the behavior of the mean vs the median in terms of their breakdown points.  

• Week 13: More examples on breakdown points for trimmed means and the Hodges-Lehmann estimator. The optimization problems for means and medians and how Huber's loss function interpolates between the squared error loss and absolute loss. The definition of mixture models. The definition of the influence function and robustness of a statistical functional. Computation of the influence function for means and medians. Introduction to Bayesian vs Frequentist statistics: subjective and frequentist interpretations of probability. The likelihood, prior, marginal, and posterior distributions. 

• Week 14: How to compute posterior distributions by ignoring normalizing constants – examples for the beta and normal distributions and how their parameters nicely update when going from the prior to posterior. The notion of a conjugate prior, and how this works for exponential family likelihoods. Bayesian point estimation via taking the mean, median, or mode of the posterior distribution. How to find the MAP (posterior mode) estimate and its links to maximum likelihood estimation. The definition of a credible interval and how they do not necessarily provide (1 - alpha) frequentist coverage of the true parameter. Highest posterior density (HPD) and equi-tailed credible intervals. A introduction to Bayesian hypothesis testing: simply compute the probability of the null and alternative hypotheses using the posterior distribution. 

• Last class: More on Bayesian hypothesis testing. How to test point null hypotheses by using a prior that is a mixture distribution. 

• Code for convergence in distribution and probability

• Code for MLEs and likelihood.

• Code for hypothesis testing 

• Code for Bayesian analysis 

Quiz 1 will be in class on September 29th (50 minutes). Please bring your student ID to the exam. The quiz will test all of the material involving probabilistic convergence as well as some basic material on maximum likelihood estimation (material up to the class on Wednesday, September 24th). It will consist of long answer questions. You will be allowed one double-sided sheet of notes in the exam. I will provide relevant information and facts about probability distributions, so you do not need to memorize or transcribe their pdfs or pmfs, but you should know how to manipulate them. 

Practice problems for the quiz can be found here. These questions are either a little harder or of a similar difficulty to those you will find on the quiz. The quiz will have 5 questions. Solutions to the practice problems will not be provided; please come talk with me in office hours if you want to work through any of the problems.

I recommend that you do the practice problems, review the homework questions, review the questions in the textbook, and review your notes. You are expected to be able to prove statements about convergence in distribution and probability. However, I will make sure that such proofs are not as demanding as some of the proofs we have done in class. 

Quiz 2 will be in class on November 3rd (50 minutes). Please bring your student ID to the exam. The quiz will test all of the material covered up to the class on Wednesday, October 29th. It will consist of long answer questions. You will be allowed one double-sided sheet of notes in the exam. I will provide relevant information and facts about probability distributions, so you do not need to memorize or transcribe their pdfs or pmfs, but you should know how to manipulate them. 

Some initial practice problems for the quiz can be found here.  These questions will be updated periodically. The quiz will have 5 questions. Solutions to the practice problems will not be provided; please come talk with me in office hours if you want to work through any of the problems.

I recommend that you do the practice problems, review the homework questions, review the questions in the textbook, and review your notes. 

Course Description: Laws of large numbers, weak convergence, some asymptotic results, delta method, maximum likelihood estimation, testing, UMP tests, LR tests, nonparametric methods (sign test, rank test), robustness, statistics and their sensitivity properties, prior and posterior distributions, Bayesian inference, conjugate priors, Bayes estimators.

Prerequisites: STAT 266 or STAT 276.

Grading:

Grade breakdown

• 5 assignments for 35% of the total grade. The lowest assignment grade is dropped.

• 2 quizzes, each worth 15%.

• The final exam is worth 35%.

Assignments: All assignments are to be submitted on Canvas. You may scan handwritten solutions or write up solutions in LaTeX (preferred). If you choose to write up your solutions by hand please make sure that they are legible. For coding questions please submit relevant code chunks and output as part of your solution, while also including your raw code in a separate file. Assignments are meant to be completed individually without the assistance from your peers or large language models.

Late policy: 25% is subtracted from the grade of a given assignment for every day that this assignment is late. Being late by one minute is equivalent to being late by a day. Assignments are due at 11:59 PM MST on the day indicated in the syllabus. For example, if an assignment is due on Wednesday and you hand it on on Friday at 1:00 PM you will obtain (Your Grade - 50%) on the assignment.  

Resources:

Textbook:

The required textbook for the course is Introduction to Mathematical Statistics, Eighth Edition, R. Hogg, J McKean, and A. Craig, Prentice Hall, 2019.

We will be following this book closely so make sure you have access to a copy. 

Other resources will be posted on the course website throughout the semester.

Software: We will be periodically using R throughout this course. Coding portions of assignments should be done in R. 

Other resources: Another standard textbook on mathematical statistics at an upper-undergraduate level is Statistical Inference by Casella and Berger. There are many other good, but more advanced, textbooks on mathematical statistics that I am happy to point you towards if you are interested.

Class Time, Office Hours, and Contact Information:

Class time: Monday, Wednesday and Friday, 1:00-1:50 PM, BS M-141.

Office hours:  Monday and Friday, 2:00-2:50 PM, U Commons 4-233 (or possibly in the classroom if it is available). 

My email is: mccorma2[AT]ualberta[DOT]ca