• Assignment 1 

• Assignment 2 

• Assignment 3 

• Assignment 4

• Week 1: Intro and a brief review of some prerequisites. The definition of convergence in probability. The weak law of large numbers along with Markov's and Chebyshev's inequalities. Algebraic rules for convergence in probabilities of sums, products, and continuous functions.

• Week 2: Definition of consistency and examples of consistent estimators, such as the sample variance. The definition of convergence in distribution and what it entails relative to convergence in probability (convergence in probability implies convergence in distribution). Examples of convergence in distribution.  

• Week 3: Slutsky's theorem and the fact that convergence in distribution of sums and products is more subtle than for convergence in probability. Using moment generating functions to prove convergence in distribution. The central limit theorem. The delta method and how to use it in tandem with the CLT. Combining all of our knowledge about convergence to obtain confidence intervals that have the desired asymptotic coverage.  

• Week 4: Basics of solving optimization problems by first and second order conditions. Definition of a convex function. Regularity conditions for the MLE (or more subtly a sequence of solutions to the score equations) to be consistent. Definition of the KL divergence and the start of a discussion on its properties.   

• Week 5: More on KL divergence and how this relates to the MLE. The definition of the score function and the Fisher information. Quiz 1. 

• Week 6: Efficiency and the Cramer-Rao lower bound. Asymptotic efficiency and a proof that the MLE is asymptotically efficient. Comparison of the asymptotic variance of the MLE to other estimators, like the method of moments estimator. Overview of hypothesis testing problems and the ideas of power, size, type I and II errors. Definition of a pointwise and uniform asymptotically level-alpha test. Beginning of a dicussion on methods for testing a simple null hypothesis, including the likelihood ratio test. 

• Week 7: The likelihood ratio, Wald, and score tests for testing a simple, univariate null hypothesis. Proof of the chi-squared convergence of the corresponding test statistics. Sufficient statistics, the conditional distribution definition, the factorization criterion and the data emulation property. Example of using the factorization theorem.

• Week 8: Conditioning on a sufficient statistic as a way to improve an estimator. The Rao-Blackwell theorem and a derivation of the law of total expectation and law of total variance. Complete statistics and the uniqueness of unbiased estimators that are functions of complete statistics. The Lehmann-Scheffe theorem that construct a MVUE from a CSS and an unbiased estimator.  Example that unbiased estimators do not always exist. Definition of an exponential family of distributions and their associated natural parameter space, natural parameter, and sufficient statistic. Connections to Lehmann-Scheffe in that the sufficient statistics is complete. Behavior of exponential families under repeated sampling. How to compute the mean and variance of the sufficient statistic by differentiating the cumulant generating function. 

• Week 9:  Definition of the uniformly most powerful test. Proof of the Neyman-Pearson lemma, which asserts that the UMP test for simple null and alternative hypotheses is the likelihood ratio test. Extending the Neyman-Pearson lemma, first find UMP tests for composite, one-sided, alternative hypotheses and then to find UMP tests for composite, one-sided, null and alternative hypotheses. The property of having a monotone likelihood ratio (MLR) is introduced as a condition that allows us to find UMP tests for one-sided null hypotheses.  Exponential families have MLRs in their sufficient statistic, which allows us to construct UMP tests for one-sided hypotheses in the canonical parameter. 

• Week 10:  Quiz 2. The duality between collections of level-alpha hypothesis tests and confidence intervals with (1-alpha) coverage. A discussion on nonparametric statistics and the tradeoff of making too many vs too few assumptions. The definition of the median and sample median and some subtleties associated with them not always being unique. The sign test and a proof that it is level-alpha. 

• Week 11:  Reading week. 

• Week 12: 

• Week 13: 

• Week 14: 

Quiz 1 will be in class on October 2nd (50 minutes). Please bring your student ID to the exam. The quiz will test all of the material involving probabilistic convergence as well as some basic material on maximum likelihood estimation. It will include the content covered in class up to and including the class on September 25th. It will consist of long answer questions. You will be allowed one double-sided sheet of notes in the exam. I will provide relevant information and facts about probability distributions, so you do not need to memorize or transcribe their pdfs or pmfs, but you should know how to manipulate them. 

Practice problems for the quiz can be found here. These questions are of a similar difficulty to those you will find on the quiz. The quiz will have 5 questions. Solutions to the practice problems will not be provided; please come talk with me in office hours if you want to work through any of the problems.

I recommend that you do the practice problems, review the homework questions, review any seemingly relevant questions in the textbook, and review your notes. You are expected to be able to prove statements about convergence in distribution and probability. However, I will make sure that such proofs are not overly demanding for the limited time you have in an exam setting. 

Quiz 2 will be in class on November 2nd (50 minutes). The format of the quiz is the same as quiz 1. Please bring your student ID to the exam. The quiz will test all of the material covered in class up to and including the class on Wednesday October 28th. That is, the quiz will cover all material up to and including Section 14 of the course notes. It will consist of long answer questions. You will be allowed one double-sided sheet of notes in the exam. I will provide relevant information and facts about probability distributions, so you do not need to memorize or transcribe their pdfs or pmfs, but you should know how to manipulate them. 

Practice problems for the quiz can be found here. Note that there are some questions on UMP testing for composite hypotheses in the problems, and that while this is good practice, such questions will not show up on this quiz. These questions are of a similar difficulty to those you will find on the quiz. The quiz will have 5 questions. Solutions to the practice problems will not be provided; please come talk with me in office hours if you want to work through any of the problems. 

I recommend that you do the practice problems, review the homework questions, review any seemingly relevant questions in the textbook, and review your notes.

Course Description: Laws of large numbers, weak convergence, some asymptotic results, delta method, maximum likelihood estimation, testing, UMP tests, LR tests, nonparametric methods (sign test, rank test), robustness, statistics and their sensitivity properties, prior and posterior distributions, Bayesian inference, conjugate priors, Bayes estimators.

Prerequisites: STAT 266 or STAT 276.

Grading:

Grade breakdown

• 5 assignments for 20% of the total grade. The lowest assignment grade is dropped.

• 2 in class quizzes, each worth 22.5%.

• The final exam is worth 35%.

Assignments: All assignments are to be submitted on Canvas. You may scan handwritten solutions or write up solutions in LaTeX (preferred). If you choose to write up your solutions by hand please make sure that they are legible. You should merge all your files together into one readable document; separate files for each question will not be accepted. For coding questions please submit relevant code chunks and output as part of your solution, while also including your raw code in a separate file. Assignments are meant to be completed individually without the assistance from your peers or large language models.

Late policy: 25% is subtracted from the grade of a given assignment for every day that this assignment is late. Being late by one minute is equivalent to being late by a day. Assignments are due at 11:59 PM MST on the day indicated in the syllabus. For example, if an assignment is due on Wednesday and you hand it on on Friday at 1:00 PM you will obtain (Your Grade - 50%) on the assignment.  

Resources:

Textbook:

The suggested textbook for the course is Introduction to Mathematical Statistics, Eighth Edition, R. Hogg, J McKean, and A. Craig, Prentice Hall, 2019.

Course notes providing a sketch of what we covered in class can be found on the course website. 

Software: We will be periodically using R throughout this course. Coding portions of assignments should be done in R. 

Other resources: Another standard textbook on mathematical statistics at an upper-undergraduate level is Statistical Inference by Casella and Berger. There are many other good, but more advanced, textbooks on mathematical statistics that I am happy to point you towards if you are interested.

Class Time, Office Hours, and Contact Information:

Class time: Monday, Wednesday and Friday, 1:00-1:50 PM, BS M-141.

Office hours:  Monday 11:50-12:50 (before class) and Wednesday, 2:00-2:50 PM (after class), U Commons 4-233. 

My email is: mccorma2[AT]ualberta[DOT]ca

• Code for convergence in distribution and probability

• Code for MLEs and likelihood.

• Code for hypothesis testing 

• Code for Bayesian analysis