(Old Version)

STAT 4003

Statistical Inference (2019 Spring)

Class Information

    • Class time: W 0830-1115
    • Location: LSB LT6
    • Outline

Instructor

    • Name: Kin Wai Chan
    • Email: kinwaichan@cuhk.edu.hk
    • Office: LSB 115
    • Tel: 3943 7923
    • Office hour: I have an open door policy. Feel free to drop by anytime and ask me questions. (Of course, you may also make an appointment with me if you want a long meeting.)

Teaching Assistants

  1. Jiasheng Shi
  2. Xiaoxiao Zhou

Description

This course provides an introduction to statistical inference. Topics include statistical models, sampling distributions, asymptotic distributions, sufficiency, maximum likelihood estimation, Bayesian estimation, Rao-Blackwell theorem, Cramér-Rao theorem and the best unbiased estimator, Neyman-Pearson lemma, uniformly most powerful test and general likelihood ratio test.

Textbooks

A self-contained lecture note is the main source of reference. Complementary textbooks include

Learning outcomes

Upon finishing the course, students are expected to

    1. apply basic statistics limit theorems and probability inequalities;
    2. perform data reduction through various kind of statistical principles;
    3. derive point estimation, interval estimation and testing procedures, and study their properties;
    4. analyze and evaluate statistical procedures by different criteria, and understand the interpretations of those criteria.

Assessment and Grading

The final score (out of 100) is given by {40a+20max(m,f)+40f}/100, where

    • a (out of 100) is the average score of approximately eight assignments with the lowest two scores dropped;
    • m (out of 100) is the score of mid-term exam; and
    • f (out of 100) is the score of final exam.

Syllabus

    1. Probability theory
      • (a) random variables, probability distribution, sampling distribution, statistics, exponential family
      • (b) stochastic convergence, probability inequalities, statistics limit theorems
      • (c) cumulant generating functions
    2. Data reduction
      • (a) sufficiency principle
      • (b) likelihood principle
    3. Point estimation
      • (a) bias, variance, mean squared error, consistency
      • (b) unbiased estimators, maximum likelihood estimators, minimax estimators, Bayes estimators, minimax estimators
      • (c) (special topics) model misspecification, nuisance parameters, robustness, missing data, non-parametric estimation
    4. Hypothesis testing
      • (a) power, size, types I/II errors, p-value
      • (b) most powerful tests, Wald tests, Rao tests, likelihood ratio tests, permutation tests, goodness-of-fit tests
      • (c) (special topics) false discovery rate, meta analysis,
    5. Interval estimation
      • (a) coverage probability, expected width
      • (b) duality, pivotal quantity, confidence intervals, credible intervals
    6. Discussion topics
      • (a) comparison of Bayesian, frequentist, and fiducial (BFF) inference
      • (b) causal inference

Lecture Notes (Draft)

Contents and Instructions

Part 1: Basic Probability and Statistics

Part II: Data Reduction

Part III: Point Estimation

Part IV: Hypothesis Testing

Part V: Interval Estimation

Appendix

Assignments

    1. Assignment 1: Stochastic convergence & statistical models (Due: 25 Jan (Fri) @ 6:00pm) Solution
    2. Assignment 2: Sufficiency principle (Due: 15 Feb (Fri) @ 6:00pm) Solution
    3. Assignment 3: Quality of point estimators & MoM Estimators (Due: 27 Feb (Wed) @ 6:00pm) Solution
    4. Assignment 4: UMVUE (Due: 20 Mar (Wed) @ 6:00pm) Solution
    5. Assignment 5: MLE (Due: 29 Mar (Fri) @ 6:00pm) Solution
        • (Click here) Simulation demonstration and incomplete R-code for Exercise 5.2 (e)
        • (Click here) Sample output of the simulation demonstration
    6. Assignment 6: UMPT (Due: 17 April (Wed) @ 6:00pm) Solution
    7. Assignment 7: LRT & asymptotic tests (Due: 26 April (Fri) @ 6:00pm) Solution
    8. Assignment 8: Confidence intervals (** Optional ** No need to hand in) Solution

Quizzes

Mid-term Exam

    • Date: 6 March (Wednesday)
    • Time: ~ 9:00 am (A short review will be given before the mid-term exam).
    • Duration: 80 minutes
    • Scope: Chapter 1 - Chapter 7 page 15 (i.e., everything until UMVUE)
    • Remarks:
      1. One piece of double-sided A4-size hand-written cheat sheet is allowed.
      2. No calculator is allowed.
      3. No make-up mid-term exam (except for those earlier approved cases) will be given.
    • Mock mid-term exam and Solution
    • Mid-term exam and Solution

Final Exam

    • Date: 2 May (Thursday)
    • Time: 1530 - 1730
    • Duration: 120 minutes
    • Location: University Gymnasium
    • Scope: Chapter 1 - Chapter 11 (i.e., everything taught in this course)
    • Remarks:
      1. One piece of double-sided A4-size hand-written cheat sheet is allowed.
      2. No calculator is allowed.
    • Mock final exam and Solution (Typos: Question 1.3-1.4, Question 3 (5))
    • Final exam and Solution

Review Lecture

      • Date: 24 April (Wednesday)
      • Time: 1030 - 1230
      • Location: LSB LT6
      • Remarks:
        1. I will go through some examples in this lecture.
        2. If time allows, some extra materials will also be taught.
Example 1.12: Illustration of finding quantiles. (Click here)
Section 4.6: Graphical illustration of minimal sufficiency. (Click here)
Example 7.9: Tradeoff between efficiency & robustness. (Click here)