Maximum Likelihood Estimation

General Notes

When teaching, provide theory (derive for normal distribution) and assumptions.

Caution:

Just because a ML estimator was a reasonable fit for sample data, there is no guarantee that the population has that distribution or property.

The R script "likelihood.R.txt" (attached below) provides a demonstration of the use of MLE methods in various situations.

There is also a function from the "TeachingDemos" package for visually demonstrating MLE.

Finally, the attached file "likelihood graphic demo.R.txt" includes my own preliminary attempt to graphically show how MLE works.

Example 1

known standard deviation

find mu by MLE

Example 2

known mu

find standard deviation by MLE

Example 3

known sigma

find mu by MLE

Likelihood allowing for right-censoring

Example 4

known mean

find sigma by MLE

Likelihood allowing for right-censoring

Example 5

Data from lognormally distributed population

known scale

find mu by MLE - but mistakenly assume normal dist

Example 6

Data from lognormally distributed population

known scale

find mu by MLE - but correctly assume lognormally dist

Example 7

Data from normally distributed population

unknown mean and standard deviation

find mu and stddev by MLE using matrix of results

Example 8

Data from exponentially distributed population

known lambda (aka rate)

find lambda by MLE

Example 9

Data from exponentially distributed population

known lambda (aka rate=15)

find scale of Weibull by MLE (should equal 1)

with Weibull scale parameter equal to 1/lambda

Example 10

Data from exponentially distributed population

known lambda (aka rate)

set Shape of Weibull to equal 1

find Weibull Scale parameter (should be equal to exp lambda)

Example 11

Data from exponentially distributed population

known lambda (aka rate)

Take random sample and estimate lambda by MLE

Demonstrate that the MLE is asymptotically normal

-----------------------------------------------------------------------------------------------

R Code from the Teaching Demos package

library(TeachingDemos)

?mle.demo

if(interactive()){

mle.demo()

m <- runif(1, 50,100)

s <- runif(1, 1, 10)

x <- rnorm(15, m, s)

mm <- mean(x)

ss <- sqrt(var(x))

ss2 <- sqrt(var(x)*11/12)

mle.demo(x)

# now find the mle from the graph and compare it to mm, ss, ss2, m, and s

}

-----------------------------------------------------------------------------------------------

Assumptions

One might use the method of maximum likelihood to decide which distribution (of several) best fits a set of data. However, there is no guarantee that the best ML fit distribution is the same as the distribution of the population.

References