Maximum Likelihood Estimation
General Notes
When teaching, provide theory (derive for normal distribution) and assumptions.
Caution:
Just because a ML estimator was a reasonable fit for sample data, there is no guarantee that the population has that distribution or property.
The R script "likelihood.R.txt" (attached below) provides a demonstration of the use of MLE methods in various situations.
There is also a function from the "TeachingDemos" package for visually demonstrating MLE.
Finally, the attached file "likelihood graphic demo.R.txt" includes my own preliminary attempt to graphically show how MLE works.
Example 1
known standard deviation
find mu by MLE
Example 2
known mu
find standard deviation by MLE
Example 3
known sigma
find mu by MLE
Likelihood allowing for right-censoring
Example 4
known mean
find sigma by MLE
Likelihood allowing for right-censoring
Example 5
Data from lognormally distributed population
known scale
find mu by MLE - but mistakenly assume normal dist
Example 6
Data from lognormally distributed population
known scale
find mu by MLE - but correctly assume lognormally dist
Example 7
Data from normally distributed population
unknown mean and standard deviation
find mu and stddev by MLE using matrix of results
Example 8
Data from exponentially distributed population
known lambda (aka rate)
find lambda by MLE
Example 9
Data from exponentially distributed population
known lambda (aka rate=15)
find scale of Weibull by MLE (should equal 1)
with Weibull scale parameter equal to 1/lambda
Example 10
Data from exponentially distributed population
known lambda (aka rate)
set Shape of Weibull to equal 1
find Weibull Scale parameter (should be equal to exp lambda)
Example 11
Data from exponentially distributed population
known lambda (aka rate)
Take random sample and estimate lambda by MLE
Demonstrate that the MLE is asymptotically normal
-----------------------------------------------------------------------------------------------
R Code from the Teaching Demos package
library(TeachingDemos)
?mle.demo
if(interactive()){
mle.demo()
m <- runif(1, 50,100)
s <- runif(1, 1, 10)
x <- rnorm(15, m, s)
mm <- mean(x)
ss <- sqrt(var(x))
ss2 <- sqrt(var(x)*11/12)
mle.demo(x)
# now find the mle from the graph and compare it to mm, ss, ss2, m, and s
}
-----------------------------------------------------------------------------------------------
Assumptions
One might use the method of maximum likelihood to decide which distribution (of several) best fits a set of data. However, there is no guarantee that the best ML fit distribution is the same as the distribution of the population.
References