Post date: Feb 18, 2015 7:44:38 PM
The expectation-maximization (EM) algorithm is a variant of the maximum likelihood (ML) algorithm. The major difference between the EM and the ML algorithm is the introduction of a latent variable to the probability/likelihood function of the data/parameter via the law of total probability. Since this latent variable is unobserved, the likelihood function must be averaged out conditioned on the previous estimated parameter. Then EM algorithm proceeds by maximizing this (expected) likelihood function, i.e. perform an ML estimation. In other words, an EM algorithm is a sequence of ML algorithms.
If underflow occurs, then see [2,3] for the log-sum-exp trick.
[1] http://en.wikipedia.org/wiki/Expectation%E2%80%93maximization_algorithm
[2] https://hips.seas.harvard.edu/blog/2013/01/09/computing-log-sum-exp/
[3] http://math.stackexchange.com/questions/648514/preventing-underflow-log-sum-exp-trick