Finding the Q function for the EM algorithm

Question

I have a situation where $X_1,...X_n$ come from $N(\mu,1)$ and there is a realization of 10 $x$ values. I want to use the EM algorithm to work out the MLE.

So, I am trying to compute the expected conditional log-likelihood (the Q function). This is given by

$$Q(\mu,\mu_n)=E[log(f(X|\mu)|\mu_n)]$$

The log-likelihood for the normal distribution with variance = 1 is given by

$$\frac{-n}{2}ln(2\pi) - \frac{1}{2}\sum(x_i-\mu)$$

Hence, plugging this into the equation above we get

$$Q(\mu,\mu_n) = \frac{-n}{2}ln(2\pi) - \frac{1}{2}E[\sum(x_i-\mu) | \mu_n]$$

From this point I am unsure how to proceed and if I'm even going in the right direction.

Answer 1

To quote verbatim from Wikipedia

The EM algorithm is used to find (local) maximum likelihood parameters of a statistical model in cases where the equations cannot be solved directly. Typically these models involve latent variables in addition to unknown parameters and known data observations. That is, either missing values exist among the data, or the model can be formulated more simply by assuming the existence of further unobserved data points. For example, a mixture model can be described more simply by assuming that each observed data point has a corresponding unobserved data point, or latent variable, specifying the mixture component to which each data point belongs.

When considering the Normal likelihood,

1. the maximum likelihood equation can be solved directly
2. there is no obvious missing data structure with latent variable $Z$ such that $$\int_\mathcal Z f(x,z;\mu)\,\text{d}z = \varphi(x;\mu) \tag{1}$$ and of course, there is (are?) an infinity of ways to create (1), e.g., $$f(x,z;\mu)=\mathbb I_{(0,\varphi(x;\mu))}(z)$$ or $$f(x,z;\mu)=\varphi(x;\mu)\varphi(z;\mu)\tag{2}$$ and one can try to apply EM to such completions but there is no reason that EM will be manageable in such cases. (Note: it works with (2).)

As a side remark, there is a fundamental misunderstanding in the following paragraph in Wikipedia

Finding a maximum likelihood solution typically requires taking the derivatives of the likelihood function with respect to all the unknown values, the parameters and the latent variables, and simultaneously solving the resulting equations.

since maximising in both $(\theta,z)$ returns the joint mode, which differs from the marginal mode.