Help in Expectation Maximization from paper :how to include prior distribution?

Question

The Question is based on the paper titled : Image reconstruction in diffuse optical tomography using the coupled radiative transport–diffusion model

Download link

The Authors apply EM algorithm with $l_1$ sparsity regularization of an unknown vector $\mu$ to estimate the pixels of an image. The model is given by

$$y=A\mu + e \tag{1}$$ The estimate is given in Eq(8) as

$$\hat{\mu} = \arg max {\ln p(y|\mu) + \gamma \ln p(\mu)} \tag{2}$$

In my case, I have considered $\mu$ to be a filter of length $L$ and $\mathbf{\mu}$ are $L \times 1$ vectors representing the filters. So,

The model can be rewritten as $$y(n) = \mathbf{\mu^T}a(n) + v(n) \tag{3}$$

Question : Problem formulation : ${\mu(n)}$ (n by 1) is the unobserved input and $\{e(n)\}$ is the zero mean with unknown variance $\sigma^2_e$ additive noise. The MLE solution will based on Expectation Maximization (EM).

In the paper Eq(19) is the $A$ function - the complete log-likelihood but for my case I do not understand how I can include the distribution of $A, \mu$ in the the complete log-likelihood expression.

What will be the complete log-likelihood using EM of $y$ including the prior distribution?

Answer 1

If we consider the target as $$\arg\max_\theta L(\theta|x)\pi(\theta) = \arg\max_\theta \log L(\theta|x) + \log \pi(\theta)$$ the representation at the basis of EM is $$\log L(\theta|x) = \mathbb{E}[\log L(\theta|x,Z)|x,\theta⁰]-\mathbb{E}[\log q(Z|x,\theta)|x,\theta⁰]$$ for an arbitrary $\theta⁰$, because of the decomposition $$q(z|x,\theta)=f(x,z|\theta) \big/ g(x|\theta)$$ or $$g(x|\theta) = f(x,z|\theta) \big/ q(z|x,\theta)$$ which works for an arbitrary value of $z$ (since there is none on the lhs) and hence also works for any expectation in $Z$: $$\log g(x|\theta) = \log f(x,z|\theta) - \log q(z|x,\theta) = \mathbb{E}[\log f(x,Z|\theta) - \log q(Z|x,\theta)|x]$$ for any conditional distribution of $Z$ given $X=x$, for instance $q(z|x,\theta⁰)$. Therefore if we maximise in $\theta$ $$\mathbb{E}[\log L(\theta|x,Z)|x,\theta⁰]+ \log \pi(\theta)$$ with solution $\theta^1$ we have $$\mathbb{E}[\log L(\theta^1|x,Z)|x,\theta⁰]+ \log \pi(\theta^1)\ge\mathbb{E}[\log L(\theta⁰|x,Z)|x,\theta⁰]+ \log \pi(\theta⁰)$$ while $$\mathbb{E}[\log q(Z|x,\theta⁰)|x,\theta⁰]\ge\mathbb{E}[\log q(Z|x,\theta^1)|x,\theta⁰]$$ by the standard arguments of EM. Therefore, $$\mathbb{E}[\log L(\theta^1|x,Z)|x,\theta⁰]+ \log \pi(\theta^1)\ge\mathbb{E}[\log L(\theta⁰|x,Z)|x,\theta⁰]+ \log \pi(\theta⁰)$$ and using as an E step the target $$\mathbb{E}[\log L(\theta|x,Z)|x,\theta⁰]+ \log \pi(\theta)$$ leads to an increase in the posterior at each M step, meaning that the modified EM algorithm converges to a local MAP.

Answer 2

I don't think showing monotonic increasing log-posterior (or log likelihood for MLE) are sufficient for showing convergence to stationary point of the MAP estimate (or MLE). For example, the increments can become arbitrarily small. In the famous paper by Wu 1983, a sufficient condition for converging to stationary point of EM is differentiability in both arguments of the lower bound function.