EM Algorithm For Bipolar Normal Distribution Question:
Let $x_1, \dots, x_m$ be an i.i.d. sample from a normal density with mean $\mu$ and variance $\sigma^2$.
Suppose for each $x_i$ we observe $y_i = |x_i|$ . Formulate an EM algorithm for
estimating $\mu$ and $\sigma^2$.
My solution:
Define a latent variable $Z$, when $z_i = 1, x_i = y_i$ and $z_i = 0, x_i = -y_i$ and the probability $p(z_i = 1| \Theta, y_i) = p$. It can be easily known that $-x_i \sim \mathcal{N}(-\mu, \sigma^2)$.
$$ \begin{equation}
  \begin{aligned}
  l(\mathbf{x},  \mathbf{z}, p, \Theta) = \sum_{i = 1}^m z_i\left[ -\frac{1}{2}\ln 2\pi - \frac{1}{2}\ln \sigma^2 -\frac{1}{2\sigma^2}(x_i - \mu)^2 + \ln p\right]\\ + \sum_{i = 1}^m (1 - z_i)\left[ -\frac{1}{2}\ln 2\pi - \frac{1}{2}\ln \sigma^2 -\frac{1}{2\sigma^2}(x_i + \mu)^2 + \ln (1-p)\right],
  \end{aligned}
 \end{equation}$$
The E step in EM algorithm is:$E_{\Theta_{n}}[l(\mathbf{x},  \mathbf{z}, p, \Theta) | \mathbf{y}]$.
My question:

*

*It seems that some problems happens in my model since two latent variables $z_i, p$ and unknown $x_i$ involved in the E step. So could anyone tell me where is the mistake?


*I see the answer for updating the $\mu$ involves $f(y_i | \Theta_n)$, but honestly speaking, from the E step: $E[x_iz_i | \Theta_n, y_i]$, there would be no $f_i$ involved. So how come the formula?
Thanks in advance!

The likelihood function can be further expressed as:
\begin{equation}
  \begin{aligned}
   Q(\Theta, \Theta_{n}) = & E_{\Theta_{n}}[l(\mathbf{x},  \mathbf{z}, \Theta) | \mathbf{y}]\\
   = & \sum_{i = 1}^m\left( -\frac{1}{2}\ln 2\pi - \frac{1}{2}\ln \sigma^2 - \frac{E_{\Theta_{n}}[x_i^2|y_i]}{2\sigma^2} - \frac{\mu^2}{2\sigma^2} - \frac{1-2\mu E_{\Theta_{n}}[x_iz_i|y_i]}{\sigma^2}\right)
  \end{aligned} 
 \end{equation}
The expectation of $E[x_iz_i | \Theta_n, y_i]$
$$
\begin{equation}
\begin{aligned}
E[x z | \Theta_n, y] = & \int \sum_l xz_lp(x_k,z_l | \Theta_n, y) dx\\
= &\int  xp(x_k,z = 1 | \Theta_n, y)dx\quad \text{only z = 1 left}\\
= & p(z = 1 | \Theta_n, y)\int x f(x | z = 1, \Theta_n, y)dx\\
= & \frac{f(y_i|\theta_n)}{f(y_i|\theta_n) + f(-y_i|\theta_n)} \mu_n
\end{aligned}
\end{equation}$$:
But still stuck.
 A: *

*There is no probability $p$ in this problem as
$$\mathbb P_\theta(Z_i=1)=\mathbb P_\theta(X_i>0)=1-\Phi(\mu/\sigma)$$

*There is only one type of latent variable, $\mathbf Z$, since $\mathbf X$ is a deterministic function of $\mathbf Y$ and $\mathbf Z$, as discussed below.

*the complete likelihood can thus be expressed in terms of $\mathbf Y$ and $\mathbf Z$ only

If $X\sim\mathcal N(\mu,\sigma^2)$, then $Y=|X|$ has a Dirac mass distribution at $|X|$ conditional on $X$. The marginal distribution of $Y$ is the folded Normal, with density
$$\sigma^{-1}\varphi(y;\mu,\sigma)+\sigma^{-1}\varphi(-y;\mu,\sigma)$$
Conversely, the distribution of $X$ conditional on $Y$ is a sum of Dirac masses at $Y$ and $-Y$ with respective masses proportional to $\varphi(y;\mu,\sigma)$ and $\varphi(-y;\mu,\sigma)$. Note that
$$Z=\mathbb I_{X=|Y|}$$
is a deterministic transform of $(X,Y)$, hence that $Z$ is known given $(X,Y)$ and that $X$ is known given $(Z,Y)$. This implies that
$$\mathbb E_{\theta_{n}}[l(\mathbf{X},  \mathbf{Z}, \theta) | \mathbf{y}]
=\mathbb E_{\theta_{n}}[l(\mathbf{X(Z,Y)},  \mathbf{Z}, \theta) | \mathbf{y}] $$
and, since
\begin{equation}
  \begin{aligned}
  l(\mathbf{x},  \mathbf{z}, p, \Theta) &= \sum_{i = 1}^m \mathbb I_{z_i=1}\left[ -\frac{1}{2}\ln 2\pi - \frac{1}{2}\ln \sigma^2 -\frac{1}{2\sigma^2}(x_i(1,y_i) - \mu)^2 \right]\\ &\quad + \sum_{i = 1}^m \mathbb I_{z_i=0}\left[ -\frac{1}{2}\ln 2\pi - \frac{1}{2}\ln \sigma^2 -\frac{1}{2\sigma^2}(x_i(0,y_i) - \mu)^2 \right],\\
&= \sum_{i = 1}^m \mathbb I_{z_i=1}\left[ -\frac{1}{2}\ln 2\pi - \frac{1}{2}\ln \sigma^2 -\frac{1}{2\sigma^2}(y_i - \mu)^2 \right]\\ &\quad + \sum_{i = 1}^m \mathbb I_{z_i=0}\left[ -\frac{1}{2}\ln 2\pi - \frac{1}{2}\ln \sigma^2 -\frac{1}{2\sigma^2}(-y_i - \mu)^2 \right],
  \end{aligned}
 \end{equation}
the E-step writes as
\begin{equation}
  \begin{aligned}
\mathbb E_{\theta_n}[l(X,Z,\theta)|y) &= -\frac{m}{2}\ln 2\pi - \frac{m}{2}\ln \sigma^2- \frac{1}{2\sigma^2}\sum_{i = 1}^m \mathbb E_{\theta_n}[\mathbb I_{z_i=1}|y] (y_i - \mu)^2 \\ &\quad -\frac{1}{2\sigma^2}
\sum_{i=1}^m \mathbb E_{\theta_n}[\mathbb I_{z_i=0}|y] (y_i + \mu)^2
  \end{aligned}
 \end{equation}
This implies that $\mu_{n+1}$ for the M-step is solution of the equation
$$\sum_{i = 1}^m \mathbb E_{\theta_n}[\mathbb I_{z_i=1}|y] (\mu-y_i)
+\sum_{i=1}^m \mathbb E_{\theta_n}[\mathbb I_{z_i=0}|y] (y_i + \mu) = 0$$
