Expectation of latent variables in Factor analysis Model I am going to through the theory behind factor analysis models given here 
Let's say our model is \begin{align} y_i = \mathcal \Lambda x_i +\epsilon, \end{align}
where $y_i$ is the $p$-dimensional observation and $x_i \in \mathcal N (0,I_q) $  is the q-dimensional underlying latent variable.  $\Lambda $ is the loading matrix. $\epsilon \in \mathcal N(0,\Psi)$ is the error term. 
Now I want to get the conditional expectation of latent variable $x$ given {$y,\Lambda,\Psi$}, i.e.
\begin{align} p(x|y,\Lambda,\Psi) \end{align}
Here is my attempt: 
\begin{align} \Lambda x & = y - \epsilon \\
\implies \Lambda^{'}\Lambda x & = \Lambda^{'}(y - \epsilon) \\ 
\implies  x & = (\Lambda^{'}\Lambda)^{-1}(\Lambda^{'}(y - \epsilon)) \\\end{align}
If I take the conditional expected value of $x$ now, I get 
 \begin{align} E(x|y,\Lambda,\Psi)&= E((\Lambda^{'}\Lambda)^{-1}(\Lambda^{'}(y - \epsilon))) \\ 
E(x|y,\Lambda,\Psi)&= ((\Lambda^{'}\Lambda)^{-1}(\Lambda^{'}(y - E(\epsilon)))) \\ 
E(x|y,\Lambda,\Psi)&= (\Lambda^{'}\Lambda)^{-1}(\Lambda^{'}y) \\\end{align}
But, the expression given on Slide 18 in the above slides, the conditional expected value of $x$ reads something likes $Λ (ΛΛ^T+ Ψ)^
{−1}y$
Can you please point out the mistake I am doing in my derivation. Thanks in advance!
 A: $\newcommand{\e}{\varepsilon}$$\newcommand{\L}{\Lambda}$We have $y_i = \mu + \L x_i + \e_i$.
We want to think of the jointly Gaussian RV 
$$
{x_i \choose y_i} \sim \mathcal N\left({0 \choose \mu}, \begin{bmatrix} I & \L^T \\ \L & \L\L^T + \Psi\end{bmatrix} \right)
$$
(slide 17 in that pdf) and from this we get
$$
x_i | y_i \sim \mathcal N\left(\L^T(\L\L^T+\Psi)^{-1}(y_i - \mu), I - \L^T(\L\L^T+\Psi)^{-1}\L\right)
$$
(slide 18) so
$$
E(x_i \vert y_i, \L, \Psi) = \L^T(\L\L^T+\Psi)^{-1}(y_i - \mu).
$$

Here's where you went wrong:
$$
E(x_i | y_i) = (\L^T\L)^{-1}\L^T(y_i - \mu - E(\e | y_i))
$$
but the mistake is that while $E(\e) = 0$, $E(\e | y_i)$ is not necessarily $0$.
Again think of the joint distribution of ${\e \choose y_i}$. Note $Cov(\e, y_i) = Cov(\e, \L x_i + \e) = \Psi$ so
$$
{\e \choose y_i} \sim \mathcal N\left({0 \choose \mu}, \begin{bmatrix} \Psi & \Psi \\ \Psi & \L\L^T + \Psi\end{bmatrix}\right).
$$
This means
$$
E(\e|y_i) = \Psi(\L\L^T + \Psi)^{-1}(y - \mu)
$$
so taking $\mu = 0$ now for simplicity we have
$$
E(x_i | y_i) = (\L^T\L)^{-1}\L^T(y_i - \Psi(\L\L^T + \Psi)^{-1}y_i) \\
= (\L^T\L)^{-1}\L^T(I - \Psi(\L\L^T + \Psi)^{-1}) y_i.
$$
Now consider
$$
\L^T(\L\L^T+\Psi)^{-1} - (\L^T\L)^{-1}\L^T(I - \Psi(\L\L^T + \Psi)^{-1}) \\
= \L^T(\L\L^T+\Psi)^{-1} - (\L^T\L)^{-1}\L^T +  (\L^T\L)^{-1}\L^T \Psi(\L\L^T + \Psi)^{-1} \\
= \left(\L^T - (\L^T\L)^{-1}\L^T(\L\L^T + \Psi) + (\L^T\L)^{-1}\L^T \Psi\right)(\L\L^T + \Psi)^{-1} \\
= \left(\L^T - (\L^T\L)^{-1}\L^T\L\L^T - (\L^T\L)^{-1}\L^T \Psi + (\L^T\L)^{-1}\L^T \Psi\right)(\L\L^T + \Psi)^{-1} \\
= \left(\L^T - \L^T - (\L^T\L)^{-1}\L^T \Psi + (\L^T\L)^{-1}\L^T \Psi\right)(\L\L^T + \Psi)^{-1} \\
= \mathbf 0.
$$
This means 
$$
\L^T(\L\L^T+\Psi)^{-1} = (\L^T\L)^{-1}\L^T(I - \Psi(\L\L^T + \Psi)^{-1})
$$
so actually this method, when correcting the mistake about $E(\e | y_i)$, yields the exact same answer! It just takes some work to turn one into the other. 
