Suppose we have $n$, $p$-dimensional, samples $\overrightarrow{X_i} \sim \mathcal{N}(\mu, \Psi+\mathbf{w^Tw})$. $\Psi$ is a diagonal matrix of specific variances, while $\mathbf{w^Tw}$ composes the remainder covariance structure in terms of factor loadings, $\mathbf{w}$. Under this model, $\mathbf X=\mu+\mathbf{Fw}+\epsilon$, where $\mathbf F$ denotes the (orthogonal) scores pertaining to $q$ factors $(q\lt p)$.
It's log-likelihood then is:
$$\mathcal{LL}=-{np\over2}\log(2\pi)-{n\over2}\log|\Psi+\mathbf{w^Tw}|-{n\over2}\text{tr}\left(\left(\Psi+\mathbf{w^Tw}\right)^{-1}\mathbf V\right)$$
Where
$$\mathbf V={(\mathbf X-\mu)^T(\mathbf X-\mu)\over n}$$
In these notes (under equation 27) it is stated that:
- A: starting from an estimate of $\Psi$ (an initial guess is easily obtained from multiple linear regression $R^2$s), the optimal $\Psi^{1/2}\mathbf w^T$ is given by the leading $q$ ($q\lt p$) eigen-vectors of $\Psi^{1/2}\mathbf V\Psi^{1/2}$
- B: likewise, starting from an estimate of $\mathbf w$, the optimal choice for $\Psi$ is $\mathbf {V - w^Tw}$. This can be derived from the definition of $\mathbf V$
Having both of these definitions, a two-stage MLE can be performed iteratively, until convergence.
My question is, how was A actually derived? I couldn't make the connection between $\Psi^{1/2}\mathbf w^T$ and $\Psi^{1/2}\mathbf V\Psi^{1/2}$.
By the way, here are the partial derivatives of the log-likelihood (unless I made some mistake):
\begin{align} &\frac{\partial\mathcal{LL}}{\partial\mathcal{\mathbf \Psi}}= (\Psi+\mathbf{w^Tw})^{-1}-(\Psi+\mathbf{w^Tw})^{-1}\mathbf V (\Psi+\mathbf{w^Tw})^{-1} \\ &\frac{\partial\mathcal{LL}}{\partial\mathcal{\mathbf w}}=2\mathbf w \frac{\partial\mathcal{LL}}{\partial\mathcal{\mathbf \Psi}} \end{align}
