0
$\begingroup$

In Alpaydin book it is stated that the reduced data set $\mathbf{Z}$ from the original data $\mathbf{X}$ can be obtained by following a multivariate linear regression:

$$\mathbf{Z}=\mathbf{X}(WW^{\intercal}+\Psi)^{-1}W$$

where $W$ is the factor loadings and $\Psi$ is the noise not explained by the factors. So far, so good. My question is why we can simply approximate $\mathbf{Z}$ as $\mathbb{E}[\mathbf{Z}]$ using the values obtained from the EM algorithm?

Additionally, Marsland's Factor Analysis code implies that

$$\mathbb{E}[\mathbf{Z}]=(WW^{\intercal}+\Psi)^{-1}W$$

so

$$\mathbf{Z}=\mathbf{X}\mathbb{E}[\mathbf{Z}]$$

Is he right? How did he obtained that expression?

$\endgroup$
  • $\begingroup$ What you call $\bf Z$ are called standardized factor scores. They are usually computed by the regression approach as (when factors are orthogonal) $\bf XR^{-1}W$, where $\bf X$ are standardized original variables and $\bf R$ is the correlation matrix of them. However, in a good factor analysis $\bf R$ is closely approximated by $\bf R^*=WW'+\Psi$, the restored correlations, and so you may use $\bf R^*$ instead, as you formula does. The technical gain here is simply that $\bf R^*$ is low rank and its inversion is cheeper. This gain is negligible with moderncomputers. $\endgroup$ – ttnphns Nov 30 '14 at 11:09
  • $\begingroup$ (cont.) On the other hand, if $\bf R^*$ is quite different from $\bf R$, as in PCA for instance, it shouldn't be used. $\endgroup$ – ttnphns Nov 30 '14 at 11:13
  • $\begingroup$ My comment was about explaining the regression method of computing factor scores, it does not answer your question. The thing is what is $\mathbb{E}[\mathbf{Z}]$. Expected mean value for a factor? But according your 2nd formula, it is the matrix of regressional coefficients to compute factor scores. So, I didn't quite understand what you mean. $\endgroup$ – ttnphns Nov 30 '14 at 11:24
  • $\begingroup$ That is the thing, Marsland implies that the expected mean values $\mathbb{E}\big[\mathbf{Z}\big]$ is the matrix of regressional coefficients. But in the book he never states how did he obtained that expression. $\endgroup$ – Gorayni Nov 30 '14 at 18:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.