# What does the Psi term in factor analysis signify?

Diagonal elements of Psi (...) represent independent noise variances for each of the variables

C.M. Bishop, Pattern Recognition and Machine Learning

...but I'm not clear on what does the Psi signify. I've applied FA to a series of images, received Psi matrix and now struggling with interpreting the results.

-
What's the source for that quote? –  Shane Dec 10 '10 at 18:31
'Pattern Recognition and Machine Learning' by C.M.Bishop –  matcheek Dec 10 '10 at 18:32

$$p(x|z)=\mathcal{N}(x|\mathbf{W}z+\boldsymbol{\mu},\boldsymbol{\psi})$$
where $\boldsymbol{\psi}$ is a $D\times D$ diagonal matrix of so-called variable uniquenesses, that is the variance not accounted for by the latent factors, whereas $\mathbf{W}$ reflects factor loadings $\lambda_i$, that is the correlation of variable $i$ with factors represented in $z$ (more exactly, the square of $\lambda_i$ is the variance explained by the latent factor).
If you're not familiar with the FA literature, I would suggest "lighter" approach, e.g. William Revelle has good tutorials on his website personality-project.org; especially, I would suggest Chapter 6 of his forthcoming book on Psychometric methods entitled Constructs, Components, and Factor models. You will shortly understand the relations between PCA and FA. Specifically, with PCA, we are constructing linear combinations of observed variables (this yields a composite variable), whereas in FA we are expressing each variable as a weighted combination of hypothesized latent factors (where weights are called loadings) plus an error term (the $\boldsymbol{\psi}$ in the above formula). In sum, the FA model incorporates a model for noise--this is what is expressed in Equation 12.65; but see What are the differences between Factor Analysis and Principal Component Analysis, for additional discussion.
I'm not familiar with Bishop's notation, but IIRC, the loading $\lambda_{ij}$ in matrix $W$ equals the correlation of observed variable $i$ with latent factor $j$ only in the orthogonal-factor model. In a model with correlated factors, matrix $W \cdot C_f$ contains the variable-factor correlations if $C_f$ is the correlation matrix of the factors (not equal to the identity matrix). –  caracal Dec 10 '10 at 22:08