Some methods of factor extraction (e.g. principal component analysis, PCA) are based on all variance in the data, while other methods (like principal axis factoring, PAF) are based on (or perhaps target) only common variance.
- How is this common variance defined mathematically?
- How is it estimated empirically?
I thought maybe it is the variance of a variable within the space spanned by all of the variables. Then it could be estimated by regressing each of the variables on the other variables and looking at the fitted values. But that does not seem correct.