Linked Questions
42 questions linked to/from How does Factor Analysis explain the covariance while PCA explains the variance?
269
votes
15
answers
299k
views
What are the differences between Factor Analysis and Principal Component Analysis?
It seems that a number of the statistical packages that I use wrap these two concepts together. However, I'm wondering if there are different assumptions or data 'formalities' that must be true to use ...
- 7,362
168
votes
5
answers
151k
views
What's the difference between principal component analysis and multidimensional scaling?
How are PCA and classical MDS different? How about MDS versus nonmetric MDS? Is there a time when you would prefer one over the other? How do the interpretations differ?
- 4,323
146
votes
6
answers
88k
views
Should one remove highly correlated variables before doing PCA?
I'm reading a paper where author discards several variables due to high correlation to other variables before doing PCA. The total number of variables is around 20.
Does this give any benefits? It ...
- 1,571
86
votes
6
answers
35k
views
Is there any good reason to use PCA instead of EFA? Also, can PCA be a substitute for factor analysis?
In some disciplines, PCA (principal component analysis) is systematically used without any justification, and PCA and EFA (exploratory factor analysis) are considered as synonyms.
I therefore ...
- 869
72
votes
8
answers
62k
views
Is PCA followed by a rotation (such as varimax) still PCA?
I have tried to reproduce some research (using PCA) from SPSS in R. In my experience, principal() function from package psych ...
- 3,748
89
votes
4
answers
35k
views
How to visualize what canonical correlation analysis does (in comparison to what principal component analysis does)?
Canonical correlation analysis (CCA) is a technique related to principal component analysis (PCA). While it is easy to teach PCA or linear regression using a scatter plot (see a few thousand examples ...
- 993
53
votes
4
answers
23k
views
Why does the correlation coefficient between X and X-Y random variables tend to be 0.7
Taken from Practical Statistics for Medical Research where Douglas Altman writes in page 285:
...for any two quantities X and Y, X will be correlated with X-Y.
Indeed, even if X and Y are ...
- 1,507
102
votes
1
answer
126k
views
What correlation makes a matrix singular and what are implications of singularity or near-singularity?
I am doing some calculations on different matrices (mainly in logistic regression) and I commonly get the error "Matrix is singular", where I have to go back and remove the correlated variables. My ...
- 1,441
57
votes
4
answers
75k
views
Would PCA work for boolean (binary) data types?
I want to reduce the dimensionality of higher order systems and capture most of the covariance on a preferably 2 dimensional or 1 dimensional field. I understand this can be done via principal ...
- 757
54
votes
2
answers
74k
views
Multiple regression or partial correlation coefficient? And relations between the two
I don't even know if this question makes sense, but what is the difference between multiple regression and partial correlation (apart from the obvious differences between correlation and regression, ...
user34927
45
votes
1
answer
69k
views
What is the intuitive reason behind doing rotations in Factor Analysis/PCA & how to select appropriate rotation?
My Questions
What is the intuitive reason behind doing rotations of factors in factor analysis (or components in PCA)?
My understanding is, if variables are almost equally loaded in the top ...
- 5,683
33
votes
1
answer
47k
views
Best factor extraction methods in factor analysis
SPSS offers several methods of factor extraction:
Principal components (which isn't factor analysis at all)
Unweighted least squares
Generalized least squares
Maximum Likelihood
Principal Axis
Alpha ...
- 14.5k
17
votes
3
answers
31k
views
What are the assumptions of factor analysis?
I want to check if I really understood [classic, linear] factor analysis (FA), especially assumptions that are made before (and possibly after) FA.
Some of the data should be initially correlated and ...
- 343
22
votes
1
answer
36k
views
What is the proper association measure of a variable with a PCA component (on a biplot / loading plot)?
I am using FactoMineR to reduce my data set of measurements to the latent variables.
The variable map above is clear for me to interpret, but I am confused when ...
- 365
24
votes
1
answer
55k
views
Methods to compute factor scores, and what is the "score coefficient" matrix in PCA or factor analysis?
As per my understanding, in PCA based on correlations we get factor (= principal component in this instance) loadings which are nothing but the correlations between variables and factors. Now when I ...
- 453