Linked Questions

1 vote
0 answers
208 views

Rotation in PCA and Factor Analysis [duplicate]

I want to know what elements are (varimax-)rotated when I rotate after PCA and after Factor Analysis. Let’s assume a standardized data vector $X$ of dimension $N \times q$. In PCA, I have the ...
MaHo's user avatar
  • 391
608 votes
5 answers
492k views

Relationship between SVD and PCA. How to use SVD to perform PCA?

Principal component analysis (PCA) is usually explained via an eigen-decomposition of the covariance matrix. However, it can also be performed via singular value decomposition (SVD) of the data matrix ...
amoeba's user avatar
  • 105k
266 votes
15 answers
295k views

What are the differences between Factor Analysis and Principal Component Analysis?

It seems that a number of the statistical packages that I use wrap these two concepts together. However, I'm wondering if there are different assumptions or data 'formalities' that must be true to use ...
Brandon Bertelsen's user avatar
108 votes
5 answers
225k views

Loadings vs eigenvectors in PCA: when to use one or another?

In principal component analysis (PCA), we get eigenvectors (unit vectors) and eigenvalues. Now, let us define loadings as $$\text{Loadings} = \text{Eigenvectors} \cdot \sqrt{\text{Eigenvalues}}.$$ I ...
user2696565's user avatar
  • 1,389
30 votes
4 answers
53k views

Minimum sample size for PCA or FA when the main goal is to estimate only few components?

If I have a dataset with $n$ observations and $p$ variables (dimensions), and generally $n$ is small ($n=12-16$), and $p$ may range from small ($p = 4-10$) to perhaps much larger ($p= 30-50$). I ...
Patrick's user avatar
  • 1,571
41 votes
2 answers
21k views

How does Factor Analysis explain the covariance while PCA explains the variance?

Here is a quote from Bishop's "Pattern Recognition and Machine Learning" book, section 12.2.4 "Factor analysis": According to the highlighted part, factor analysis captures the covariance between ...
avocado's user avatar
  • 3,581
44 votes
1 answer
66k views

What is the intuitive reason behind doing rotations in Factor Analysis/PCA & how to select appropriate rotation?

My Questions What is the intuitive reason behind doing rotations of factors in factor analysis (or components in PCA)? My understanding is, if variables are almost equally loaded in the top ...
GeorgeOfTheRF's user avatar
17 votes
3 answers
65k views

How to compute varimax-rotated principal components in R?

I ran PCA on 25 variables and selected the top 7 PCs using prcomp. prc <- prcomp(pollutions, center=T, scale=T, retx=T) I ...
Scott's user avatar
  • 609
24 votes
1 answer
54k views

Methods to compute factor scores, and what is the "score coefficient" matrix in PCA or factor analysis?

As per my understanding, in PCA based on correlations we get factor (= principal component in this instance) loadings which are nothing but the correlations between variables and factors. Now when I ...
Kartikeya Pandey's user avatar
20 votes
2 answers
20k views

PCA and exploratory Factor Analysis on the same dataset: differences and similarities; factor model vs PCA

I would like to know if it makes any logical sense to perform principal component analysis (PCA) and exploratory factor analysis (EFA) on the same data set. I have heard professionals expressly ...
user42538's user avatar
  • 301
15 votes
1 answer
26k views

Steps done in factor analysis compared to steps done in PCA

I know how to perform PCA (principal component analysis), but I would like to know steps that should be used for factor analysis. To perform PCA, let us consider some matrix $A$, for instance: ...
dato datuashvili's user avatar
15 votes
1 answer
19k views

What are "rotated" and "unrotated" principal components, given that PCA always rotates the coordinates axes?

As far as I understand, principal components are obtained by rotating the coordinate axes to align them with the directions of maximum variance. Nevertheless, I keep reading about "unrotated ...
Srewashi Lahiri's user avatar
9 votes
4 answers
10k views

Very different results of principal component analysis in SPSS and Stata after rotation

For my PhD thesis I have to do a Principal Component Analysis (PCA). I didn't find it too difficult in Stata and was happy interpreting the results (I know there is a difference between factor and ...
hanne's user avatar
  • 93
5 votes
1 answer
17k views

What's the relationship between initial eigenvalues and sums of squared loadings in factor analysis?

On the one hand I read in a comment here that: You can't speak of "eigenvalues" after rotation, even orthogonal rotation. Perhaps you mean sum of squared loadings for a principal component, ...
user1205901 - Слава Україні's user avatar
9 votes
1 answer
8k views

Using varimax-rotated PCA components as predictors in linear regression

After doing PCA, the first component describes the largest part of variability. This is important e.g. in study of body measurements where it is commonly known (Jolliffe, 2002) that PC1 axis captures ...
Fedja Blagojevic's user avatar

15 30 50 per page