Linked Questions
42 questions linked to/from How does Factor Analysis explain the covariance while PCA explains the variance?
20
votes
3
answers
21k
views
PCA and exploratory Factor Analysis on the same dataset: differences and similarities; factor model vs PCA
I would like to know if it makes any logical sense to perform principal component analysis (PCA) and exploratory factor analysis (EFA) on the same data set. I have heard professionals expressly ...
16
votes
1
answer
26k
views
Steps done in factor analysis compared to steps done in PCA
I know how to perform PCA (principal component analysis), but I would like to know steps that should be used for factor analysis.
To perform PCA, let us consider some matrix $A$, for instance:
...
12
votes
1
answer
9k
views
Data space, variable space, observation space, model space (e.g. in linear regression)
Suppose we have the data matrix $\mathbf{X}$, which is $n$-by-$p$, and the label vector $Y$, which is $n$-by-one. Here, each row of the matrix is an observation, and each column corresponds to a ...
10
votes
2
answers
2k
views
Is there an elegant/insightful way to understand this linear regression identity for multiple $R^2$?
In linear regression I have come across a delightful result that if we fit the model
$$E[Y] = \beta_1 X_1 + \beta_2 X_2 + c,$$
then, if we standardize and centre the $Y$, $X_1$ and $X_2$ data,
$$R^...
5
votes
1
answer
18k
views
What's the relationship between initial eigenvalues and sums of squared loadings in factor analysis?
On the one hand I read in a comment here that:
You can't speak of "eigenvalues" after rotation, even orthogonal
rotation. Perhaps you mean sum of squared loadings for a principal
component, ...
8
votes
1
answer
10k
views
What's the relationship between covariance, shared variance, and common variance?
I've generally assumed that shared variance and common variance were the same thing. However, here it is written that "Common variance is the realm of total collinearity. On the other hand, the term "...
9
votes
1
answer
3k
views
Under which conditions do PCA and FA yield similar results?
Under which conditions can principal components analysis (PCA) and factor analysis (FA) be expected to yield similar results?
13
votes
2
answers
667
views
What do the first $k$ factors from factor analysis maximize?
In principal components analysis, the first $k$ principal components are the $k$ orthogonal directions with the maximum variance. In other words, the first principal component is chosen to be the ...
2
votes
2
answers
3k
views
Singular covariance matrix in exploratory Factor Analysis
I'm kind of a noob to EFA and am trying to use the FANode object in Python. This is from the MDP library. I am using it on survey data to see which variables are tied together. Whenever I run it on my ...
4
votes
1
answer
5k
views
How do I interpret high residuals for the reproduced correlations in factor analysis?
I am doing a principal components analysis on a 4 item psychological scale (response format is a 0-10 point Likert scale for each item). As I hoped, an exploratory factor analysis yielded one factor. ...
4
votes
2
answers
2k
views
What is a "principal component factor analysis"?
I am currently researching silence in the social sciences and am reviewing surveys and statistical methods implemented by researchers to get an idea methods in both survey design and the analysis ...
5
votes
2
answers
2k
views
Where is the indeterminacy of factor values on this plot explaining factor analysis?
It is a well-known fact that in principal component analysis (PCA) we can obtain true values of components but in factor analysis (FA) we cannot obtain true values of common factors. We can compute ...
5
votes
1
answer
2k
views
Are principal components reflective, formative, both, or neither?
In reading various summaries on the similarities and differences between principal components and common factor models, I have noticed that there seems to be conflicting information about whether PCA ...
2
votes
1
answer
2k
views
Orthogonality of the basis in factor analysis
I have been studying principal component analysis (PCA) and then I have gone up to factor analysis (FA). I understood that PCA seeks orthonormal basis, but I am not so sure if this is the case for ...
2
votes
1
answer
2k
views
Basic question on PCA: can it be used to divide original features into groups?
I know this is basic question, but I have trouble understanding principal component analysis (PCA).
PCA can be used for dimensionality reduction. Let's assume we have 100 features that we think are ...