Linked Questions

20 votes
3 answers
21k views

PCA and exploratory Factor Analysis on the same dataset: differences and similarities; factor model vs PCA

I would like to know if it makes any logical sense to perform principal component analysis (PCA) and exploratory factor analysis (EFA) on the same data set. I have heard professionals expressly ...
user42538's user avatar
  • 301
16 votes
1 answer
26k views

Steps done in factor analysis compared to steps done in PCA

I know how to perform PCA (principal component analysis), but I would like to know steps that should be used for factor analysis. To perform PCA, let us consider some matrix $A$, for instance: ...
dato datuashvili's user avatar
12 votes
1 answer
9k views

Data space, variable space, observation space, model space (e.g. in linear regression)

Suppose we have the data matrix $\mathbf{X}$, which is $n$-by-$p$, and the label vector $Y$, which is $n$-by-one. Here, each row of the matrix is an observation, and each column corresponds to a ...
user3813057's user avatar
  • 1,122
10 votes
2 answers
2k views

Is there an elegant/insightful way to understand this linear regression identity for multiple $R^2$?

In linear regression I have come across a delightful result that if we fit the model $$E[Y] = \beta_1 X_1 + \beta_2 X_2 + c,$$ then, if we standardize and centre the $Y$, $X_1$ and $X_2$ data, $$R^...
Corvus's user avatar
  • 5,415
5 votes
1 answer
18k views

What's the relationship between initial eigenvalues and sums of squared loadings in factor analysis?

On the one hand I read in a comment here that: You can't speak of "eigenvalues" after rotation, even orthogonal rotation. Perhaps you mean sum of squared loadings for a principal component, ...
user1205901 - Слава Україні's user avatar
8 votes
1 answer
10k views

What's the relationship between covariance, shared variance, and common variance?

I've generally assumed that shared variance and common variance were the same thing. However, here it is written that "Common variance is the realm of total collinearity. On the other hand, the term "...
user1205901 - Слава Україні's user avatar
9 votes
1 answer
3k views

Under which conditions do PCA and FA yield similar results?

Under which conditions can principal components analysis (PCA) and factor analysis (FA) be expected to yield similar results?
stats's user avatar
  • 93
13 votes
2 answers
667 views

What do the first $k$ factors from factor analysis maximize?

In principal components analysis, the first $k$ principal components are the $k$ orthogonal directions with the maximum variance. In other words, the first principal component is chosen to be the ...
raegtin's user avatar
  • 10.1k
2 votes
2 answers
3k views

Singular covariance matrix in exploratory Factor Analysis

I'm kind of a noob to EFA and am trying to use the FANode object in Python. This is from the MDP library. I am using it on survey data to see which variables are tied together. Whenever I run it on my ...
DeepLackofLearning's user avatar
4 votes
1 answer
5k views

How do I interpret high residuals for the reproduced correlations in factor analysis?

I am doing a principal components analysis on a 4 item psychological scale (response format is a 0-10 point Likert scale for each item). As I hoped, an exploratory factor analysis yielded one factor. ...
user13313's user avatar
4 votes
2 answers
2k views

What is a "principal component factor analysis"?

I am currently researching silence in the social sciences and am reviewing surveys and statistical methods implemented by researchers to get an idea methods in both survey design and the analysis ...
John Smith's user avatar
5 votes
2 answers
2k views

Where is the indeterminacy of factor values on this plot explaining factor analysis?

It is a well-known fact that in principal component analysis (PCA) we can obtain true values of components but in factor analysis (FA) we cannot obtain true values of common factors. We can compute ...
ttnphns's user avatar
  • 58.8k
5 votes
1 answer
2k views

Are principal components reflective, formative, both, or neither?

In reading various summaries on the similarities and differences between principal components and common factor models, I have noticed that there seems to be conflicting information about whether PCA ...
jsakaluk's user avatar
  • 5,566
2 votes
1 answer
2k views

Orthogonality of the basis in factor analysis

I have been studying principal component analysis (PCA) and then I have gone up to factor analysis (FA). I understood that PCA seeks orthonormal basis, but I am not so sure if this is the case for ...
Marcel's user avatar
  • 158
2 votes
1 answer
2k views

Basic question on PCA: can it be used to divide original features into groups?

I know this is basic question, but I have trouble understanding principal component analysis (PCA). PCA can be used for dimensionality reduction. Let's assume we have 100 features that we think are ...
bigTree's user avatar
  • 909

15 30 50 per page