Linked Questions

4
votes
1answer
5k views

What is the difference between Exploratory Factor Analysis and Principal Components Analysis (PCA)? [duplicate]

I know what you're thinking, this is a duplicate of "What are the differences between Factor Analysis and Principal Component Analysis", but it isn't really. That other question deals with ...
4
votes
0answers
1k views

What is the difference between scores in Princomp vs. factanal? [duplicate]

In R the princomp()and the factanal() are somewhat similar. At least their output looks pretty similar. I learned that this is ...
0
votes
0answers
823 views

Difference between FA and PCA [duplicate]

Possible Duplicate: What are the differences between Factor Analysis and Principal Component Analysis I am trying to understand the difference between PCA and FA. Through google research, I have ...
1
vote
0answers
305 views

PCA vs. Factor Analysis [duplicate]

When should we use PCA over factor analysis? Aren't they essentially the same thing except that factor analysis is modeling observed variables as linear combinations of unobserved factors? Whereas PCA ...
0
votes
0answers
75 views

What are the main differences between principal component analysis PCA and factor analysis [duplicate]

I have used PCA in my thesis and would like to argue (in best way) during viva the choice in addition to the fact that PCA analyses the variance of the observed items whereas FA analyses covariance.
73
votes
6answers
22k views

Is there any good reason to use PCA instead of EFA? Also, can PCA be a substitute for factor analysis?

In some disciplines, PCA (principal component analysis) is systematically used without any justification, and PCA and EFA (exploratory factor analysis) are considered as synonyms. I therefore ...
64
votes
8answers
40k views

Is PCA followed by a rotation (such as varimax) still PCA?

I have tried to reproduce some research (using PCA) from SPSS in R. In my experience, principal() function from package psych ...
98
votes
4answers
129k views

PCA and proportion of variance explained

In general, what is meant by saying that the fraction $x$ of the variance in an analysis like PCA is explained by the first principal component? Can someone explain this intuitively but also give a ...
125
votes
1answer
97k views

How to reverse PCA and reconstruct original variables from several principal components?

Principal component analysis (PCA) can be used for dimensionality reduction. After such dimensionality reduction is performed, how can one approximately reconstruct the original variables/features ...
76
votes
4answers
127k views

Loadings vs eigenvectors in PCA: when to use one or another?

In principal component analysis (PCA), we get eigenvectors (unit vectors) and eigenvalues. Now, let us define loadings as $$\text{Loadings} = \text{Eigenvectors} \cdot \sqrt{\text{Eigenvalues}}.$$ I ...
32
votes
3answers
11k views

Visualizing a million, PCA edition

Is it possible to visualize the output of Principal Component Analysis in ways that give more insight than just summary tables? Is it possible to do it when the number of observations is large, say ~...
37
votes
2answers
15k views

How does Factor Analysis explain the covariance while PCA explains the variance?

Here is a quote from Bishop's "Pattern Recognition and Machine Learning" book, section 12.2.4 "Factor analysis": According to the highlighted part, factor analysis captures the covariance between ...
27
votes
4answers
32k views

Minimum sample size for PCA or FA when the main goal is to estimate only few components?

If I have a dataset with $n$ observations and $p$ variables (dimensions), and generally $n$ is small ($n=12-16$), and $p$ may range from small ($p = 4-10$) to perhaps much larger ($p= 30-50$). I ...
32
votes
3answers
40k views

Latent Class Analysis vs. Cluster Analysis - differences in inferences?

What are the differences in inferences that can be made from a latent class analysis (LCA) versus a cluster analysis? Is it correct that a LCA assumes an underlying latent variable that gives rise to ...
29
votes
1answer
32k views

Best factor extraction methods in factor analysis

SPSS offers several methods of factor extraction: Principal components (which isn't factor analysis at all) Unweighted least squares Generalized least squares Maximum Likelihood Principal Axis Alpha ...
34
votes
1answer
41k views

What is the intuitive reason behind doing rotations in Factor Analysis/PCA & how to select appropriate rotation?

My Questions What is the intuitive reason behind doing rotations of factors in factor analysis (or components in PCA)? My understanding is, if variables are almost equally loaded in the top ...
20
votes
2answers
36k views

Methods to compute factor scores, and what is the “score coefficient” matrix in PCA or factor analysis?

As per my understanding, in PCA based on correlations we get factor (= principal component in this instance) loadings which are nothing but the correlations between variables and factors. Now when I ...
19
votes
2answers
14k views

PCA and exploratory Factor Analysis on the same dataset: differences and similarities; factor model vs PCA

I would like to know if it makes any logical sense to perform principal component analysis (PCA) and exploratory factor analysis (EFA) on the same data set. I have heard professionals expressly ...
11
votes
3answers
14k views

What are the assumptions of factor analysis?

I want to check if I really understood [classic, linear] factor analysis (FA), especially assumptions that are made before (and possibly after) FA. Some of the data should be initially correlated and ...
17
votes
1answer
17k views

What is the proper association measure of a variable with a PCA component (on a biplot / loading plot)?

I am using FactoMineR to reduce my data set of measurements to the latent variables. The variable map above is clear for me to interpret, but I am confused when ...
12
votes
1answer
19k views

Steps done in factor analysis compared to steps done in PCA

I know how to perform PCA (principal component analysis), but I would like to know steps that should be used for factor analysis. To perform PCA, let us consider some matrix $A$, for instance: ...
9
votes
3answers
12k views

On the use of oblique rotation after PCA

Several statistical packages, such as SAS, SPSS, and R, allow you to perform some kind of factor rotation following a PCA. Why is a rotation necessary after a PCA? Why would you apply an oblique ...
6
votes
3answers
3k views

Whether to use EFA or PCA to assess dimensionality of a set of Likert items

This follows on from my previous question on assessing reliability. I designed a questionnaire (six 5-points Likert items) to evaluate the attitude of a group of users toward a product. I would like ...
7
votes
4answers
8k views

Very different results of principal component analysis in SPSS and Stata after rotation

For my PhD thesis I have to do a Principal Component Analysis (PCA). I didn't find it too difficult in Stata and was happy interpreting the results (I know there is a difference between factor and ...
5
votes
2answers
12k views

Determining number of factors in exploratory factor analysis

I'm using R's factanal function for factor analysis. I know from reading that there are various ways of picking how many factors to use in the analysis. I don't ...
4
votes
3answers
2k views

Whether to use factor analysis based on binary multiple response data?

I have a survey where I have asked people which type of computer games they enjoy and whether they consider themselves a hardcore gamer. I allowed people to select multiple genres, but now I am unsure ...
5
votes
3answers
4k views

What's the difference between a component and a factor in parallel analysis?

The psych package in R has a fa.parallel function to help determine the number of factors or components. From the documentation: ...
7
votes
2answers
2k views

What is the relation between singular correlation matrix and PCA?

Can anyone kindly give me some information about the statement (last sentence) at the end of below definition. What does it mean by "It can be used when a correlation matrix is singular"? This quote ...
8
votes
1answer
2k views

Under which conditions do PCA and FA yield similar results?

Under which conditions can principal components analysis (PCA) and factor analysis (FA) be expected to yield similar results?
4
votes
1answer
5k views

Factor scores vs. construct mean scores in regression analysis

I have 48 items in my questionnaire that represent 8 constructs. After conducting an exploratory factor analysis (EFA) with a principal components extraction method and Varimax rotation method, 8 ...
4
votes
1answer
3k views

High KMO but low communality in factor analysis

I'm performing a factor analysis and I have for a variable a Kaiser-Meyer-Olkin (KMO) measurement of .710 and a communality of ...
2
votes
2answers
2k views

Where can I find information about using SPSS for EFA and CFA? Is PCA (two samples) and reliability sufficient for scale development?

Context: I am in the process of developing a scale for my thesis. My advisor has guided me to using SPSS PCA to complete my analyses. Initially we reduced my scale to 3 factors (her insistence), ...
2
votes
2answers
3k views

Differences on exploratory factor analysis, confirmatory factor analysis and principal component analysis

Before it is pointed, I am aware that a very similar question was already asked. Still, I am in doubt regarding the concept. More specifically, it is mentioned by the most voted answer that: In ...
3
votes
1answer
3k views

Metrics of correlation for 3+ variables

I have a basic question, which hopefully you all can resolve for me. What is the best way to determine correlations between 3 or more variables? I have a dataset in which 5 continuous variables each ...
5
votes
2answers
1k views

Where is the indeterminacy of factor values on this plot explaining factor analysis?

It is a well-known fact that in principal component analysis (PCA) we can obtain true values of components but in factor analysis (FA) we cannot obtain true values of common factors. We can compute ...
4
votes
1answer
2k views

Fundamental difference between PCA and FA?

According to this, the fundamental difference between PCA and FA can be illustrated via the following image: So, the direction of arrows changes. According to this answer and a few others: ...
4
votes
1answer
1k views

What does the Psi term in factor analysis signify?

Diagonal elements of Psi (...) represent independent noise variances for each of the variables C.M. Bishop, Pattern Recognition and Machine Learning ...but I'm not clear on what does the ...
1
vote
1answer
961 views

Recoded items as separate factors in factor analysis

Hello and thanks in advance for your help! I have conducted and EFA for a scale I made that includes 13 items. Of these items, 2 are recoded. Looking at the rotated component matrix I see that ...
0
votes
0answers
2k views

PCA vs PAF for exploratory factor analysis

I've been reading about performing exploratory factor analysis via principal axis factor extraction (PAF) and principal component analysis (PCA). I'm a bit confused about why the difference between ...
2
votes
1answer
933 views

How to reverse factor analysis (FA) and reconstruct original variables?

I saw this interesting topic: How to reverse PCA and reconstruct original variables from several principal components? and a nice answer with a very useful example of Iris data in Matlab. I would like ...
1
vote
0answers
2k views

When plotting PCA analysis with loadings on top, loading arrows come out way too short [duplicate]

I am trying to make a reasonable looking PCA analysis, where not only data are projected in two axis, but also the loadings of the data are projected on top of the data. Similar to the following ...
4
votes
0answers
919 views

Are principal components reflective, formative, both, or neither?

In reading various summaries on the similarities and differences between principal components and common factor models, I have noticed that there seems to be conflicting information about whether PCA ...
4
votes
1answer
420 views

Is it valid to use only some of the components as regressors from a PCA?

I was trying a 4 components PCA where the 1st and 2nd components have emerged exactly as I expected, but the other two components do not commensurate with the theory. I mean, I expected a little ...
0
votes
1answer
639 views

Eliminated items in Factor Analysis

I'd like to get your opinions on how to interpret items that had to be eliminated in factor analysis (FA). I've been researching consumer shopping motivations and ran a survey with 40 items, which ...
1
vote
1answer
944 views

In principal component analysis, is PC a linear combination of input variables or vice versa?

In every literature I've read each principal component is expressed as a linear combination of input variable. And the coefficient matrix is called factor loading. But why in John Hull's book (where ...
0
votes
1answer
654 views

Can I physically determine the number of factors in Factor Analysis?

I have completed a Factor Analysis on a around 180 responses for a questionnaire with 56 Variables. I got 15 factors using Oblique Rotation PCA. Some of the factors have only two variables of ...
1
vote
2answers
237 views

Which is more appropriate in this case — PCA or Factor Analysis?

Suppose I have an experiment that has 8 factors. These 8 factors are probably related and can hopefully be reduced. For each combination of these 8 factors that I test, I get a single output. My goal ...
1
vote
1answer
632 views

Is it necessary to do confirmatory factor analysis for structure equation modelling?

I have done Principal Component Analysis for a scale comprising multiple latent variables. Is it necessary to do Confirmatory Factor Analysis before doing Structure Equation Modelling?
2
votes
1answer
380 views

What are the differences between these two kinds of PCA?

The book "Elements of Statistical Learning" describes Principal Components Analysis through SVD as follows: $$X = UDV^T$$ Then $ UD $ are the Principal Components and $ V $ are the directions. ...
3
votes
1answer
487 views

Understanding (exploratory) factor analysis: some points for clarification

[A question about what we optimize in FA, is FA a clustering of variables, and when/how we choose the number of factors] I have read some tutorials and looked at some of the questions here, as well, ...

15 30 50 per page