Questions tagged [factor-analysis]

Factor analysis is a dimensionality reduction latent variable technique which replaces inter-correlating variables with a smaller number of continuous latent variables called factors. The factors are believed to be responsible for the inter-correlations. [For confirmatory factor analysis, please use the tag 'confirmatory-factor'. Also, the term "factor" of factor analysis should not be confused with "factor" as categorical predictor of a regression/ANOVA.]

Filter by
Sorted by
Tagged with
210 votes
7 answers
202k views

PCA on correlation or covariance?

What are the main differences between performing principal component analysis (PCA) on the correlation matrix and on the covariance matrix? Do they give the same results?
Random's user avatar
  • 2,280
264 votes
15 answers
294k views

What are the differences between Factor Analysis and Principal Component Analysis?

It seems that a number of the statistical packages that I use wrap these two concepts together. However, I'm wondering if there are different assumptions or data 'formalities' that must be true to use ...
Brandon Bertelsen's user avatar
55 votes
4 answers
54k views

Does the sign of scores or of loadings in PCA or FA have a meaning? May I reverse the sign?

I performed principal component analysis (PCA) with R using two different functions (prcomp and princomp) and observed that the ...
user1320502's user avatar
  • 1,007
43 votes
1 answer
60k views

Doing principal component analysis or factor analysis on binary data

I have a dataset with a large number of Yes/No responses. Can I use principal components (PCA) or any other data reduction analyses (such as factor analysis) for this type of data? Please advise how I ...
Cathy's user avatar
  • 431
24 votes
1 answer
54k views

Methods to compute factor scores, and what is the "score coefficient" matrix in PCA or factor analysis?

As per my understanding, in PCA based on correlations we get factor (= principal component in this instance) loadings which are nothing but the correlations between variables and factors. Now when I ...
Kartikeya Pandey's user avatar
83 votes
6 answers
33k views

Is there any good reason to use PCA instead of EFA? Also, can PCA be a substitute for factor analysis?

In some disciplines, PCA (principal component analysis) is systematically used without any justification, and PCA and EFA (exploratory factor analysis) are considered as synonyms. I therefore ...
Carine's user avatar
  • 839
40 votes
1 answer
42k views

Is there Factor analysis or PCA for ordinal or binary data?

I have completed the principal component analysis (PCA), exploratory factor analysis (EFA), and confirmatory factor analysis (CFA), treating data with likert scale (5-level responses: none, a little, ...
user116948's user avatar
72 votes
8 answers
59k views

Is PCA followed by a rotation (such as varimax) still PCA?

I have tried to reproduce some research (using PCA) from SPSS in R. In my experience, principal() function from package psych ...
Roman Luštrik's user avatar
20 votes
2 answers
20k views

PCA and exploratory Factor Analysis on the same dataset: differences and similarities; factor model vs PCA

I would like to know if it makes any logical sense to perform principal component analysis (PCA) and exploratory factor analysis (EFA) on the same data set. I have heard professionals expressly ...
user42538's user avatar
  • 301
41 votes
2 answers
21k views

How does Factor Analysis explain the covariance while PCA explains the variance?

Here is a quote from Bishop's "Pattern Recognition and Machine Learning" book, section 12.2.4 "Factor analysis": According to the highlighted part, factor analysis captures the covariance between ...
avocado's user avatar
  • 3,551
34 votes
1 answer
46k views

Best factor extraction methods in factor analysis

SPSS offers several methods of factor extraction: Principal components (which isn't factor analysis at all) Unweighted least squares Generalized least squares Maximum Likelihood Principal Axis Alpha ...
Placidia's user avatar
  • 14.3k
44 votes
1 answer
65k views

What is the intuitive reason behind doing rotations in Factor Analysis/PCA & how to select appropriate rotation?

My Questions What is the intuitive reason behind doing rotations of factors in factor analysis (or components in PCA)? My understanding is, if variables are almost equally loaded in the top ...
GeorgeOfTheRF's user avatar
21 votes
1 answer
33k views

What is the proper association measure of a variable with a PCA component (on a biplot / loading plot)?

I am using FactoMineR to reduce my data set of measurements to the latent variables. The variable map above is clear for me to interpret, but I am confused when ...
Fredrik Nylén's user avatar
15 votes
1 answer
26k views

Steps done in factor analysis compared to steps done in PCA

I know how to perform PCA (principal component analysis), but I would like to know steps that should be used for factor analysis. To perform PCA, let us consider some matrix $A$, for instance: ...
dato datuashvili's user avatar
17 votes
3 answers
30k views

What are the assumptions of factor analysis?

I want to check if I really understood [classic, linear] factor analysis (FA), especially assumptions that are made before (and possibly after) FA. Some of the data should be initially correlated and ...
Sihem's user avatar
  • 343
18 votes
3 answers
28k views

Factor analysis of questionnaires composed of Likert items

I used to analyse items from a psychometric point of view. But now I am trying to analyse other types of questions on motivation and other topics. These questions are all on Likert scales. My initial ...
pbneau's user avatar
  • 1,241
36 votes
3 answers
58k views

PCA on correlation or covariance: does PCA on correlation ever make sense? [closed]

In principal component analysis (PCA), one can choose either the covariance matrix or the correlation matrix to find the components (from their respective eigenvectors). These give different results (...
Lucozade's user avatar
  • 659
19 votes
2 answers
22k views

Creating a single index from several principal components or factors retained from PCA/FA

I am using Principal Component Analysis (PCA) to create an index required for my research. My question is how I should create a single index by using the retained principal components calculated ...
user179313's user avatar
6 votes
1 answer
6k views

Estimating specific variance for items in factor analysis - how to achieve the theoretical maximum?

(Remark: this is a "scholastic" question - I'm reviewing my implementation of factor analysis procedures; I'm not looking for good approximations for an actual survey/actual data or the like.) ...
Gottfried Helms's user avatar
15 votes
2 answers
21k views

Factor rotation methods (varimax, quartimax, oblimin, etc.) - what do the names mean and what do the methods do?

Factor analysis has several rotation methods, such as varimax, quartimax, equamax, promax, oblimin, etc. I am unable to find any information that relates their names to their actual mathematical or ...
Estatistics's user avatar
14 votes
2 answers
5k views

Sum of rating scores vs estimated factor scores?

I'd be interested to receive suggestions about when to use "factor scores" over plain sum of scores when constructing scales. I.e. "Refined" over "non-refined" methods of scoring a factor. From ...
Eric Green's user avatar
12 votes
1 answer
58k views

Which matrix should be interpreted in factor analysis: pattern matrix or structure matrix?

When doing a factor analysis (by principal axis factoring, for example) or a principal component analysis as factor analysis, and having performed an oblique rotation of the loadings, - which matrix ...
Katigkou's user avatar
  • 143
1 vote
1 answer
2k views

Regression testing after dimension reduction

I have a 12 item Likert scale for my predictor variable [IV], and a 9 item Likert scale for my dependent variable [DV]. I used SPSS and did factor analysis on both scales, and found that the IV had 3 ...
haroldcampbell's user avatar
5 votes
2 answers
2k views

Where is the indeterminacy of factor values on this plot explaining factor analysis?

It is a well-known fact that in principal component analysis (PCA) we can obtain true values of components but in factor analysis (FA) we cannot obtain true values of common factors. We can compute ...
ttnphns's user avatar
  • 57.2k
4 votes
2 answers
2k views

General advice on forming an index of attitude to the environment from a set of Likert items

I am going to make an environmental index to be used as an explanatory variable in a regression model. For making this index, I asked respondents a set of questions about their environmental ...
Alireza's user avatar
  • 41
9 votes
1 answer
3k views

Under which conditions do PCA and FA yield similar results?

Under which conditions can principal components analysis (PCA) and factor analysis (FA) be expected to yield similar results?
stats's user avatar
  • 93
7 votes
1 answer
7k views

Analyzing Ranked Data: Correlation and Factor Analysis?

I had survey respondents rank a series of items in order of importance, from 1 to 7. So, if a respondent assigned a score of 1 to one variable, then none of the other variables could get a 1 as well. ...
spindoctor's user avatar
30 votes
4 answers
52k views

Minimum sample size for PCA or FA when the main goal is to estimate only few components?

If I have a dataset with $n$ observations and $p$ variables (dimensions), and generally $n$ is small ($n=12-16$), and $p$ may range from small ($p = 4-10$) to perhaps much larger ($p= 30-50$). I ...
Patrick's user avatar
  • 1,571
9 votes
2 answers
10k views

Skewed variables in PCA or factor analysis

I want to do principal component analysis (factor analysis) on SPSS based on 22 variables. However, some of my variables are very skewed (skewness calculated from SPSS ranges from 2–80!). So ...
Meo's user avatar
  • 173
8 votes
1 answer
9k views

What's the relationship between covariance, shared variance, and common variance?

I've generally assumed that shared variance and common variance were the same thing. However, here it is written that "Common variance is the realm of total collinearity. On the other hand, the term "...
user1205901 - Слава Україні's user avatar
6 votes
3 answers
4k views

What does it mean if I have high Cronbach alpha, but poor results in Exploratory Factor Analysis

I have been given a survey to analyse. There are 50 questions, and about 400 respondents. I have calculated Cronbach alpha for the entire thing, and I get about 0.9. When I do factor analysis, I do ...
Old_Mortality's user avatar
16 votes
1 answer
10k views

How does "Fundamental Theorem of Factor Analysis" apply to PCA, or how are PCA loadings defined?

I'm currently going through a slide set I have for "factor analysis" (PCA as far as I can tell). In it, the "fundamental theorem of factor analysis" is derived which claims that the correlation ...
user2249626's user avatar
10 votes
1 answer
3k views

What are dangers of calculating Pearson correlations (instead of tetrachoric ones) for binary variables in factor analysis?

I do research on educational games, and some of my current projects involve using data from BoardGameGeek (BGG) and VideoGameGeek (VGG) to examine relationships between design elements of games (i.e., ...
Spencer Greenhalgh's user avatar
7 votes
2 answers
9k views

Factor analysis for ordinal variables that have different categories

I have a data set that contains about 40 categorical variables that are taken as independent variables (and believed to be related to some unobservable human resource factors) and 4 categorical ...
Blain Waan's user avatar
  • 3,595
3 votes
2 answers
7k views

Factor analysis and regression

I have a question about how to interpret a regression analysis I did following a factor analysis. I did principal axis factoring (direct oblimin) I got a 3 factor solution. The three factors were (1)...
Sandi's user avatar
  • 31
9 votes
2 answers
5k views

Are data transformations on non-normal data necessary for an exploratory factor analysis when using the principal axis factoring extraction method?

I am developing a questionnaire to measure four factors which constitute spirituality, and I would like to ask the following question: Are data transformations on non-normal data necessary for an ...
Madeline's user avatar
  • 401
9 votes
1 answer
6k views

Why are regression coefficients in a factor analysis model called "loadings"?

In this thread @ttnphns writes that Because it is regression coefficients [...] I insist that it is better to say "factor loads variable" than "variable loads factor". I learned from here that a ...
user1205901 - Слава Україні's user avatar
8 votes
1 answer
6k views

Based on factor loadings (in factor analysis) can we give unequal weights to Likert scale items?

After gathering data, we compute score of any Likert (summative) scale (previously identified as factor in factor analysis) by adding up its individual item scores (and maybe dividing the sum by the ...
user12483's user avatar
5 votes
1 answer
17k views

What's the relationship between initial eigenvalues and sums of squared loadings in factor analysis?

On the one hand I read in a comment here that: You can't speak of "eigenvalues" after rotation, even orthogonal rotation. Perhaps you mean sum of squared loadings for a principal component, ...
user1205901 - Слава Україні's user avatar
4 votes
1 answer
5k views

Categorical Principal Component Analysis - using Count, Continuous, Ordinal variables together

I have some variables and I want to reduce their number for further analysis. I initially thought of combining them using factor analysis. But since the variables are of all kinds (rating, count, ...
Rahul's user avatar
  • 53
3 votes
1 answer
7k views

What is the difference between the anti-image covariance and the anti-image correlation?

What is the difference between the anti-image covariance and the anti-image correlation? How are the matrices of these coefficients computed, and what is the meaning of their elements?
Ponjul Zwalda's user avatar
16 votes
1 answer
14k views

Looking for a step through an example of a factor analysis on dichotomous data (binary variables) using R

I have some dichotomous data, only binary variables, and my boss asked me to perform a factor analysis using the tetrachoric correlations matrix. I’ve previously been able to teach myself how to run ...
Eric Fail's user avatar
  • 439
11 votes
3 answers
14k views

On the use of oblique rotation after PCA

Several statistical packages, such as SAS, SPSS, and R, allow you to perform some kind of factor rotation following a PCA. Why is a rotation necessary after a PCA? Why would you apply an oblique ...
pbneau's user avatar
  • 1,241
11 votes
2 answers
4k views

Is there a reason to leave an exploratory factor analysis solution unrotated?

Are there any reasons to not rotate an exploratory factor analysis solution? It's easy to find discussions comparing orthogonal solutions with oblique solutions, and I think I completely understand ...
Jon's user avatar
  • 349
9 votes
3 answers
13k views

Differences between tetrachoric and Pearson correlation

Currently I'm analysing around 300 items in the field of education. I'm interested in the dimensionality of the dataset. So I compute a matrix of tetrachoric correlation. The goal is to do a factor ...
pbneau's user avatar
  • 1,241
5 votes
2 answers
872 views

Scree plot: $m$ vs $m-1$ components/factors

@ttnphns comments here that there exist two expositions of the Cattell scree-plot rule: If the "elbow" is the m-th eigenvalue, (1) choose to extract m components; or (2) choose to extract m-...
user1205901 - Слава Україні's user avatar
5 votes
1 answer
2k views

Why is covariance matrix not positive-definite when number of observations is less than number of dimensions?

I have a data matrix $X$ of size $n\times p$ with $n < p$, where $n$ is the number of observations and $p$ is the number of dimensions. My question is: why $n < p$ results in not a positive-...
pierre's user avatar
  • 85
4 votes
1 answer
5k views

Factor scores vs. construct mean scores in regression analysis

I have 48 items in my questionnaire that represent 8 constructs. After conducting an exploratory factor analysis (EFA) with a principal components extraction method and Varimax rotation method, 8 ...
Cyrus's user avatar
  • 287
4 votes
1 answer
4k views

High KMO but low communality in factor analysis

I'm performing a factor analysis and I have for a variable a Kaiser-Meyer-Olkin (KMO) measurement of .710 and a communality of ...
Frederik's user avatar
4 votes
2 answers
42k views

What is the difference between PCA and PAF method in factor analysis?

What is the difference between principal component analyses (PCA) and principal axis factoring (PAF)? Also, I understand the difference between varimax and oblimin rotations, but is that the same as ...
Nadav Keyson's user avatar