Questions tagged [factor-analysis]

Factor analysis is a dimensionality reduction latent variable technique which replaces inter-correlating variables with a smaller number of continuous latent variables called factors. The factors are believed to be responsible for the inter-correlations. [For confirmatory factor analysis, please use the tag 'confirmatory-factor'. Also, the term "factor" of factor analysis should not be confused with "factor" as categorical predictor of a regression/ANOVA.]

Filter by
Sorted by
Tagged with
264 votes
15 answers
294k views

What are the differences between Factor Analysis and Principal Component Analysis?

It seems that a number of the statistical packages that I use wrap these two concepts together. However, I'm wondering if there are different assumptions or data 'formalities' that must be true to use ...
Brandon Bertelsen's user avatar
210 votes
7 answers
202k views

PCA on correlation or covariance?

What are the main differences between performing principal component analysis (PCA) on the correlation matrix and on the covariance matrix? Do they give the same results?
Random's user avatar
  • 2,280
83 votes
6 answers
33k views

Is there any good reason to use PCA instead of EFA? Also, can PCA be a substitute for factor analysis?

In some disciplines, PCA (principal component analysis) is systematically used without any justification, and PCA and EFA (exploratory factor analysis) are considered as synonyms. I therefore ...
Carine's user avatar
  • 839
72 votes
8 answers
59k views

Is PCA followed by a rotation (such as varimax) still PCA?

I have tried to reproduce some research (using PCA) from SPSS in R. In my experience, principal() function from package psych ...
Roman Luštrik's user avatar
70 votes
2 answers
38k views

What is the relationship between independent component analysis and factor analysis?

I am new to Independent Component Analysis (ICA) and have just a rudimentary understanding of the the method. It seems to me that ICA is similar to Factor Analysis (FA) with one exception: ICA assumes ...
stats_student's user avatar
55 votes
4 answers
54k views

Does the sign of scores or of loadings in PCA or FA have a meaning? May I reverse the sign?

I performed principal component analysis (PCA) with R using two different functions (prcomp and princomp) and observed that the ...
user1320502's user avatar
  • 1,007
44 votes
1 answer
65k views

What is the intuitive reason behind doing rotations in Factor Analysis/PCA & how to select appropriate rotation?

My Questions What is the intuitive reason behind doing rotations of factors in factor analysis (or components in PCA)? My understanding is, if variables are almost equally loaded in the top ...
GeorgeOfTheRF's user avatar
43 votes
1 answer
60k views

Doing principal component analysis or factor analysis on binary data

I have a dataset with a large number of Yes/No responses. Can I use principal components (PCA) or any other data reduction analyses (such as factor analysis) for this type of data? Please advise how I ...
Cathy's user avatar
  • 431
41 votes
2 answers
21k views

How does Factor Analysis explain the covariance while PCA explains the variance?

Here is a quote from Bishop's "Pattern Recognition and Machine Learning" book, section 12.2.4 "Factor analysis": According to the highlighted part, factor analysis captures the covariance between ...
avocado's user avatar
  • 3,571
40 votes
1 answer
42k views

Is there Factor analysis or PCA for ordinal or binary data?

I have completed the principal component analysis (PCA), exploratory factor analysis (EFA), and confirmatory factor analysis (CFA), treating data with likert scale (5-level responses: none, a little, ...
user116948's user avatar
36 votes
3 answers
58k views

PCA on correlation or covariance: does PCA on correlation ever make sense? [closed]

In principal component analysis (PCA), one can choose either the covariance matrix or the correlation matrix to find the components (from their respective eigenvectors). These give different results (...
Lucozade's user avatar
  • 659
34 votes
1 answer
46k views

Best factor extraction methods in factor analysis

SPSS offers several methods of factor extraction: Principal components (which isn't factor analysis at all) Unweighted least squares Generalized least squares Maximum Likelihood Principal Axis Alpha ...
Placidia's user avatar
  • 14.3k
30 votes
4 answers
52k views

Minimum sample size for PCA or FA when the main goal is to estimate only few components?

If I have a dataset with $n$ observations and $p$ variables (dimensions), and generally $n$ is small ($n=12-16$), and $p$ may range from small ($p = 4-10$) to perhaps much larger ($p= 30-50$). I ...
Patrick's user avatar
  • 1,571
24 votes
1 answer
54k views

Methods to compute factor scores, and what is the "score coefficient" matrix in PCA or factor analysis?

As per my understanding, in PCA based on correlations we get factor (= principal component in this instance) loadings which are nothing but the correlations between variables and factors. Now when I ...
Kartikeya Pandey's user avatar
21 votes
1 answer
34k views

What is the proper association measure of a variable with a PCA component (on a biplot / loading plot)?

I am using FactoMineR to reduce my data set of measurements to the latent variables. The variable map above is clear for me to interpret, but I am confused when ...
Fredrik Nylén's user avatar
20 votes
2 answers
20k views

PCA and exploratory Factor Analysis on the same dataset: differences and similarities; factor model vs PCA

I would like to know if it makes any logical sense to perform principal component analysis (PCA) and exploratory factor analysis (EFA) on the same data set. I have heard professionals expressly ...
user42538's user avatar
  • 301
19 votes
6 answers
8k views

Interpreting discrepancies between R and SPSS with exploratory factor analysis

I am a graduate student in computer science. I have been doing some exploratory factor analysis for a research project. My colleagues (who are leading the project) use SPSS, while I prefer to use R. ...
Oliver's user avatar
  • 528
19 votes
2 answers
22k views

Creating a single index from several principal components or factors retained from PCA/FA

I am using Principal Component Analysis (PCA) to create an index required for my research. My question is how I should create a single index by using the retained principal components calculated ...
user179313's user avatar
18 votes
3 answers
28k views

Factor analysis of questionnaires composed of Likert items

I used to analyse items from a psychometric point of view. But now I am trying to analyse other types of questions on motivation and other topics. These questions are all on Likert scales. My initial ...
pbneau's user avatar
  • 1,241
17 votes
3 answers
30k views

What are the assumptions of factor analysis?

I want to check if I really understood [classic, linear] factor analysis (FA), especially assumptions that are made before (and possibly after) FA. Some of the data should be initially correlated and ...
Sihem's user avatar
  • 343
17 votes
2 answers
12k views

What is the precise definition of a "Heywood Case"?

I had been using the term "Heywood Case" somewhat informally to refer to situations where an online, 'finite response' iteratively updated estimate of the variance became negative due to numerical ...
shabbychef's user avatar
  • 14.8k
17 votes
1 answer
526 views

Can I use optimally scaled variables for a factor analysis to account for rotation? If I can then how?

I have discussed this issue several times in this site, but I am asking it again for a final justification from the experts of our community. I wanted to extract four factors (I should call dimensions ...
Blain Waan's user avatar
  • 3,595
16 votes
1 answer
14k views

Looking for a step through an example of a factor analysis on dichotomous data (binary variables) using R

I have some dichotomous data, only binary variables, and my boss asked me to perform a factor analysis using the tetrachoric correlations matrix. I’ve previously been able to teach myself how to run ...
Eric Fail's user avatar
  • 439
16 votes
1 answer
10k views

How does "Fundamental Theorem of Factor Analysis" apply to PCA, or how are PCA loadings defined?

I'm currently going through a slide set I have for "factor analysis" (PCA as far as I can tell). In it, the "fundamental theorem of factor analysis" is derived which claims that the correlation ...
user2249626's user avatar
15 votes
1 answer
26k views

Steps done in factor analysis compared to steps done in PCA

I know how to perform PCA (principal component analysis), but I would like to know steps that should be used for factor analysis. To perform PCA, let us consider some matrix $A$, for instance: ...
dato datuashvili's user avatar
15 votes
2 answers
21k views

Factor rotation methods (varimax, quartimax, oblimin, etc.) - what do the names mean and what do the methods do?

Factor analysis has several rotation methods, such as varimax, quartimax, equamax, promax, oblimin, etc. I am unable to find any information that relates their names to their actual mathematical or ...
Estatistics's user avatar
14 votes
2 answers
33k views

In Factor Analysis (or in PCA), what does it mean a factor loading greater than 1?

I've just run a FA using a oblique rotation (promax) and an item yielded a factor loading of 1.041 on one factor, (and factor loadings of -.131, -.119 and .065 on the other factors using pattern ...
Rodrigo M. Rosales's user avatar
14 votes
2 answers
5k views

Sum of rating scores vs estimated factor scores?

I'd be interested to receive suggestions about when to use "factor scores" over plain sum of scores when constructing scales. I.e. "Refined" over "non-refined" methods of scoring a factor. From ...
Eric Green's user avatar
13 votes
3 answers
5k views

Factor scores from discrete, ordinal responses

Is there a principled way to estimate factor scores when you have ordinal, discrete variables. I have $n$ ordinal, discrete, variables. If I make the assumption that underlying each response is a ...
fgregg's user avatar
  • 1,200
13 votes
2 answers
649 views

What do the first $k$ factors from factor analysis maximize?

In principal components analysis, the first $k$ principal components are the $k$ orthogonal directions with the maximum variance. In other words, the first principal component is chosen to be the ...
raegtin's user avatar
  • 9,860
12 votes
1 answer
46k views

The difference between varimax and oblimin rotations in factor analysis

What is the difference between varimax rotation and oblimin rotation in factor analysis? Also, I am confused about the relationship between principal component analysis, varimax rotation and ...
xiongmao's user avatar
  • 181
12 votes
1 answer
58k views

Which matrix should be interpreted in factor analysis: pattern matrix or structure matrix?

When doing a factor analysis (by principal axis factoring, for example) or a principal component analysis as factor analysis, and having performed an oblique rotation of the loadings, - which matrix ...
Katigkou's user avatar
  • 143
12 votes
5 answers
9k views

Can I use PCA to do variable selection for cluster analysis?

I have to reduce the number of variables to conduct a cluster analysis. My variables are strongly correlated, so I thought to do a Factor Analysis PCA (principal component analysis). However, if I use ...
en.'s user avatar
  • 173
12 votes
1 answer
5k views

Dynamic factor analysis vs state space model

The MARSS package in R offers function for dynamic factor analysis. In this package, the dynamic factor model is written as a special form of state space model and they assume the common trends follow ...
user2380022's user avatar
12 votes
2 answers
16k views

How to reduce number of items using factor analysis, internal consistency, and item response theory in conjunction?

I am in the process of empirically developing a questionnaire and I will be using arbitrary numbers in this example to illustrate. For context, I am developing a psychological questionnaire aimed at ...
Behacad's user avatar
  • 5,064
12 votes
2 answers
23k views

Difference between exploratory and confirmatory factor analysis in determining construct independence

Researchers often use two measures that have very similar items and argue that they measure different things (e.g., "I always worry when I am around cars"; "I am fearful of cars"). Lets call the ...
Behacad's user avatar
  • 5,064
11 votes
7 answers
639 views

Data reduction technique to identify types of countries

I teach an introductory economic geography course. To help my students develop a better understanding of the kinds of countries found in the contemporary world economy and an appreciation of data ...
rabidotter's user avatar
11 votes
3 answers
23k views

Is it acceptable to have only two (or less) items (variables) loaded by a factor in factor analysis?

I have a set of 20 variables that I have put through factor analysis in SPSS. For purposes of the research, I need to develop 6 factors. SPSS has shown that 8 variables (out of 20) have been loaded ...
Mitja's user avatar
  • 119
11 votes
3 answers
14k views

On the use of oblique rotation after PCA

Several statistical packages, such as SAS, SPSS, and R, allow you to perform some kind of factor rotation following a PCA. Why is a rotation necessary after a PCA? Why would you apply an oblique ...
pbneau's user avatar
  • 1,241
11 votes
3 answers
908 views

Is it always better to extract more factors when they exist?

Unlike principal components analysis, the solutions to factor analysis models are not necessarily nested. That is, the loadings (for example) for the first factor won't necessarily be identical when ...
gung - Reinstate Monica's user avatar
11 votes
2 answers
4k views

Is there a reason to leave an exploratory factor analysis solution unrotated?

Are there any reasons to not rotate an exploratory factor analysis solution? It's easy to find discussions comparing orthogonal solutions with oblique solutions, and I think I completely understand ...
Jon's user avatar
  • 349
11 votes
1 answer
6k views

What is the relationship between scale reliability measures (Cronbach's alpha etc.) and component/factor loadings?

Let's say I have a dataset with scores on a bunch of questionnaire items, which are theoretically comprised of a smaller number of scales, like in psychology research. I know a common approach here ...
adb's user avatar
  • 111
11 votes
1 answer
6k views

How to make a matrix positive definite?

I'm trying to implement an EM algorithm for the following factor analysis model; $$W_j = \mu+B a_j+e_j \quad\text{for}\quad j=1,\ldots,n$$ where $W_j$ is p-dimensional random vector, $a_j$ is a q-...
Andy Amos's user avatar
  • 223
11 votes
3 answers
3k views

How to do factor analysis when the covariance matrix is not positive definite?

I have a data set that consists of 717 observations (rows) which are described by 33 variables (columns). The data are standardized by z-scoring all the variables. No two variables are linearly ...
Vasek's user avatar
  • 133
10 votes
4 answers
26k views

Is structural equation modeling (SEM) just another name of confirmatory factor analysis (CFA)?

I am reading some material about structural equation modeling. I found it to be extremely similar to confirmatory factor analysis - modeling a construct as the linear combination of several other ...
Tony's user avatar
  • 1,803
10 votes
2 answers
806 views

Understanding factor analysis

Can I understand factor analysis in the following way? Assume I have 5 independent variables (A,B,C,D,E) Factor analysis allows me to make (D,E) to be dependent variables and allow me to make them ...
Marco's user avatar
  • 221
10 votes
3 answers
16k views

How to correctly interpret a parallel analysis in exploratory factor analysis?

Some scientific papers report results of parallel analysis of principal axis factor analysis in a way inconsistent with my understanding of the methodology. What am I missing? Am I wrong or are they. ...
jhg's user avatar
  • 325
10 votes
1 answer
7k views

Is there a standard measure of fit to validate Exploratory factor analysis?

I am modeling Exploratory Factor Analysis in R, Python, Mplus, and SPSS with maximum likelihood method and Varimax orthogonal rotation. However, each software gives different measures of fit and I am ...
Ishant Sharma's user avatar
10 votes
2 answers
8k views

Recommended procedure for factor analysis on dichotomous data with R

I have to run a factor analysis on a dataset made up of dichotomous variables (0=yes, 1= no) and I don´t know if I'm on the right track. Using tetrachoric() I ...
cada's user avatar
  • 101
10 votes
1 answer
3k views

What are dangers of calculating Pearson correlations (instead of tetrachoric ones) for binary variables in factor analysis?

I do research on educational games, and some of my current projects involve using data from BoardGameGeek (BGG) and VideoGameGeek (VGG) to examine relationships between design elements of games (i.e., ...
Spencer Greenhalgh's user avatar

1
2 3 4 5
25