# Questions tagged [factor-analysis]

Factor analysis is a dimensionality reduction latent variable technique which replaces inter-correlating variables with a smaller number of continuous latent variables called factors. The factors are believed to be responsible for the inter-correlations. [For confirmatory factor analysis, please use the tag 'confirmatory-factor'. Also, the term "factor" of factor analysis should not be confused with "factor" as categorical predictor of a regression/ANOVA.]

1,208 questions
Filter by
Sorted by
Tagged with
294k views

### What are the differences between Factor Analysis and Principal Component Analysis?

It seems that a number of the statistical packages that I use wrap these two concepts together. However, I'm wondering if there are different assumptions or data 'formalities' that must be true to use ...
• 7,152
202k views

### PCA on correlation or covariance?

What are the main differences between performing principal component analysis (PCA) on the correlation matrix and on the covariance matrix? Do they give the same results?
• 2,280
33k views

### Is there any good reason to use PCA instead of EFA? Also, can PCA be a substitute for factor analysis?

In some disciplines, PCA (principal component analysis) is systematically used without any justification, and PCA and EFA (exploratory factor analysis) are considered as synonyms. I therefore ...
• 839
59k views

### Is PCA followed by a rotation (such as varimax) still PCA?

I have tried to reproduce some research (using PCA) from SPSS in R. In my experience, principal() function from package psych ...
• 3,638
38k views

### What is the relationship between independent component analysis and factor analysis?

I am new to Independent Component Analysis (ICA) and have just a rudimentary understanding of the the method. It seems to me that ICA is similar to Factor Analysis (FA) with one exception: ICA assumes ...
54k views

### Does the sign of scores or of loadings in PCA or FA have a meaning? May I reverse the sign?

I performed principal component analysis (PCA) with R using two different functions (prcomp and princomp) and observed that the ...
• 1,007
65k views

### What is the intuitive reason behind doing rotations in Factor Analysis/PCA & how to select appropriate rotation?

My Questions What is the intuitive reason behind doing rotations of factors in factor analysis (or components in PCA)? My understanding is, if variables are almost equally loaded in the top ...
• 5,583
60k views

### Doing principal component analysis or factor analysis on binary data

I have a dataset with a large number of Yes/No responses. Can I use principal components (PCA) or any other data reduction analyses (such as factor analysis) for this type of data? Please advise how I ...
• 431
21k views

### How does Factor Analysis explain the covariance while PCA explains the variance?

Here is a quote from Bishop's "Pattern Recognition and Machine Learning" book, section 12.2.4 "Factor analysis": According to the highlighted part, factor analysis captures the covariance between ...
• 3,571
42k views

### Is there Factor analysis or PCA for ordinal or binary data?

I have completed the principal component analysis (PCA), exploratory factor analysis (EFA), and confirmatory factor analysis (CFA), treating data with likert scale (5-level responses: none, a little, ...
• 443
58k views

### PCA on correlation or covariance: does PCA on correlation ever make sense? [closed]

In principal component analysis (PCA), one can choose either the covariance matrix or the correlation matrix to find the components (from their respective eigenvectors). These give different results (...
• 659
46k views

### Best factor extraction methods in factor analysis

SPSS offers several methods of factor extraction: Principal components (which isn't factor analysis at all) Unweighted least squares Generalized least squares Maximum Likelihood Principal Axis Alpha ...
• 14.3k
52k views

### Minimum sample size for PCA or FA when the main goal is to estimate only few components?

If I have a dataset with $n$ observations and $p$ variables (dimensions), and generally $n$ is small ($n=12-16$), and $p$ may range from small ($p = 4-10$) to perhaps much larger ($p= 30-50$). I ...
• 1,571
54k views

### Methods to compute factor scores, and what is the "score coefficient" matrix in PCA or factor analysis?

As per my understanding, in PCA based on correlations we get factor (= principal component in this instance) loadings which are nothing but the correlations between variables and factors. Now when I ...
34k views

### What is the proper association measure of a variable with a PCA component (on a biplot / loading plot)?

I am using FactoMineR to reduce my data set of measurements to the latent variables. The variable map above is clear for me to interpret, but I am confused when ...
20k views

### PCA and exploratory Factor Analysis on the same dataset: differences and similarities; factor model vs PCA

I would like to know if it makes any logical sense to perform principal component analysis (PCA) and exploratory factor analysis (EFA) on the same data set. I have heard professionals expressly ...
• 301
8k views

### Interpreting discrepancies between R and SPSS with exploratory factor analysis

I am a graduate student in computer science. I have been doing some exploratory factor analysis for a research project. My colleagues (who are leading the project) use SPSS, while I prefer to use R. ...
• 528
22k views

### Creating a single index from several principal components or factors retained from PCA/FA

I am using Principal Component Analysis (PCA) to create an index required for my research. My question is how I should create a single index by using the retained principal components calculated ...
• 293
28k views

### Factor analysis of questionnaires composed of Likert items

I used to analyse items from a psychometric point of view. But now I am trying to analyse other types of questions on motivation and other topics. These questions are all on Likert scales. My initial ...
• 1,241
30k views

### What are the assumptions of factor analysis?

I want to check if I really understood [classic, linear] factor analysis (FA), especially assumptions that are made before (and possibly after) FA. Some of the data should be initially correlated and ...
• 343
12k views

### What is the precise definition of a "Heywood Case"?

I had been using the term "Heywood Case" somewhat informally to refer to situations where an online, 'finite response' iteratively updated estimate of the variance became negative due to numerical ...
• 14.8k
526 views

### Can I use optimally scaled variables for a factor analysis to account for rotation? If I can then how?

I have discussed this issue several times in this site, but I am asking it again for a final justification from the experts of our community. I wanted to extract four factors (I should call dimensions ...
• 3,595
14k views

### Looking for a step through an example of a factor analysis on dichotomous data (binary variables) using R

I have some dichotomous data, only binary variables, and my boss asked me to perform a factor analysis using the tetrachoric correlations matrix. I’ve previously been able to teach myself how to run ...
• 439
10k views

### How does "Fundamental Theorem of Factor Analysis" apply to PCA, or how are PCA loadings defined?

I'm currently going through a slide set I have for "factor analysis" (PCA as far as I can tell). In it, the "fundamental theorem of factor analysis" is derived which claims that the correlation ...
• 583
26k views

### Steps done in factor analysis compared to steps done in PCA

I know how to perform PCA (principal component analysis), but I would like to know steps that should be used for factor analysis. To perform PCA, let us consider some matrix $A$, for instance: ...
21k views

### Factor rotation methods (varimax, quartimax, oblimin, etc.) - what do the names mean and what do the methods do?

Factor analysis has several rotation methods, such as varimax, quartimax, equamax, promax, oblimin, etc. I am unable to find any information that relates their names to their actual mathematical or ...
• 284
33k views

### In Factor Analysis (or in PCA), what does it mean a factor loading greater than 1?

I've just run a FA using a oblique rotation (promax) and an item yielded a factor loading of 1.041 on one factor, (and factor loadings of -.131, -.119 and .065 on the other factors using pattern ...
5k views

### Sum of rating scores vs estimated factor scores?

I'd be interested to receive suggestions about when to use "factor scores" over plain sum of scores when constructing scales. I.e. "Refined" over "non-refined" methods of scoring a factor. From ...
• 733
5k views

### Factor scores from discrete, ordinal responses

Is there a principled way to estimate factor scores when you have ordinal, discrete variables. I have $n$ ordinal, discrete, variables. If I make the assumption that underlying each response is a ...
• 1,200
649 views

### What do the first $k$ factors from factor analysis maximize?

In principal components analysis, the first $k$ principal components are the $k$ orthogonal directions with the maximum variance. In other words, the first principal component is chosen to be the ...
• 9,860
46k views

### The difference between varimax and oblimin rotations in factor analysis

What is the difference between varimax rotation and oblimin rotation in factor analysis? Also, I am confused about the relationship between principal component analysis, varimax rotation and ...
• 181
58k views

### Which matrix should be interpreted in factor analysis: pattern matrix or structure matrix?

When doing a factor analysis (by principal axis factoring, for example) or a principal component analysis as factor analysis, and having performed an oblique rotation of the loadings, - which matrix ...
• 143
9k views

### Can I use PCA to do variable selection for cluster analysis?

I have to reduce the number of variables to conduct a cluster analysis. My variables are strongly correlated, so I thought to do a Factor Analysis PCA (principal component analysis). However, if I use ...
• 173
5k views

### Dynamic factor analysis vs state space model

The MARSS package in R offers function for dynamic factor analysis. In this package, the dynamic factor model is written as a special form of state space model and they assume the common trends follow ...
• 153
16k views

### How to reduce number of items using factor analysis, internal consistency, and item response theory in conjunction?

I am in the process of empirically developing a questionnaire and I will be using arbitrary numbers in this example to illustrate. For context, I am developing a psychological questionnaire aimed at ...
• 5,064
23k views

### Difference between exploratory and confirmatory factor analysis in determining construct independence

Researchers often use two measures that have very similar items and argue that they measure different things (e.g., "I always worry when I am around cars"; "I am fearful of cars"). Lets call the ...
• 5,064
639 views

### Data reduction technique to identify types of countries

I teach an introductory economic geography course. To help my students develop a better understanding of the kinds of countries found in the contemporary world economy and an appreciation of data ...
• 171
23k views

### Is it acceptable to have only two (or less) items (variables) loaded by a factor in factor analysis?

I have a set of 20 variables that I have put through factor analysis in SPSS. For purposes of the research, I need to develop 6 factors. SPSS has shown that 8 variables (out of 20) have been loaded ...
• 119
14k views

### On the use of oblique rotation after PCA

Several statistical packages, such as SAS, SPSS, and R, allow you to perform some kind of factor rotation following a PCA. Why is a rotation necessary after a PCA? Why would you apply an oblique ...
• 1,241
908 views

### Is it always better to extract more factors when they exist?

Unlike principal components analysis, the solutions to factor analysis models are not necessarily nested. That is, the loadings (for example) for the first factor won't necessarily be identical when ...
4k views

### Is there a reason to leave an exploratory factor analysis solution unrotated?

Are there any reasons to not rotate an exploratory factor analysis solution? It's easy to find discussions comparing orthogonal solutions with oblique solutions, and I think I completely understand ...
• 349
6k views

### What is the relationship between scale reliability measures (Cronbach's alpha etc.) and component/factor loadings?

Let's say I have a dataset with scores on a bunch of questionnaire items, which are theoretically comprised of a smaller number of scales, like in psychology research. I know a common approach here ...
• 111
6k views

### How to make a matrix positive definite?

I'm trying to implement an EM algorithm for the following factor analysis model; $$W_j = \mu+B a_j+e_j \quad\text{for}\quad j=1,\ldots,n$$ where $W_j$ is p-dimensional random vector, $a_j$ is a q-...
• 223
3k views

### How to do factor analysis when the covariance matrix is not positive definite?

I have a data set that consists of 717 observations (rows) which are described by 33 variables (columns). The data are standardized by z-scoring all the variables. No two variables are linearly ...
• 133
26k views

### Is structural equation modeling (SEM) just another name of confirmatory factor analysis (CFA)?

I am reading some material about structural equation modeling. I found it to be extremely similar to confirmatory factor analysis - modeling a construct as the linear combination of several other ...
• 1,803
806 views

### Understanding factor analysis

Can I understand factor analysis in the following way? Assume I have 5 independent variables (A,B,C,D,E) Factor analysis allows me to make (D,E) to be dependent variables and allow me to make them ...
• 221
16k views

### How to correctly interpret a parallel analysis in exploratory factor analysis?

Some scientific papers report results of parallel analysis of principal axis factor analysis in a way inconsistent with my understanding of the methodology. What am I missing? Am I wrong or are they. ...
• 325
7k views

### Is there a standard measure of fit to validate Exploratory factor analysis?

I am modeling Exploratory Factor Analysis in R, Python, Mplus, and SPSS with maximum likelihood method and Varimax orthogonal rotation. However, each software gives different measures of fit and I am ...