Linked Questions
38 questions linked to/from What are the assumptions of factor analysis?
269
votes
15
answers
299k
views
What are the differences between Factor Analysis and Principal Component Analysis?
It seems that a number of the statistical packages that I use wrap these two concepts together. However, I'm wondering if there are different assumptions or data 'formalities' that must be true to use ...
- 7,362
137
votes
7
answers
30k
views
Is there an intuitive interpretation of $A^TA$ for a data matrix $A$?
For a given data matrix $A$ (with variables in columns and data points in rows), it seems like $A^TA$ plays an important role in statistics. For example, it is an important part of the analytical ...
- 2,415
146
votes
6
answers
88k
views
Should one remove highly correlated variables before doing PCA?
I'm reading a paper where author discards several variables due to high correlation to other variables before doing PCA. The total number of variables is around 20.
Does this give any benefits? It ...
- 1,571
102
votes
1
answer
126k
views
What correlation makes a matrix singular and what are implications of singularity or near-singularity?
I am doing some calculations on different matrices (mainly in logistic regression) and I commonly get the error "Matrix is singular", where I have to go back and remove the correlated variables. My ...
- 1,441
42
votes
2
answers
22k
views
How does Factor Analysis explain the covariance while PCA explains the variance?
Here is a quote from Bishop's "Pattern Recognition and Machine Learning" book, section 12.2.4 "Factor analysis":
According to the highlighted part, factor analysis captures the covariance between ...
- 3,653
42
votes
1
answer
45k
views
Is there Factor analysis or PCA for ordinal or binary data?
I have completed the principal component analysis (PCA), exploratory factor analysis (EFA), and confirmatory factor analysis (CFA), treating data with likert scale (5-level responses: none, a little, ...
- 463
37
votes
3
answers
59k
views
PCA on correlation or covariance: does PCA on correlation ever make sense? [closed]
In principal component analysis (PCA), one can choose either the covariance matrix or the correlation matrix to find the components (from their respective eigenvectors). These give different results (...
- 669
43
votes
1
answer
62k
views
Doing principal component analysis or factor analysis on binary data
I have a dataset with a large number of Yes/No responses. Can I use principal components (PCA) or any other data reduction analyses (such as factor analysis) for this type of data? Please advise how I ...
- 431
33
votes
1
answer
47k
views
Best factor extraction methods in factor analysis
SPSS offers several methods of factor extraction:
Principal components (which isn't factor analysis at all)
Unweighted least squares
Generalized least squares
Maximum Likelihood
Principal Axis
Alpha ...
- 14.5k
20
votes
3
answers
21k
views
PCA and exploratory Factor Analysis on the same dataset: differences and similarities; factor model vs PCA
I would like to know if it makes any logical sense to perform principal component analysis (PCA) and exploratory factor analysis (EFA) on the same data set. I have heard professionals expressly ...
- 301
17
votes
2
answers
17k
views
Why does sphericity diagnosed by Bartlett's Test mean a PCA is inappropriate?
I understand that Bartlett's Test is concerned with determining if your samples are from populations with equal variances.
If the samples are from populations with equal variances, then we fail to ...
- 1,215
14
votes
2
answers
36k
views
In Factor Analysis (or in PCA), what does it mean a factor loading greater than 1?
I've just run a FA using a oblique rotation (promax) and an item yielded a factor loading of 1.041 on one factor, (and factor loadings of -.131, -.119 and .065 on the other factors using pattern ...
11
votes
3
answers
24k
views
Is it acceptable to have only two (or less) items (variables) loaded by a factor in factor analysis?
I have a set of 20 variables that I have put through factor analysis in SPSS. For purposes of the research, I need to develop 6 factors. SPSS has shown that 8 variables (out of 20) have been loaded ...
- 119
6
votes
3
answers
4k
views
What does it mean if I have high Cronbach alpha, but poor results in Exploratory Factor Analysis
I have been given a survey to analyse. There are 50 questions, and about 400 respondents.
I have calculated Cronbach alpha for the entire thing, and I get about 0.9.
When I do factor analysis, I do ...
- 359
9
votes
2
answers
10k
views
Skewed variables in PCA or factor analysis
I want to do principal component analysis (factor analysis) on SPSS based on 22 variables. However, some of my variables are very skewed (skewness calculated from SPSS ranges from 2–80!).
So ...
- 169
8
votes
3
answers
4k
views
Difference between correlation and covariance: is covariance only useful if the relation is linear?
I'm trying to understand better the difference between covariance and correlation, besides the fact that the correlation coefficient is a dimensional and has values between $-1$ and $1$.
One ...
- 557
9
votes
2
answers
5k
views
Are data transformations on non-normal data necessary for an exploratory factor analysis when using the principal axis factoring extraction method?
I am developing a questionnaire to measure four factors which constitute spirituality, and I would like to ask the following question:
Are data transformations on non-normal data necessary for an ...
- 401
9
votes
2
answers
4k
views
What is the relation between singular correlation matrix and PCA?
Can anyone kindly give me some information about the statement (last sentence) at the end of below definition. What does it mean by "It can be used when a correlation matrix is singular"?
This quote ...
- 193
7
votes
1
answer
7k
views
Analyzing Ranked Data: Correlation and Factor Analysis?
I had survey respondents rank a series of items in order of importance, from 1 to 7. So, if a respondent assigned a score of 1 to one variable, then none of the other variables could get a 1 as well. ...
- 551
4
votes
1
answer
4k
views
High KMO but low communality in factor analysis
I'm performing a factor analysis and I have for a variable a Kaiser-Meyer-Olkin (KMO) measurement of .710 and a communality of ...
- 43
10
votes
1
answer
3k
views
What are dangers of calculating Pearson correlations (instead of tetrachoric ones) for binary variables in factor analysis?
I do research on educational games, and some of my current projects involve using data from BoardGameGeek (BGG) and VideoGameGeek (VGG) to examine relationships between design elements of games (i.e., ...
user66705
4
votes
1
answer
10k
views
Do Heywood cases render EFA/CFA solutions invalid?
If communality = 1, then we have a Heywood case, and if a communality > 1, it is known as an ultra-Heywood case. I read in a SAS manual that an ultra-Heywood case renders a factor solution invalid, ...
- 565
3
votes
1
answer
6k
views
Is it valid to perform PCA if Kaiser-Meyer-Olkin (KMO) index is very low?
I have a dataset that contains data from $307$ subjects and nine variables for each subject. I would like to run a PCA. My problem is that I get a Kaiser-Meyer-Olkin (KMO) value of $0.06$.
Can it be ...
- 33
4
votes
1
answer
4k
views
Is continuous inputs an assumption of factor analysis?
Should we use only continuous inputs for factor analysis (FA)? My data is a mix of continuous and categorical inputs: one of the inputs has only 600, 700 and 1000 as values.
I found that principal ...
- 4,271
6
votes
3
answers
1k
views
Scree plot: $m$ vs $m-1$ components/factors
@ttnphns comments here that
there exist two expositions of the Cattell scree-plot rule: If the "elbow" is the m-th eigenvalue, (1) choose to extract m components; or (2) choose to extract m-...
5
votes
1
answer
2k
views
Why is covariance matrix not positive-definite when number of observations is less than number of dimensions?
I have a data matrix $X$ of size $n\times p$ with $n < p$, where $n$ is the number of observations and $p$ is the number of dimensions.
My question is: why $n < p$ results in not a positive-...
- 85
2
votes
0
answers
5k
views
What is the intuition behind the KMO formula?
In answer to a different question about data assumptions of factor analysis rolando2 writes:
There is another condition that is sometimes treated as an "assumption": that the zero-order (vanilla) ...
3
votes
3
answers
4k
views
Highly correlated variables in exploratory factor analysis
Do I have to eliminate variables that are highly correlated before doing an exploratory factor analysis, like it has been discussed for PCA already here?
To specify, some items of my data are highly ...
- 141
1
vote
1
answer
3k
views
What kind of outlier should be removed from factor analysis?
We know outlier obs should be remove from factor analysis.
Attach plot of this column data
What kind of outlier should be removed and use which function in R?
- 507
5
votes
2
answers
2k
views
Factors with only two variables in factor analysis
I am running a factor analysis and have a couple of questions.
I have 10 variables, all of them come from a survey, with each answer is in the scale of 1 to 7. I have calculated a correlation matrix ...
- 51
2
votes
0
answers
2k
views
Are CFA Assumptions the same as EFA assumptions
I am learning currently CFA after completing EFA using Andy Fields book and Using Multi-variate statistics by Barbara Tabachnick and Linda Fidell.
In EFA the assumptions of the test are mentioned as
...
- 333
1
vote
2
answers
2k
views
Why cannot perform factor analysis when correlation below 0.3?
If the input sample correlation matrix consists of low correlations (say
below 0.3), do not perform factor analysis. There is not much to model!
I read this from statistics textbook. I don't ...
- 507
3
votes
1
answer
644
views
Regularization methods for factor analysis (in the $n<p$ situation)
Is there any covariance matrix regularization suitable for factor analysis?
I have a data matrix where number of observations is smaller than the number of dimensions: $n<p$.
I am thinking of ...
- 85
1
vote
1
answer
699
views
Factor analysis using Maximum likelihood or Principle axis factor to extract factors for 6 point likert type questions?
We have a questionnaire, which have many questions on 6-point likert scale.
So these variables are ordinal, not normally distributed.
In performing factor analysis, there are two major methods in ...
- 529
1
vote
0
answers
838
views
hierarchical cluster analysis in SPSS with ordinal data
I have ordinal data on scale 1-5 for detected pollutants in water (1 = detectable in small proportions; 5= detectable in higher proportions; also 0 was asaigned - not detectable).
I want to do HCA ...
- 21
0
votes
1
answer
679
views
What are the assumptions of Multiple Correspondence Analysis?
Is it possible to make Multiple Correspondence Analysis (MCA) with nominal data (such as country or gender) ? And more broadly, what are the assumptions of MCA?
For me, MCA is a type of factor ...
- 23
3
votes
1
answer
271
views
After a factor analysis, is there a way to see how much of the relationship between variables is from the latent factor?
Lets imagine a factor analysis on 20 different variables.
X1, X2 and X3 all load onto Factor 1.
X1 and X2 are highly correlated.
Is there a way to know if the relationship between X1 and X2 is: coming ...
- 2,701
2
votes
0
answers
20
views
Relationship between overall KMO and variance explained by factors
The Wikipedia page for the Kaiser–Meyer–Olkin test says that KMO "is a measure of the proportion of variance among variables that might be common variance."
So you will see people get an ...