Linked Questions

267 votes
15 answers

What are the differences between Factor Analysis and Principal Component Analysis?

It seems that a number of the statistical packages that I use wrap these two concepts together. However, I'm wondering if there are different assumptions or data 'formalities' that must be true to use ...
Brandon Bertelsen's user avatar
137 votes
7 answers

Is there an intuitive interpretation of $A^TA$ for a data matrix $A$?

For a given data matrix $A$ (with variables in columns and data points in rows), it seems like $A^TA$ plays an important role in statistics. For example, it is an important part of the analytical ...
Alec's user avatar
  • 2,395
146 votes
6 answers

Should one remove highly correlated variables before doing PCA?

I'm reading a paper where author discards several variables due to high correlation to other variables before doing PCA. The total number of variables is around 20. Does this give any benefits? It ...
type2's user avatar
  • 1,571
101 votes
1 answer

What correlation makes a matrix singular and what are implications of singularity or near-singularity?

I am doing some calculations on different matrices (mainly in logistic regression) and I commonly get the error "Matrix is singular", where I have to go back and remove the correlated variables. My ...
Error404's user avatar
  • 1,431
41 votes
2 answers

How does Factor Analysis explain the covariance while PCA explains the variance?

Here is a quote from Bishop's "Pattern Recognition and Machine Learning" book, section 12.2.4 "Factor analysis": According to the highlighted part, factor analysis captures the covariance between ...
avocado's user avatar
  • 3,633
41 votes
1 answer

Is there Factor analysis or PCA for ordinal or binary data?

I have completed the principal component analysis (PCA), exploratory factor analysis (EFA), and confirmatory factor analysis (CFA), treating data with likert scale (5-level responses: none, a little, ...
user116948's user avatar
36 votes
3 answers

PCA on correlation or covariance: does PCA on correlation ever make sense? [closed]

In principal component analysis (PCA), one can choose either the covariance matrix or the correlation matrix to find the components (from their respective eigenvectors). These give different results (...
Lucozade's user avatar
  • 659
43 votes
1 answer

Doing principal component analysis or factor analysis on binary data

I have a dataset with a large number of Yes/No responses. Can I use principal components (PCA) or any other data reduction analyses (such as factor analysis) for this type of data? Please advise how I ...
Cathy's user avatar
  • 431
33 votes
1 answer

Best factor extraction methods in factor analysis

SPSS offers several methods of factor extraction: Principal components (which isn't factor analysis at all) Unweighted least squares Generalized least squares Maximum Likelihood Principal Axis Alpha ...
Placidia's user avatar
  • 14.5k
20 votes
3 answers

PCA and exploratory Factor Analysis on the same dataset: differences and similarities; factor model vs PCA

I would like to know if it makes any logical sense to perform principal component analysis (PCA) and exploratory factor analysis (EFA) on the same data set. I have heard professionals expressly ...
user42538's user avatar
  • 301
16 votes
2 answers

Why does sphericity diagnosed by Bartlett's Test mean a PCA is inappropriate?

I understand that Bartlett's Test is concerned with determining if your samples are from populations with equal variances. If the samples are from populations with equal variances, then we fail to ...
tumultous_rooster's user avatar
14 votes
2 answers

In Factor Analysis (or in PCA), what does it mean a factor loading greater than 1?

I've just run a FA using a oblique rotation (promax) and an item yielded a factor loading of 1.041 on one factor, (and factor loadings of -.131, -.119 and .065 on the other factors using pattern ...
Rodrigo M. Rosales's user avatar
11 votes
3 answers

Is it acceptable to have only two (or less) items (variables) loaded by a factor in factor analysis?

I have a set of 20 variables that I have put through factor analysis in SPSS. For purposes of the research, I need to develop 6 factors. SPSS has shown that 8 variables (out of 20) have been loaded ...
Mitja's user avatar
  • 119
6 votes
3 answers

What does it mean if I have high Cronbach alpha, but poor results in Exploratory Factor Analysis

I have been given a survey to analyse. There are 50 questions, and about 400 respondents. I have calculated Cronbach alpha for the entire thing, and I get about 0.9. When I do factor analysis, I do ...
Old_Mortality's user avatar
9 votes
2 answers

Skewed variables in PCA or factor analysis

I want to do principal component analysis (factor analysis) on SPSS based on 22 variables. However, some of my variables are very skewed (skewness calculated from SPSS ranges from 2–80!). So ...
Meo's user avatar
  • 173

15 30 50 per page