20k views

### How does Factor Analysis explain the covariance while PCA explains the variance?

Here is a quote from Bishop's "Pattern Recognition and Machine Learning" book, section 12.2.4 "Factor analysis": According to the highlighted part, factor analysis captures the covariance between ...
28k views

### What are the assumptions of factor analysis?

I want to check if I really understood [classic, linear] factor analysis (FA), especially assumptions that are made before (and possibly after) FA. Some of the data should be initially correlated and ...
54k views

### When can we speak of collinearity

In linear models we need to check if a relationship exists among the explanatory variables. If they correlate too much then there is collinearity (i.e., the variables partly explain each other). I am ...
33k views

### Do I need to drop variables that are correlated/collinear before running kmeans?

I am running kmeans to identify clusters of customers. I have approximately 100 variables to identify clusters. Each of these variables represent the % of spend by a customer on a category. So, if I ...
27k views

### Can the scaling values in a linear discriminant analysis (LDA) be used to plot explanatory variables on the linear discriminants?

Using a biplot of values obtained through principal component analysis, it is possible to explore the explanatory variables that make up each principle component. Is this also possible with Linear ...
1k views

I am often disappointed with PCA plots in the scientific literature. Typically PCA plots do not provide a breakdown of the variables and their weights, just something like PCA1 (70% variance explained)...
6k views

### How to interpret this PCA biplot coming from a survey of what areas people are interested in?

Background: I asked hundreds of participants in my survey how much they are interested in selected areas (by five point Likert scales with 1 indicating "not interested" and 5 indicating "interested"). ...
11k views

### What are the units in this PCA biplot? [duplicate]

This is a plot of my data These are the values: ...
4k views

### Can PCA allow to identify redundant variables that can be removed before doing cluster analysis?

I hope this is suitable for this forum: I am new to PCA and what I ultimately want to do is perform cluster analysis on my dataset. I have 20 physical descriptor variables for organisms, each with 300 ... 1k views

### PCA, dimensionality, and k-means results: reaction to duplicating of variables

There are many excellent conversations on CV about the curse of dimensionality when applied to methods like k-means. The answer in the same post and other research (e.g., the paper titled "When Is ‘...
1k views

### Which font to use in a plot to maximize clarity?

I was reading the Wikipedia page of the Bitstream Vera font (used by default by matplotlib), which says: The Bitstream Vera Sans Mono typeface in particular is suitable for technical work, as it ...
3k views

### Geometric understanding of PCA in the subject (dual) space

I am trying to get an intuitive understanding of how principal component analysis (PCA) works in subject (dual) space. Consider 2D dataset with two variables, $x_1$ and $x_2$, and $n$ data points (...
4k views

### What can be the reason to do feature selection based on variance before doing PCA?

I have noticed that when applying PCA to large datasets, people often will first subset the data considerably. Sometimes people just randomly take a subset of the features/variables, but often they ...
2k views

### Interpreting overlapping arrows on a PCA biplot: does it mean that the variables are redundant?

I'm new in principal component analysis (PCA) and I don't really understand the biplot representation of its results, so I would really appreciate some guidance. Having the example of the illustration ...
I'm running a PCA over a data set of $N \times p$ size ($N\approx 1000$ being the number of measurements and $p\approx 200$ being the number of dimensions/predictors). I expect many of the predictors ...