Linked Questions

0
votes
0answers
41 views

Is it possible that PCA works better without data scaling? [duplicate]

I am a beginner. I have a dataset of 1700 samples with 4 features and I have to perform Hierarchical Clustering (the agglomerative version) and I need to decide whether or not to scale the data and ...
151
votes
7answers
114k views

PCA on correlation or covariance?

What are the main differences between performing principal component analysis (PCA) on the correlation matrix and on the covariance matrix? Do they give the same results?
107
votes
6answers
12k views

Is there an intuitive interpretation of $A^TA$ for a data matrix $A$?

For a given data matrix $A$ (with variables in columns and data points in rows), it seems like $A^TA$ plays an important role in statistics. For example, it is an important part of the analytical ...
36
votes
1answer
18k views

How does centering the data get rid of the intercept in regression and PCA?

I keep reading about instances where we center the data (e.g., with regularization or PCA) in order to remove the intercept (as mentioned in this question). I know it's simple, but I'm having a hard ...
18
votes
3answers
18k views

Why does gap statistic for k-means suggest one cluster, even though there are obviously two of them?

I am using K-means to cluster my data and was looking for a way to suggest an "optimal" cluster number. Gap statistics seems to be a common way to find a good cluster number. For some reason it ...
11
votes
3answers
10k views

What are the assumptions of factor analysis?

I want to check if I really understood [classic, linear] factor analysis (FA), especially assumptions that are made before (and possibly after) FA. Some of the data should be initially correlated and ...
12
votes
1answer
17k views

Steps done in factor analysis compared to steps done in PCA

I know how to perform PCA (principal component analysis), but I would like to know steps that should be used for factor analysis. To perform PCA, let us consider some matrix $A$, for instance: ...
15
votes
1answer
13k views

What is the proper association measure of a variable with a PCA component (on a biplot / loading plot)?

I am using FactoMineR to reduce my data set of measurements to the latent variables. The variable map above is clear for me to interpret, but I am confused when ...
8
votes
1answer
9k views

Questions on PCA: when are PCs independent? why is PCA sensitive to scaling? why are PCs constrained to be orthogonal?

I am trying to understand some descriptions of PCA (the first two are from Wikipedia), emphasis added: Principal components are guaranteed to be independent only if the data set is jointly normally ...
9
votes
1answer
8k views

Standardizing features when using LDA as a pre-processing step

If a multi-class Linear Discriminant Analysis (or I also read Multiple Discriminant Analysis sometimes) is used for dimensionality reduction (or transformation after dimensionality reduction via PCA), ...
10
votes
1answer
1k views

Confused about the visual explanation of eigenvectors: how can visually different datasets have the same eigenvectors?

A lot of statistics textbooks provide an intuitive illustration of what the eigenvectors of a covariance matrix are: The vectors u and z form the eigenvectors (well, eigenaxes). This makes sense. But ...
6
votes
0answers
8k views

How to normalize data prior to computation of covariance matrix

In all my self-study, I have come across many different ways in which people seem to normalize their data, prior to the computation of the covariance matrix. I am confused as to what ways are 'correct'...
3
votes
2answers
2k views

Is common factor analysis ever performed using the covariance matrix?

Common factor analysis entails the eigendecomposition and subsequent interpretation of $\mathbf{C}$ which is often described the correlation matrix with the diagonal elements replaced with approximate ...
3
votes
1answer
1k views

Is continuous inputs an assumption of factor analysis?

Should we use only continuous inputs for factor analysis (FA)? My data is a mix of continuous and categorical inputs: one of the inputs has only 600, 700 and 1000 as values. I found that principal ...
1
vote
2answers
2k views

Using PCA in Matlab: Is it based on the covariance or correlation matrix?

I want to produce a scree plot to assess if there is an 'elbow' in the eigenvalues to aid in my identification of the number of PCs to retain. However, upon reading further into the topic, I realised ...

15 30 50 per page