Linked Questions

0 votes
0 answers

PCA: When using eigendecompostion instead of single value decompostion [duplicate]

If you want to perform a PCA, I guess that using SVD will always work. Eigendecomposition on the covariance matrix only works when your data is not high dimensional(so n > p). But I'm wonder if there ...
statastic's user avatar
  • 311
578 votes
4 answers

Relationship between SVD and PCA. How to use SVD to perform PCA?

Principal component analysis (PCA) is usually explained via an eigen-decomposition of the covariance matrix. However, it can also be performed via singular value decomposition (SVD) of the data matrix ...
amoeba's user avatar
  • 101k
133 votes
7 answers

Is there an intuitive interpretation of $A^TA$ for a data matrix $A$?

For a given data matrix $A$ (with variables in columns and data points in rows), it seems like $A^TA$ plays an important role in statistics. For example, it is an important part of the analytical ...
Alec's user avatar
  • 2,355
129 votes
4 answers

PCA and proportion of variance explained

In general, what is meant by saying that the fraction $x$ of the variance in an analysis like PCA is explained by the first principal component? Can someone explain this intuitively but also give a ...
user9097's user avatar
  • 3,233
82 votes
4 answers

What is the difference between R functions prcomp and princomp?

I compared ?prcomp and ?princomp and found something about Q-mode and R-mode principal component analysis (PCA). But honestly – ...
hans0l0's user avatar
  • 2,125
85 votes
2 answers

What is a "kernel" in plain English?

There are several distinct usages: kernel density estimation kernel trick kernel smoothing Please explain what the "kernel" in them means, in plain English, in your own words.
Neil McGuigan's user avatar
28 votes
2 answers

Is there any advantage of SVD over PCA?

I know how to calculate PCA and SVD mathematically, and I know that both can be applied to Linear Least Squares regression. The main advantage of SVD mathematically seems to be that it can be applied ...
Baz's user avatar
  • 1,743
19 votes
3 answers

What fast algorithms exist for computing truncated SVD?

Possibly off topic here, but there exist several (one, two) related questions already. Poking around in the literature (or a google search for Truncated SVD Algorithms) turns up a lot of papers that ...
John Doucette's user avatar
23 votes
2 answers

Difference between scikit-learn implementations of PCA and TruncatedSVD

I understand the relation between Principal Component Analysis and Singular Value Decomposition at an algebraic/exact level. My question is about the scikit-learn implementation. The documentation ...
drake's user avatar
  • 332
15 votes
1 answer

Steps done in factor analysis compared to steps done in PCA

I know how to perform PCA (principal component analysis), but I would like to know steps that should be used for factor analysis. To perform PCA, let us consider some matrix $A$, for instance: ...
dato datuashvili's user avatar
15 votes
1 answer

Is PCA still done via the eigendecomposition of the covariance matrix when dimensionality is larger than the number of observations?

I have a $20\times100$ matrix $X$, containing my $N=20$ samples in the $D=100$-dimensional space. I now wish to code up my own principal component analysis (PCA) in Matlab. I demean $X$ to $X_0$ first....
Sibbs Gambling's user avatar
9 votes
3 answers

Understanding the output of SVD when used for PCA [duplicate]

I'm doing principal components analysis (PCA) on quite a bit of data (3000 variables, 100079 data points). I'm doing this mostly for fun; data analysis is not my day job. Normally, to do a PCA I ...
N. Virgo's user avatar
  • 395
3 votes
1 answer

PCA vs SVD - understanding difference and preference of SVD over PCA [duplicate]

I understand that PCA and SVD are similar - PCA removes the mean and SVD doesn't? I think I have an understanding of PCA - you would use it to reduce dimensions of data and separate it out into ...
user3788557's user avatar
  • 1,599
3 votes
1 answer

Princomp() outputs seemingly wrong PCA scores with cor=TRUE input argument

Given a 2D data frame d that is centered and scaled: ...
user3022875's user avatar
2 votes
2 answers

Is there any situation where PCA performs better than SVD?

It's for a text clustering application. There are around 25 documents and 50k features (from TF-IDF), so I was expecting SVD to be a better choice. I am using sklearn's PCA and TruncatedSVD functions ...
Leela Prabhu's user avatar

15 30 50 per page