Linked Questions

1191
votes
28answers
753k views

Making sense of principal component analysis, eigenvectors & eigenvalues

In today's pattern recognition class my professor talked about PCA, eigenvectors and eigenvalues. I understood the mathematics of it. If I'm asked to find eigenvalues etc. I'll do it correctly like ...
55
votes
7answers
29k views

Best PCA algorithm for huge number of features (>10K)?

I previously asked this on StackOverflow, but it seems like it might be more appropriate here, given that it didn't get any answers on SO. It's kind of at the intersection between statistics and ...
11
votes
1answer
5k views

Is large scale PCA even possible?

Principal component analysis' (PCA) classical way is to do it on an input data matrix which columns have zero mean (then PCA can "maximize variance"). This can be achieved easily by centering the ...
2
votes
1answer
5k views

PCA demeaning the data [duplicate]

What is the motivation for demeaning the data when doing PCA. I've been told to do it, but I've never heard a good and/or intuitive reason for it. Is this a case where doing it just makes the math ...
3
votes
2answers
2k views

Using the 'U' Matrix of SVD as Feature Reduction [duplicate]

This is a follow-up to the question asked regarding SVD and dimensionality reduction (question). In that question I asked how to use SVD for dimensionality reduction. Although not stated, the ...
3
votes
2answers
2k views

In PCA, do we have to center and normalize eigenvectors or solely to normalize them?

Given such a matrix about the grades of 6 students in Maths, Computer Sciences and French: $$A=\begin{bmatrix} 1 & 0 & 0 \\ 0 & 0 & 1 \\ 0 & 1 & 2 \\ 2 & 2 ...
7
votes
1answer
722 views

Scores are still correlated after PCA

I have 22 variables with more than 6000 observations. They are highly correlated. I know these data would work as great explanatory variables to a dichotomous event (present absent). Therefore, I ...
2
votes
1answer
410 views

Approaches to reduce dimensions (feature selection/extraction) with high dimensional count data before running tree based model

My dataset has ~100k samples and 3000 dimensions. The data are counts, anywhere between 0-8 and it's pretty sparse. Because of 'curse of high dimension', I want to shrink the number of variables ...
1
vote
1answer
148 views

Testing of hypothesis for the linearity of a data? PCA suggested, but how do we design a statistical test using it? [closed]

Suppose we're given the data set $\{x_1 \dots x_n\}$ in $\mathbb{R}^D$ the $D$-dimensional Euclidean space, and assume this data has intrinsic dimension $d < D.$ N.B. this just means that data is ...
0
votes
0answers
100 views

Centering variables before running PCA [duplicate]

I am learning about PCA, regarding PCA I need to know that given a dataset is it always necessary to use centering? what if I don't center the variables used in PCA?
2
votes
1answer
77 views

Scaling a sparse matrix

I want to apply sparse PCA to a sparse matrix. I was wondering if scaling to mean 0 and unit variance would be appropriate given that my input is sparse?