Hot answers tagged

20 votes

Practical usefulness of PCA

One important use of PCA is in analysis of electroencephalography (EEG) data. To measure an EEG, dozens of electrodes are attached to your scalp and measure electric currents in your brain, either at ...
13 votes
Accepted

Why am I getting different PCA loadings for the same data set using two different r packages?

The values you get from FactoMineR are not loadings: they are coordinates. Loadings are coordinates divided by the square root of eigenvalues. I don't know if you can directly get loadings from ...
J-J-J's user avatar
  • 3,862
11 votes
Accepted

Is it possible to turn PCA into ICA by rotating the eigenvectors?

No, in general, you can't rotate the principal components to obtain ICA. One of the defining traits of PCA is that the component directions are orthogonal. If you rotate the principal components, they'...
Sycorax's user avatar
  • 90.4k
11 votes

Why am I getting different PCA loadings for the same data set using two different r packages?

Principal components are the eigenvectors of the covariance matrix. Eigenvectors are only defined up to a multiplicative constant. In this case, both results are identical up to a multiplicative ...
cdalitz's user avatar
  • 5,002
9 votes

Practical usefulness of PCA

First, from the perspective of education, PCA is a good entryway to the world of dimension-reduction techniques and associated methods. Whether we're talking ICA, non-negative matrix factorization, ...
8 votes

Practical usefulness of PCA

I use three examples in my lectures to illustrate what PCA can do (click the links for pointers to the slides). They're chosen to show how useful it is in general data science practice and how ...
6 votes
Accepted

Does the average eigenvalue equals 1 in PCA applied to standardised data?

Definitely yes. To see this let's consider the population case. Suppose $X$ is a $p\times 1$ random vector with mean $\mu$ and covariance matrix $\Sigma$ and consider $Y = X\text{diag}(\Sigma)^{-1/2}$....
utobi's user avatar
  • 11.6k
6 votes

Interpreting PCA Output

First, high negative values would be just as key as high positive ones. But, more importantly, I don't think this is a good approach. If you want to use a subset of the original variables in your ...
Peter Flom's user avatar
  • 117k
6 votes
Accepted

Imaginary numbers in PCA output

A correlation matrix is symmetrical and hence all the eigenvalues are real. I would verify a few things: I would check if the correlation matrix is indeed a symmetric matrix. Is the magnitude large ...
Siong Thye Goh's user avatar
5 votes

Practical usefulness of PCA

One application of PCA that I have used a few times is the construction of social indicators. We use the projection of each observation (usually households) over a component axis (usually the first), ...
5 votes

Can you only apply PCA feature reduction only when there are linear relationship between the variables?

I think neither of you is correct. You can do a PCA on any set of data, even random noise. But, for random noise, it won't give you anything. But you talk about having it "make sense" which ...
Peter Flom's user avatar
  • 117k
5 votes

Can you only apply PCA feature reduction only when there are linear relationship between the variables?

I have a new case study on what is essentially nonlinear sparse principal components here. A redundancy analysis is also included, and missing data is handled by stacking multiple imputation-...
Frank Harrell's user avatar
5 votes

Regarding multiplying PCA values by -1 in R during Principal Component Analysis

The direction of principal components is arbitrary - flipping the sign of any or all dimensions of a principal component decomposition yields an equally valid set of principal components. Principal ...
Nuclear Hoagie's user avatar
4 votes

Practical usefulness of PCA

It is absolutely not clear, obvious, or agreed-upon what "intelligence" is in humans (or, for that matter, in nonhuman animals). What is clear is that people's performance on a variety of ...
4 votes
Accepted

Confused by an article on princomp in R

If some online tutorial disagrees with the official documentation of the software, almost always you should trust the documentation. What the documentation says is: Description ...
Tim's user avatar
  • 138k
4 votes

Interpreting PCA Output

The princmp function in the R Hmisc package is designed to aid in interpreting principal components and sparse PCs. See here ...
Frank Harrell's user avatar
4 votes
Accepted

If I do a PCA on centered only variables (of the same unit), does it mean that the principal components created expresses themselves with that unit?

Principle components are linear combinations of the original variables. The first one has the largest variance, the next has the second largest, and so on, subject to a fixed constraint on each set of ...
David Smith's user avatar
  • 1,188
4 votes

What kind of data that I need to do my PCA?

PCA does not require normally distributed data. You should take logs if they make substantive sense (this often happens with variables that relate to money). However, extreme outliers and such can ...
Peter Flom's user avatar
  • 117k
3 votes

Practical usefulness of PCA

PCA just comes down to using the eigendecomposition of the (empirical) covariance matrix of the data. The full eigendecomposition of the covariance matrix results in a set of eigenvectors and ...
3 votes

Practical usefulness of PCA

I've used PCA in facial motion capture for real time animatronic control of the 'lots of dots on a face' variety. I was able to find out which dots - which is to say regions of the face - encoded the ...
3 votes
Accepted

PCA via a Neural Network

Just doing PCA inside of a neural network is not much of a stretch, since the most naïve implementation will simply employ gradient updates to compute the $QR$ algorithm for the covariance matrix. It'...
Sycorax's user avatar
  • 90.4k
3 votes
Accepted

Which features corresponds to which eigenvalues when use SVD in PCA?

Let $\mathbb{X}$ be the $n\times p$ matrix of observations which has been centred so that each of its columns has an average 0. Now consider the singular value decomposition (SVD) $$ \mathbb{X} = U D ...
utobi's user avatar
  • 11.6k
3 votes

Can you only apply PCA feature reduction only when there are linear relationship between the variables?

This is a response to this comment: As far as I understand, you can't do any dimensionality reduction if all the variables are independent. You can perform dimensionality reduction on independent ...
Galen's user avatar
  • 8,165
3 votes

Feature selection using backward feature selection in scikit-learn and PCA

First: Doing principal components regression is mostly an alternative to lasso, random forest, and so on, not a preliminary step. Second, lasso is a method of adjusting OLS (or other regressions such ...
Peter Flom's user avatar
  • 117k
2 votes

Relationship between SVD and PCA. How to use SVD to perform PCA?

Let's try to understand using a data matrix $X$ of dimension $n \times d$, where $d \gg n$ and $rank(X)=n$. Then $\underset{n \times d}{X}=\underset{n \times n}{U}\underset{n \times n}{\Sigma} \...
Sandipan Dey's user avatar
2 votes

Whitening/Decorrelation - why does it work?

This is a couple years late, but I think this is a very good question and there are not many clear intuitive answers about it. I could be wrong about some of this, so if someone could verify this, ...
user19402204's user avatar
2 votes

Why is Pearson Principal Component Analysis so much more common than Spearman or Kendall PCA?

Let's start from the correlations. Pearson's correlation is a linear measure and the hypothesis test for it (cor.test in R, for example) assumes (approximate) ...
András Aszódi's user avatar
2 votes

Can PCA factors be correlated?

You are using some nonstandard terminology, but you seem to mean simply the principal components in pca. For terminology see What exactly is called "principal component" in PCA? Then the ...
kjetil b halvorsen's user avatar
2 votes

Why is it necessary to eliminate components in PCR in order to 'solve' multicollinearity?

I don't know if it's necessary to remove some PCs, but, with colinear data, the later PCs are probably not going to be useful. Here is an example in R: ...
Peter Flom's user avatar
  • 117k
2 votes

Suggestion for a statistical test

This sounds like an appropriate situation for regression analysis. In this form of analysis you would let the chromatographic efficiency be your response variable and you would let the other ...
Ben's user avatar
  • 123k

Only top scored, non community-wiki answers of a minimum length are eligible