Skip to main content
13 votes
Accepted

Why am I getting different PCA loadings for the same data set using two different r packages?

The values you get from FactoMineR are not loadings: they are coordinates. Loadings are coordinates divided by the square root of eigenvalues. I don't know if you can directly get loadings from ...
J-J-J's user avatar
  • 4,934
11 votes

Why am I getting different PCA loadings for the same data set using two different r packages?

Principal components are the eigenvectors of the covariance matrix. Eigenvectors are only defined up to a multiplicative constant. In this case, both results are identical up to a multiplicative ...
cdalitz's user avatar
  • 5,382
8 votes
Accepted

Imaginary numbers in PCA output

A correlation matrix is symmetrical and hence all the eigenvalues are real. I would verify a few things: I would check if the correlation matrix is indeed a symmetric matrix. Is the magnitude large ...
Siong Thye Goh's user avatar
6 votes

Interpreting PCA Output

First, high negative values would be just as key as high positive ones. But, more importantly, I don't think this is a good approach. If you want to use a subset of the original variables in your ...
Peter Flom's user avatar
  • 125k
6 votes

Regarding multiplying PCA values by -1 in R during Principal Component Analysis

The direction of principal components is arbitrary - flipping the sign of any or all dimensions of a principal component decomposition yields an equally valid set of principal components. Principal ...
Nuclear Hoagie's user avatar
6 votes

Are there any situations where orthogonality is not optimal?

You could take any layer of an autoencoder, apply SVD to the inputs and you would just need to apply have a linear transformation to how those inputs were used and you end up with an identical ...
Cliff AB's user avatar
  • 21.4k
6 votes

When is multidimensional scaling exact for a graph?

If the double centration [1, 2] matrix of your distance (dissimilarity) matrix is gramian (positive semidefinite, that is, all eigenvalues nonnegative) with rank m, then it perfectly spans Euclidean m-...
ttnphns's user avatar
  • 58.3k
5 votes

Can you only apply PCA feature reduction only when there are linear relationship between the variables?

I think neither of you is correct. You can do a PCA on any set of data, even random noise. But, for random noise, it won't give you anything. But you talk about having it "make sense" which ...
Peter Flom's user avatar
  • 125k
5 votes

Can you only apply PCA feature reduction only when there are linear relationship between the variables?

I have a new case study on what is essentially nonlinear sparse principal components here. A redundancy analysis is also included, and missing data is handled by stacking multiple imputation-...
Frank Harrell's user avatar
5 votes
Accepted

If I do a PCA on centered only variables (of the same unit), does it mean that the principal components created expresses themselves with that unit?

Principle components are linear combinations of the original variables. The first one has the largest variance, the next has the second largest, and so on, subject to a fixed constraint on each set of ...
David Smith's user avatar
  • 1,368
5 votes

When is multidimensional scaling exact for a graph?

It can't be always true, because the embedding must satisfy the triangle inequality and your graph might not. Necessary and sufficient conditions are known, but you aren't going to like them>
Thomas Lumley's user avatar
4 votes

What kind of data that I need to do my PCA?

PCA does not require normally distributed data. You should take logs if they make substantive sense (this often happens with variables that relate to money). However, extreme outliers and such can ...
Peter Flom's user avatar
  • 125k
4 votes

Interpreting PCA Output

The princmp function in the R Hmisc package is designed to aid in interpreting principal components and sparse PCs. See here ...
Frank Harrell's user avatar
3 votes

Can you only apply PCA feature reduction only when there are linear relationship between the variables?

This is a response to this comment: As far as I understand, you can't do any dimensionality reduction if all the variables are independent. You can perform dimensionality reduction on independent ...
Galen's user avatar
  • 9,381
3 votes

Can PCA factors be correlated?

You are using some nonstandard terminology, but you seem to mean simply the principal components in pca. For terminology see What exactly is called "principal component" in PCA? Then the ...
kjetil b halvorsen's user avatar
3 votes

Feature selection using backward feature selection in scikit-learn and PCA

First: Doing principal components regression is mostly an alternative to lasso, random forest, and so on, not a preliminary step. Second, lasso is a method of adjusting OLS (or other regressions such ...
Peter Flom's user avatar
  • 125k
3 votes

Do features with high variance contribute more to top principal components that explain much of the variance in dataset and vice versa?

Not necessarily. I can't give you the math, but a) I've seen cases where this doesn't happen. b) Intuitively, if you had one variable (or feature) with high variance that was unrelated to any other ...
Peter Flom's user avatar
  • 125k
2 votes

Why is it necessary to eliminate components in PCR in order to 'solve' multicollinearity?

I don't know if it's necessary to remove some PCs, but, with colinear data, the later PCs are probably not going to be useful. Here is an example in R: ...
Peter Flom's user avatar
  • 125k
2 votes

Suggestion for a statistical test

This sounds like an appropriate situation for regression analysis. In this form of analysis you would let the chromatographic efficiency be your response variable and you would let the other ...
Ben's user avatar
  • 129k
2 votes

Geometric Intuition Behind Whitening for ICA

The TL;DR is that whitening isn't essential but it does simplify the task. Specifically, whitening the data reduces the number of parameters under estimation. "Independent Component Analysis: ...
Sycorax's user avatar
  • 92.5k
2 votes
Accepted

Nonlinear PCA vs Encoder in Autoencoders

Could you provide a reference that discusses the use of autoencoders instead of nonlinear PCA? I ask this because I believe autoencoders are not an alternative to nonlinear PCA, but instead an ...
Preston Botter's user avatar
2 votes

Interpreting PCA Output

In the question, you ask about how to get variables associated with each PC, but also state that ultimately what you really need is a set of variables to move forward with. This answer is going to ...
John Madden's user avatar
  • 4,560
2 votes

Whitening/Decorrelation - why does it work?

This is a couple years late, but I think this is a very good question and there are not many clear intuitive answers about it. I could be wrong about some of this, so if someone could verify this, ...
user19402204's user avatar
2 votes

Purpose of expressing data in principal components

When the number of variables is large, relative to the number of observations, lots of problems can happen. PCA reduces the number of variables, while losing as little information as possible. Why do ...
Peter Flom's user avatar
  • 125k
2 votes

post processing in PCA and making sense of an example

I find it useful to decode PCs using stepwise regression. Examples are here and here.
Frank Harrell's user avatar
2 votes

PCA with correlated variables

For the purposes you are considering, this would actually be desirable. When creating a composite, it is generally helpful for the items to be highly correlated with each other to ensure that they ...
Shawn Hemelstrand's user avatar
1 vote

Can you combine two principal components into one variable when carrying out a principal component analysis?

First, it sounds like you want factor analysis not PCA. You have a latent variable (overall size). Finding latent factors is the goal of factor analysis. My favorite professor in grad school used to ...
Peter Flom's user avatar
  • 125k
1 vote
Accepted

Direction of PC1 and PC2 in Principal Component Analysis (PCA)

Take a step back: what is the first sample principal component? It is the linear combination $\mathbf a_1^\top \mathbf y$ that maximizes the corresponding sample variance $\mathbf a_1^\top\mathbf S\...
User1865345's user avatar
  • 9,427
1 vote
Accepted

Variances explained by each feature on PC in PCA

It is helpful to use stepwise regression (which you should avoid at all costs when predicting Y) to provide a concise explanation of what PCs are doing. This is done automatically with the R ...
Frank Harrell's user avatar
1 vote

Analysis of the bias resulting from PCA

The issue is that how the feature-reduced matrix $A$ approximates the full matrix $X$ depends only on the feature values of $X$ and has nothing at all to do with the target $y$. According to the beta ...
Nuclear Hoagie's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible