Tag Info

Hot answers tagged pca

Accepted

Why am I getting different PCA loadings for the same data set using two different r packages?

The values you get from FactoMineR are not loadings: they are coordinates. Loadings are coordinates divided by the square root of eigenvalues. I don't know if you can directly get loadings from ...
• 4,934

Why am I getting different PCA loadings for the same data set using two different r packages?

Principal components are the eigenvectors of the covariance matrix. Eigenvectors are only defined up to a multiplicative constant. In this case, both results are identical up to a multiplicative ...
• 5,382
Accepted

Imaginary numbers in PCA output

A correlation matrix is symmetrical and hence all the eigenvalues are real. I would verify a few things: I would check if the correlation matrix is indeed a symmetric matrix. Is the magnitude large ...
• 6,871

Interpreting PCA Output

First, high negative values would be just as key as high positive ones. But, more importantly, I don't think this is a good approach. If you want to use a subset of the original variables in your ...
• 125k

Regarding multiplying PCA values by -1 in R during Principal Component Analysis

The direction of principal components is arbitrary - flipping the sign of any or all dimensions of a principal component decomposition yields an equally valid set of principal components. Principal ...
• 9,900

Are there any situations where orthogonality is not optimal?

You could take any layer of an autoencoder, apply SVD to the inputs and you would just need to apply have a linear transformation to how those inputs were used and you end up with an identical ...
• 21.4k

When is multidimensional scaling exact for a graph?

If the double centration [1, 2] matrix of your distance (dissimilarity) matrix is gramian (positive semidefinite, that is, all eigenvalues nonnegative) with rank m, then it perfectly spans Euclidean m-...
• 58.3k

Can you only apply PCA feature reduction only when there are linear relationship between the variables?

I think neither of you is correct. You can do a PCA on any set of data, even random noise. But, for random noise, it won't give you anything. But you talk about having it "make sense" which ...
• 125k

Can you only apply PCA feature reduction only when there are linear relationship between the variables?

I have a new case study on what is essentially nonlinear sparse principal components here. A redundancy analysis is also included, and missing data is handled by stacking multiple imputation-...
• 95.7k
Accepted

If I do a PCA on centered only variables (of the same unit), does it mean that the principal components created expresses themselves with that unit?

Principle components are linear combinations of the original variables. The first one has the largest variance, the next has the second largest, and so on, subject to a fixed constraint on each set of ...
• 1,368

When is multidimensional scaling exact for a graph?

It can't be always true, because the embedding must satisfy the triangle inequality and your graph might not. Necessary and sufficient conditions are known, but you aren't going to like them>
• 41.6k

What kind of data that I need to do my PCA?

PCA does not require normally distributed data. You should take logs if they make substantive sense (this often happens with variables that relate to money). However, extreme outliers and such can ...
• 125k

Interpreting PCA Output

The princmp function in the R Hmisc package is designed to aid in interpreting principal components and sparse PCs. See here ...
• 95.7k

Can you only apply PCA feature reduction only when there are linear relationship between the variables?

This is a response to this comment: As far as I understand, you can't do any dimensionality reduction if all the variables are independent. You can perform dimensionality reduction on independent ...
• 9,381

Can PCA factors be correlated?

You are using some nonstandard terminology, but you seem to mean simply the principal components in pca. For terminology see What exactly is called "principal component" in PCA? Then the ...
• 81.4k

Feature selection using backward feature selection in scikit-learn and PCA

First: Doing principal components regression is mostly an alternative to lasso, random forest, and so on, not a preliminary step. Second, lasso is a method of adjusting OLS (or other regressions such ...
• 125k

Do features with high variance contribute more to top principal components that explain much of the variance in dataset and vice versa?

Not necessarily. I can't give you the math, but a) I've seen cases where this doesn't happen. b) Intuitively, if you had one variable (or feature) with high variance that was unrelated to any other ...
• 125k

Why is it necessary to eliminate components in PCR in order to 'solve' multicollinearity?

I don't know if it's necessary to remove some PCs, but, with colinear data, the later PCs are probably not going to be useful. Here is an example in R: ...
• 125k

Suggestion for a statistical test

This sounds like an appropriate situation for regression analysis. In this form of analysis you would let the chromatographic efficiency be your response variable and you would let the other ...
• 129k

Geometric Intuition Behind Whitening for ICA

The TL;DR is that whitening isn't essential but it does simplify the task. Specifically, whitening the data reduces the number of parameters under estimation. "Independent Component Analysis: ...
• 92.5k
Accepted

Nonlinear PCA vs Encoder in Autoencoders

Could you provide a reference that discusses the use of autoencoders instead of nonlinear PCA? I ask this because I believe autoencoders are not an alternative to nonlinear PCA, but instead an ...
• 2,856

Interpreting PCA Output

In the question, you ask about how to get variables associated with each PC, but also state that ultimately what you really need is a set of variables to move forward with. This answer is going to ...
• 4,560

Whitening/Decorrelation - why does it work?

This is a couple years late, but I think this is a very good question and there are not many clear intuitive answers about it. I could be wrong about some of this, so if someone could verify this, ...

Purpose of expressing data in principal components

When the number of variables is large, relative to the number of observations, lots of problems can happen. PCA reduces the number of variables, while losing as little information as possible. Why do ...
• 125k

post processing in PCA and making sense of an example

I find it useful to decode PCs using stepwise regression. Examples are here and here.
• 95.7k

PCA with correlated variables

For the purposes you are considering, this would actually be desirable. When creating a composite, it is generally helpful for the items to be highly correlated with each other to ensure that they ...
• 15.8k
1 vote

Can you combine two principal components into one variable when carrying out a principal component analysis?

First, it sounds like you want factor analysis not PCA. You have a latent variable (overall size). Finding latent factors is the goal of factor analysis. My favorite professor in grad school used to ...
• 125k
1 vote
Accepted

Take a step back: what is the first sample principal component? It is the linear combination $\mathbf a_1^\top \mathbf y$ that maximizes the corresponding sample variance $\mathbf a_1^\top\mathbf S\... • 9,427 1 vote Accepted Variances explained by each feature on PC in PCA It is helpful to use stepwise regression (which you should avoid at all costs when predicting Y) to provide a concise explanation of what PCs are doing. This is done automatically with the R ... • 95.7k 1 vote Analysis of the bias resulting from PCA The issue is that how the feature-reduced matrix$A$approximates the full matrix$X$depends only on the feature values of$X$and has nothing at all to do with the target$y\$. According to the beta ...
• 9,900

Only top scored, non community-wiki answers of a minimum length are eligible