I have a 6 dimensions dataset where I want to apply PCA to remove one dimension.

I did a small analysis to check for relationships in my data and concluded that there is very low linear correlation between my variables (maximum is 0.4 for two of them).

Since my data seem to be very poorly linearly correlated, does it make sense to use PCA here?

To illustrate my concerns, I drew a 2D example. Let’s say I have the below distribution.

PCA gives the two eigenvectors showed below. The method that consists in keeping the vector with the highest magnitude (say the green vector) would lead to a poor fit i.e. projecting the points on this axe would remove important information of the data.

After reading quite some materials on PCA, I am surprised that I’ve never read about checking linearity before applying such method. Is this a strong recommended prerequisite? Can PCA be efficiently applied on any distribution?

  • 1
    $\begingroup$ This is not a prerequisite; but linear PCA is linear by algorithm and it gives you straight eigenvectors as you've shown. $\endgroup$
    – ttnphns
    Nov 23 '21 at 11:41
  • $\begingroup$ It is unclear what you might even mean by "linearity" in this context. Evidently it has something to do with the uneven spatial distribution of the points in the scatterplot, but it is difficult to determine what specific quality of that distribution you are thinking of. $\endgroup$
    – whuber
    Nov 23 '21 at 16:25
  • $\begingroup$ I usually measure linearity with Pearson coefficient. In the case of my scatterplot, it is obviously very low (10%). $\endgroup$
    – savoga
    Nov 25 '21 at 8:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.