I have a 6 dimensions dataset where I want to apply PCA to remove one dimension.
I did a small analysis to check for relationships in my data and concluded that there is very low linear correlation between my variables (maximum is 0.4 for two of them).
Since my data seem to be very poorly linearly correlated, does it make sense to use PCA here?
To illustrate my concerns, I drew a 2D example. Let’s say I have the below distribution.
PCA gives the two eigenvectors showed below. The method that consists in keeping the vector with the highest magnitude (say the green vector) would lead to a poor fit i.e. projecting the points on this axe would remove important information of the data.
After reading quite some materials on PCA, I am surprised that I’ve never read about checking linearity before applying such method. Is this a strong recommended prerequisite? Can PCA be efficiently applied on any distribution?