I have a dataset with 500+ variables and have done PCA.
There is a particular Principal Component that is correlated more than the others with an Event of Interest -- a binary variable -- but still is one of the least important Principal Components ranking at the 425 position.
I scaled the loadings of these Principal Component to be able to discern more clearly which Variables contribute significantly more --either in a positive or negative way-- to the formation of this Principal Component.
There is a Variable measuring the number of Mobile Transactions and there is a variable that is the Square of the Former (say X and X^2).
Variable X has a scaled loading value of 2.33 (2.33 standard deviations from the mean which is zero) and the Variable X^2 has a scaled loading value of -1.4.
Since the loadings have been scaled and these numbers represent std. deviations, the information they convey is by how much the loadings of X and X^2 respectively are larger or smaller from the average loadings of all the Variables in this Principal Component. So the specific values 2.3 and -1.4 tells us that on the one hand X contributes much more to the Principal Component compared to the other Variables of the Dataset and on the other hand conversely X^2 contributes much less. But how can this be possible? If X is important should not also X^2 be important --and actually more so?
Your advice will be appreciated.
The loadings are returned from the R function prcomp. The loadings represent the coordinates of the original variables on the new axes (The Principal Component axes). We can use the loadings matrix and its inverse to map points in the original feature space to the transformed PC space and vice versa.
The loadings are derived using the Covariance matrix of the original dataset.
However you interpret them though, my question remains: How can X have a high loading on a given Principal Component and X^2 have a low loading on the same PC?