I read book "Data mining methods and models" by Daniel T.Larose chapter 1 for finding correlation in multidimensional dataset by PCA. The chapter explain theory well. I had implemented PCA algorithm in java and also extracted variables as you mentioned in book.

I have one concerned , I want to prioritize/ show the extracted correlated variables from different principal components in deceasing order.

For an example, if first principal component with 1.67 eigen value give me two variables after omitting component weight less than 0.5:

X1 with partial correlation 0.623 X2 with partial correlation 0.567 .

Second principal component with 1.12 eigen value give me three correlated variables after omitting component weight less than 0.5

X1 with 0.98 partial correlation X3 with 0.68 partial correlation X4 with 0.96 partial correlation.

So and so forth might be until first four principal components or principal components that cover 60 % variability. So if i want to show extracted correlation variables from different principal components in descending orders. How i can do that.

Within one principal component is much easy . Variable with highest component weight within same principal component has high correlation than other variable. What about extracted variable from first principal component partial component weight comparison with other extracted variable partial component weight from second principal component. In other words, how i can compare X1 from first principal component and X4 from second principal component.

I read whole chapter, i did not find my answer. Could someone has idea about it ? Kindly suggest me , how in theory i could assign absolute correlation strength to each extracted variable in each principal component.

  • $\begingroup$ I can't think of a good reason you may want to do that. Which is your goal? If you just want patterns of correlations you may want to calculate the correlation matrix (which is the base for a PCA on standardized variables). The correlations you are mentioning are mostly useful to describe the new axes (dimensions) in terms of the original variables. $\endgroup$ – FairMiles Aug 20 '13 at 22:04
  • $\begingroup$ My all variables under consideration except target variable have intercorrelation . So in such a case, when predicators itself have intercorrelation with each other. It cause multicollinearity . In order to avoid multicollinearity, PCA is good technique to solve it. Many people uses it to find target variable correlation with predictors by PCA. statisticalinnovations.com/products/xlstattutorials/… $\endgroup$ – sash Aug 21 '13 at 7:22
  • $\begingroup$ Yes, I know what a PCA is and what it may be used for (though I wouldn't say that it solves multicollineality). What I don't get is why you may want to compare the correlation between (some) variables and the first axis with the correlation of (some) variables with a different axis (that I understand is what you are asking) $\endgroup$ – FairMiles Aug 21 '13 at 19:30
  • $\begingroup$ Let say i have 20 variables in the system e.g. X1,X2,X3,.............Xn. I took let say X1 as my target variable and i want to find correlation strength of all remaining variables (X2,X3,X4............Xn) with target variable X1. I also wanna show these correlation strength in descending order. If X4 has highest correlation among all other variables , it show show on top ,so and so forth. Why i am interested for it , i think it is my requirement in dataset. Why i am using PCA because i know it solve it well but if you know any other technique for same purpose then please let me know. $\endgroup$ – sash Aug 22 '13 at 7:26
  • $\begingroup$ To test for correlations among each of the original p variables with each other you do not want a PCA. You just need the correlation matrix, i.e., a p x p simmetrical matrix with correlations among pairs of variables and 1s in the diagonal (the correlation of a variable with itself). You may want, then, to sort those numbers in order to show the most correlated pair and so on. That is the actual input for most PCAs (in fact, it is the variance-covariance matrix, that results in correlation matrix if you standardize your variables), but you do not seem to want the PCA product. $\endgroup$ – FairMiles Aug 23 '13 at 18:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.