# Non positive definitive correlation matrix - how to proceed?

I have a correlation matrix where by for a good chunk of the data set the correlations for all pairs of variables are more correlated than +/-0.5 Infact many of these are more correlated than +/-0.75

The program i have reports the correlation matrix is not positive definite. Therefore principle components (although generated) seem to be collinear when examined in regression

I have provided a sample of the correlation matrix

http://imgur.com/44ddFIW

• The collinearity is not just "seeming;" it is real. I see some "1.0000" values on the off-diagonals and some other numbers very close to that, so at least to this level of precision you have perfectly redundant data. Any algorithm to process this matrix will produce eigenvalues extremely close to zero; any machine-precision algorithm will be as likely to represent such near-zero values as tiny negative numbers as tiny positive numbers. – whuber Feb 18 '14 at 17:35
• That only means there is still real collinearity. Not all collinearity is as evident as the presence of ones in the off-diagonal entries. Anyway, what is your question? There's nothing unusual or wrong with collinear independent variables in regression problems. What exactly do you mean, then, by "proceed"? – whuber Feb 20 '14 at 15:32
• I am really attempting to try and use PCA to reduce the number of variables i use to model my dependent response variable by using a number of methods from linear to neural nets. My other option really is to remove variables by judgement. – Samuel Feb 20 '14 at 16:50
• You might be very interested, then, in the recent answer at stats.stackexchange.com/a/87231, which discusses exactly this issue. It points out that because PCA on the independent variables has nothing to do with how they might be related to the dependent variable, you risk throwing out important variables. Your case is probably not quite that bad, because some of the near-zero eigenvalues really are just "noise" due to floating point rounding error. But after you identify and remove those components you will need to think hard about whether to remove any others. – whuber Feb 20 '14 at 17:10
• Thanks for the link. Yes the lack of relationship to the dependent var is a problem. I have seen the same effect as in the references. where, say, PC 8 of 10 may be the most significant predictor out of all PCs. The current data has PCs with eigenvalues ~0 have no loadings of more than 0.1-0.2 in them. Therefore i thought it safe to disregard these. Often the more PCs i generate the easier they are to interpret where one or two variables may load highly onto one PC. To sum up my current thought track..does it seem acceptable therefore to pick those with high var loadings? – Samuel Feb 21 '14 at 10:30