I have large correlation matrix in Excel that I'd like to use to inform my choice of explanatory variables in a multiple linear regression model. One problem is that the initial data was very sparse, and some columns had significantly more zeroes than others. How would I go about choosing the variables with the lowest correlation without falling into the trap of choosing those that have the lowest pairwise correlation solely because they were both very sparse?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
I am a little confused by the question. Here are a few pointers, a few of which I hope will help you
1) If variable selection is your goal, you could use a LASSO-type l1-penalized scheme to do the job. Look up glmnet for more details.
2) If you have a sparse covariance matrix, why do you care about closeness to 0.If anything because data is so sparse, the general practice is to in some sense "ignore" the small correlations so that there are more independence assumptions in the model. Examples of this kind of thing include naive bays, bayes net etc.
3) When designing a sparse covariance matrix (if you think in a frequentist sense), the model penalizes the matrix for having non zero entries so it is rare to see entries very "close" to 0
-
$\begingroup$ With respect to your second and third points I was not clear. I meant that I have a matrix, many of whose columns are sparse, and I'm just running the correlation matrix procedure on this matrix. These will have very little correlation with columns whose values are largely non-zero but this would not be a good choice of variable because of the meaning of the data. $\endgroup$– 114Commented Jul 22, 2014 at 21:44
-
$\begingroup$ Ahh, I see..Well there are a couple of things people do. In a lot of the cases they chose what "small" is supposed to mean and filter of at that (0.05 etc.). A method I prefer is to do a sparse-low rank reconstruction and then keep whatever entries remain $\endgroup$– SidCommented Jul 22, 2014 at 23:36