I have 200 continuous variables, all positively correlated to the dependent variable. My problem is that I can't exclude any from my model as I need to attribute to each and everyone a weight. They are also correlated between themselves. RF is great for prediction but does not provide weights,Linear has the issue of multicolinearity and Ridge doesn't help much. Any suggestion on how to best tackle this problem?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
4
If you're not concerned with interpretability, you can try Principal Components Analysis to extract a low-dimensional subspace with maximal variation, but I'm surprised ridge regression didn't help with the multicollinearity. Perhaps try LASSO instead?
See also answers here:
Dealing with correlated regressors
https://stats.stackexchange.com/a/38095/135759
How to deal with multicollinearity when performing variable selection?
answered Nov 28, 2016 at 13:55
-
$\begingroup$ Seems more a comment than an answer. Can you elaborate a little bit further? We are looking for high-quality answers. $\endgroup$– utobiNov 28, 2016 at 14:12
-
$\begingroup$ Thank you for answering but I am indeed concerned with interpretability. $\endgroup$ Nov 28, 2016 at 14:18
-
$\begingroup$ @utobi I would, but low rep doesn't allow me to post comments. $\endgroup$ Nov 28, 2016 at 14:18
-
$\begingroup$ For commenting you need to wait until you've got enough reputation points that you gain by placing proper answers. $\endgroup$– utobiNov 28, 2016 at 14:22