I understand why ridge regression (equivalent to using a Gaussian prior on the coefficients in a Bayesian setting) works well in the presence of multi-collinearity, but couldn't one argue that other shrinking priors would also help? (e.g. say a Laplace prior, as in LASSO).

Moreover, if OLS is effectively equivalent to using a uniform prior (with a Gaussian likelihood), what type of priors can arguably with multi-collinearity? In other words, what conditions does a prior have to have to help with multi-collinearity?

  • 2
    $\begingroup$ In terms of shrinkage and multicollinearity, elastic net regression gives you the benefits of LASSO and ridge regression at the cost of an additional hyperparameter. $\endgroup$ – Sycorax Apr 24 '17 at 22:21
  • $\begingroup$ Thanks @Sycorax I understand one can obviously combine both with Elastic Net. I am particularly interested in the relationship these priors have with multi-collinearity (I have updated the OP to clarify this point). $\endgroup$ – Josh Apr 25 '17 at 12:12
  • 1
    $\begingroup$ If I understand correctly, it seems like any unimodal prior may work. All else equal, the selected fit would be the one which is most probable under the prior. $\endgroup$ – user795305 Apr 30 '17 at 17:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.