1
$\begingroup$

My understanding is that regularization will generally help prediction tasks. What about for situations where we want to conduct a study to understand the effect of a specific predictor on the dependent variable? Is it fair to say that we should use multiple regression instead of something like ridge in this case since the multiple regression coefficients aren't biased?

Correct me if I'm wrong, but I think what I'm basically trying to ask if there is any place for Ridge Regression in inference tasks?

|cite|improve this question
$\endgroup$
  • $\begingroup$ Nothing stops you from using ridge in a multiple instance regression setting (i.e. with multiple predictors). I think most people would use Lasso instead of ridge in the case where they want to identify which predictor matters. But in general, if your model does not do a good job at generalising, it would not make sense to quantify importance of predictors. $\endgroup$ – Tom May 22 at 16:40
  • 1
    $\begingroup$ There is no choice between ridge regression and multiple regression: either, both, or neither may be applied in any circumstance. $\endgroup$ – whuber♦ May 22 at 16:46
1
$\begingroup$

Ridge regression does not need to be all-or-none with respect to penalization. It's quite acceptable to penalize covariates that are likely to be important to outcome but not of primary interest in terms of inference. Chen et al show that:

Ridge regression and penalized logistic regression models that penalize all but the coefficient of the exposure may be considered in these types of studies.

Even with penalization of all predictors, it's possible to try bootstrapping to identify predictors that are highly related to outcome. See this page for some ideas.

|cite|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.