5
$\begingroup$

From what I've studied, the LRT is used to compare two nested models, i.e. 2 models having different sets of nested features, in my case e.g.

  • Model1: binary_outcome ~ X1 + X2
  • Model2: binary_outcome ~ X1 + X2 + X3 + X4

All the examples I could find online used simple linear/logistic regressions.

I'd like to use LASSO to prevent overfitting since I have a lot of features. However, I also have high collinearity because many features are correlated. I also read that LASSO is unstable with highly correlated features...

So my question is: Does performing a LRT test of two nested LASSO models make statistical sense?

$\endgroup$

1 Answer 1

5
$\begingroup$

In the frequentist world of traditional statistics, penalized maximum likelihood is somewhat of an ad hoc procedure with limited inferential methods available after the fit. Likelihood ratio tests don’t know about the penalization so are not really valid. Contrast this with Bayesian models where penalization uses shrinkage priors and the posterior distribution is a regular posterior distribution with all the simple inferential methods that provides. Bayes also gives you the ability to use more sensible shrinkage (e.g., horseshoe priors) than what lasso uses.

$\endgroup$
2
  • $\begingroup$ +1 but I do wonder if it makes sense to do the LRT, just that it is difficult to can a handle on what the degrees of freedom should be in the $\chi^2$ distribution (think about if you cross-validate the LASSO penalty). $\endgroup$
    – Dave
    Commented Jan 25 at 15:30
  • 1
    $\begingroup$ To my knowledge no one has created a correct LRT when regression parameters are purposefully biased (penalized towards zero). Some recommend to use the unpenalized model for statistical testing. All of this is another reason to use Bayesian modeling. $\endgroup$ Commented Jan 25 at 18:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.