3
$\begingroup$

Assume a linear regression problem where I want to force sparsity of some parameters. However, due to some physics, I know that one of my parameters is always positive. For instance, I have that

$$ y=\sum \beta_ix_i+\epsilon $$ where $\beta_5\geq0$

Is it safe to find the parameter estimates through maximizing the penalized likelihood below while just adding the constraint $\beta_5\geq0$

$$l_p=l(\boldsymbol\beta)+\lambda \sum |\beta_i|$$

By safe I mean, can we still interpret the sparsity results the same way we do in the lasso and if yes why, is there another way to do it using an $l_1$ norm, or does this minimization retain the lasso properties at the MLE.

$\endgroup$
  • 3
    $\begingroup$ What are the "lasso properties at the MLE"? $\endgroup$ – AdamO Apr 9 '18 at 2:23
  • 1
    $\begingroup$ @AdamO consistency in estimation and selection, under some regularity conditions first introduced by Tibshirani and then expanded by Hui Zou in 2006. $\endgroup$ – Wis Apr 9 '18 at 2:25
  • $\begingroup$ Wis, There was a related question recently, but I cannot find it anymore. Did you delete it? $\endgroup$ – Richard Hardy Apr 9 '18 at 5:24
  • 1
    $\begingroup$ @RichardHardy yes I did, it was put on hold as it was unclear. $\endgroup$ – Wis Apr 9 '18 at 5:25
0
$\begingroup$

The MLE estimate that you get for your parameters will not include the prior you have of B5 being positive. However unless your model has a hard time fitting and you end up having negative estimates for B5, it's unlikely that not introducing that prior will change your interpretation in any meaningful way - it might just mean that your confidence intervals would have been slightly different if the prior had been there.

$\endgroup$
  • $\begingroup$ Could you please clarify this answer so that I can accept it. $\endgroup$ – Wis Apr 19 '18 at 3:37
  • $\begingroup$ What do you mean by "clarify" ? $\endgroup$ – f.g. Apr 22 '18 at 13:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.