2
$\begingroup$

We know that LASSO and ridge and ElasticNet all apply regularization terms on the coefficients of least squares regression. However, I have not yet found any R / python libraries that compute regularization of of Least Absolute Deviation (LAD):

$$\sum |y-X\beta|+\lambda||\beta||$$

using either the $l1$ norm (equivalent of LASSO) or $l2$ norm (equivalent of ridge). Does it make sense to apply regularization to LAD?

$\endgroup$
  • 1
    $\begingroup$ Please register &/or merge your accounts (you can find information on how to do this in the My Account section of our help center), then you will be able to edit & comment on your own question. $\endgroup$ – gung - Reinstate Monica Nov 12 '18 at 20:24
  • 1
    $\begingroup$ Note that with LAD regression and an L1 penalty, you can use the same trick to turn it into a pure LAD regression problem as you can use with least squares regression and an L2 penalty: augment the data with several observations with target variable value 0 and right hand side variable values equal to $\lambda$. Not efficient, but at least it works. $\endgroup$ – jbowman Nov 12 '18 at 20:47
3
$\begingroup$

Check out rqPen and hqreg packages in R which claim to perform quantile regression with lasso and elastic net respectively. Maybe you know this already but least absolute deviation regression is median regression or quantile regression at the 50% percentile. Minimizing the absolute deviation results in the median (with potential problem of multiple solutions), same as minimizing the squared deviation results in the mean.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.