4
$\begingroup$

We know that LASSO and ridge and ElasticNet all apply regularization terms on the coefficients of least squares regression. However, I have not yet found any R / python libraries that compute regularization of of Least Absolute Deviation (LAD):

$$\sum |y-X\beta|+\lambda||\beta||$$

using either the $l1$ norm (equivalent of LASSO) or $l2$ norm (equivalent of ridge). Does it make sense to apply regularization to LAD?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Please register &/or merge your accounts (you can find information on how to do this in the My Account section of our help center), then you will be able to edit & comment on your own question. $\endgroup$
    – gung - Reinstate Monica
    Nov 12, 2018 at 20:24
  • 1
    $\begingroup$ Note that with LAD regression and an L1 penalty, you can use the same trick to turn it into a pure LAD regression problem as you can use with least squares regression and an L2 penalty: augment the data with several observations with target variable value 0 and right hand side variable values equal to $\lambda$. Not efficient, but at least it works. $\endgroup$
    – jbowman
    Nov 12, 2018 at 20:47

3 Answers 3

Reset to default
5
$\begingroup$

Check out rqPen and hqreg packages in R which claim to perform quantile regression with lasso and elastic net respectively. Maybe you know this already but least absolute deviation regression is median regression or quantile regression at the 50% percentile. Minimizing the absolute deviation results in the median (with potential problem of multiple solutions), same as minimizing the squared deviation results in the mean.

Improve this answer
$\endgroup$
1
$\begingroup$

If you use the SGDRegressor in scikit-learn with the epsilon_insensitive loss function specified and the epsilon value set to zero, you will get a model equivalent to LAD with L2 regularization.

Improve this answer
$\endgroup$
0
$\begingroup$

QuantileRegressor in scikit-learn is (rather heavily, by default) regularized.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.