# Robust regularized regression

I've been using elastic net implemented in R (via glmnet) for some modeling, but I was wondering, due to the number of outliers in my data, if there was some sort of modeling approach for regularized robust regression? e.g. something like elastic net applied to robust regression. if there's something already in R, even better. Just curious to know what's out there.

• Have a look at the answers to this similar questions. May 25 '16 at 20:39

http://projecteuclid.org/DPubS?service=UI&version=1.0&verb=Display&handle=euclid.ejs/1316092867

http://arxiv.org/abs/0811.1790

www.econ.kuleuven.be/public/ndbae06/PDF.../sparseLTS.pdf

... but I don't know about implementations.

Suppose we are using L1 norm on error term, the objective is still convex.

The objective is

$$\|Ax-b\|_1+\lambda_2\|x\|^2_2+\lambda_1\|x\|_1$$

$$A^T \text{sign}(Ax-b)+2\lambda_2x+\lambda_1 \text{sign}(x)$$

Here is a R implementation

f<-function(x,A,b,l1,l2){
e=A %*% x - b
# v=crossprod(e)+l2*crossprod(x)+l1*sum(abs(x))
v=sum(abs(e))+l2*crossprod(x)+l1*sum(abs(x))
return(c(v))
}

gr<-function(x,A,b,l1,l2){
v=t(A) %*% sign(A %*% x -b)
return(c(v)+2*l2*x+l1*sign(x))
}

set.seed(0)
par(cex=1.3)
n_data=20
n_feature=2

A=matrix(runif(n_data*n_feature),ncol=n_feature)
b=matrix(runif(n_data),ncol=1)

l1=0.01
l2=0.01

opt=optimx::optimx(runif(n_feature),f, gr,A=A,b=b,l1=l1,l2=l2, method="BFGS")
opt


Here is the results and visualization

> opt
p1        p2    value fevals gevals niter convcode  kkt1 kkt2 xtimes
BFGS 0.2600659 0.7781711 4.995662    179     29    NA        0 FALSE   NA      0