3
$\begingroup$

I've been using elastic net implemented in R (via glmnet) for some modeling, but I was wondering, due to the number of outliers in my data, if there was some sort of modeling approach for regularized robust regression? e.g. something like elastic net applied to robust regression. if there's something already in R, even better. Just curious to know what's out there.

$\endgroup$
  • $\begingroup$ Have a look at the answers to this similar questions. $\endgroup$ – user603 May 25 '16 at 20:39
1
$\begingroup$

There seem to be some relevant papers. You can start with:

http://projecteuclid.org/DPubS?service=UI&version=1.0&verb=Display&handle=euclid.ejs/1316092867

http://arxiv.org/abs/0811.1790

www.econ.kuleuven.be/public/ndbae06/PDF.../sparseLTS.pdf

http://www.google.cl/url?sa=t&rct=j&q=&esrc=s&source=web&cd=20&cad=rja&ved=0CGYQFjAJOAo&url=http%3A%2F%2Fwww.econ.kuleuven.be%2Fpublic%2Fndbae06%2FPDF-FILES%2FsparseLTS.pdf&ei=pSJkUJTiNYug8gTX8IDwAw&usg=AFQjCNFoKj1zoHwYIcXJDbLhUYHYDAscYw&sig2=btUee_XNA9tXs0MsJw_DNA

... but I don't know about implementations.

$\endgroup$
0
$\begingroup$

Suppose we are using L1 norm on error term, the objective is still convex.

The objective is

$$\|Ax-b\|_1+\lambda_2\|x\|^2_2+\lambda_1\|x\|_1$$

Gradient is

$$A^T \text{sign}(Ax-b)+2\lambda_2x+\lambda_1 \text{sign}(x)$$

Here is a R implementation

f<-function(x,A,b,l1,l2){
  e=A %*% x - b
  # v=crossprod(e)+l2*crossprod(x)+l1*sum(abs(x))
  v=sum(abs(e))+l2*crossprod(x)+l1*sum(abs(x))
  return(c(v))
}

gr<-function(x,A,b,l1,l2){
  v=t(A) %*% sign(A %*% x -b)
  return(c(v)+2*l2*x+l1*sign(x))
}

set.seed(0)
par(cex=1.3)
n_data=20
n_feature=2

A=matrix(runif(n_data*n_feature),ncol=n_feature)
b=matrix(runif(n_data),ncol=1)

l1=0.01
l2=0.01

opt=optimx::optimx(runif(n_feature),f, gr,A=A,b=b,l1=l1,l2=l2, method="BFGS")
opt

Here is the results and visualization

> opt
            p1        p2    value fevals gevals niter convcode  kkt1 kkt2 xtimes
BFGS 0.2600659 0.7781711 4.995662    179     29    NA        0 FALSE   NA      0

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.