# Elastic net: dealing with wide data with outliers

Recently I was working on a dataset with ~300 observations and 1500 predictors. I used the glmnet package in R to fit an elastic net model, which gave me a cross-validated (regularised) R-square of 99%. It was suggested by subject matter experts that the data might contain influential/leverage points, that were distorting the model fit. To test this, I refit my model on an 80% subsample, using the remaining 20% as a validation dataset. Sure enough, my R-square on the validation data dropped to 10%.

What are the suggested strategies for detecting/handling outliers and leverage points in wide datasets? The standard definitions for leverage and Cook's distance involve calculating the hat matrix; does this still make sense for a regularised model with $p \gg n$?

Also, is there any R package that robustifies the basic elastic net algorithm to handle outliers and influential points? (I realise that it may be hard to do this sensibly for a 1500-dimensional problem.)

• – user603 Mar 15 '15 at 20:28
• @user603 Thanks for the link. On the one hand, linear in $n$ and massively parallel. On the other hand, $O(2^p)$.... – Hong Ooi Mar 16 '15 at 2:33
• the point was Cook's distance can only reveal a single outliers. You can find high dimensional methods to find more than one outliers (ROBPCA, PcaPP) on the design space and use the clean observation to fit the model using glmnet – user603 Mar 16 '15 at 10:03
• @user603 with 1500 variables though? – Hong Ooi Mar 16 '15 at 10:22
• Oh, yea: they are high dimensional algorithms: their complexity doesn't scale with the number of columns but with the number of components. Sign PCA based method will easily churn this. – user603 Mar 16 '15 at 11:29