Recently I was working on a dataset with ~300 observations and 1500 predictors. I used the
glmnet package in R to fit an elastic net model, which gave me a cross-validated (regularised) R-square of 99%. It was suggested by subject matter experts that the data might contain influential/leverage points, that were distorting the model fit. To test this, I refit my model on an 80% subsample, using the remaining 20% as a validation dataset. Sure enough, my R-square on the validation data dropped to 10%.
What are the suggested strategies for detecting/handling outliers and leverage points in wide datasets? The standard definitions for leverage and Cook's distance involve calculating the hat matrix; does this still make sense for a regularised model with $p \gg n$?
Also, is there any R package that robustifies the basic elastic net algorithm to handle outliers and influential points? (I realise that it may be hard to do this sensibly for a 1500-dimensional problem.)