I am struggling with the interpretation of my elastic net results and hope someone might be able to help ...
I've done an elastic net regression in R (based on glmnet), with different levels of alpha and selected a final model. I have around 15 predictors, and 3 are highly correlated.
Now two correlated predictors occur in the results, with one having a negative sign and the other having a positive sign. I wonder how this can be interpreted?
What happens with correlated predictors is that it's hard to say exactly how either is associated with outcome with the other one held constant, which is the interpretation of a regression coefficient. If one is changing, the other is likely to be changing too. So there are high-magnitude correlations among the coefficient estimates for correlated predictors, which you can see by looking at the coefficient covariance matrix of the model (e.g., the vcov() function in R applied to the model).
The way I think about this type of situation is that one of the predictors is tending to overfit its relationship with outcome, and the second is taking advantage of its correlation with that first predictor to correct a bit for that overfitting (or vice-versa). That's just a heuristic, and you should think about how well that explains your situation.
