I have a dataset with $n=1500$ observations and $p=2700$ variables.
I fitted an Elastic Net model with $\alpha=0.4$ and $\lambda=0.1$
I chose the $\lambda$ with cross validation, and the $\alpha$ and $\lambda$ values that gave the lowest RMSE (0.88 for the test set).

from the plot of observed vs predicted the model is not a great fit. enter image description here

How can I improve my model?

  • $\begingroup$ This could be a problem of scaling your variables (incorrectly). $\endgroup$ – Arya McCarthy Jun 11 at 16:33
  • $\begingroup$ I used the R function of glmnet, which standardizes the variables. I also try to standardize on my own and set standardize=False, but got the same results. $\endgroup$ – ari6739 Jun 11 at 17:39

Your model seems to be systematically biased: it systematically overpredicts small actuals, and underpredicts large actuals. To be honest, I am a bit surprised this would happen in your setup.

One way to address this is to use a very simple second model. Feed it the actual outcome as the target variable, and the fits/predictions of your elastic net as the only predictor. Based on your plots, I would assume this second model (which could be as simple as a linear regression) will give you a positive coefficient, so it will reduce low predictions and increase large predictions.

  • $\begingroup$ Do you mean $Y \sim predict(model)$ ? the plot looks a bit better. but, is there a way to change the original model to fit better? $\endgroup$ – ari6739 Jun 11 at 16:17
  • $\begingroup$ Yes, I do. As I wrote, your plot is not what I expect from an Elastic Net, so with Arya I suspect some bug. $\endgroup$ – Stephan Kolassa Jun 12 at 9:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.