1
$\begingroup$

I have a large dataset with 6 predictors, with a goal to predict bank loan interest, based on year income, time at work, loan amount, credit balance, credit utilization rate, etc. I use python with scikit-learn to do this prediction, specifically ElasticNet. When I train the model and check final model coefficients, I see something like [0.09342575, 0.06866633, 0.01102091, 0.07865676, -0. , 0.16359722] which means that some predictors are more important than others, so predictor #5 got coefficient = 0, and probably does not influence the final result much (it might change if I change ElasticNet parameters, though). Does it mean that I should reconsider my selection of predictors, remake training data set (it might be useful in terms of memory utilization, and calculation time for example), exclude this "unimportant" predictor #5 and re-run the training? Or should I leave them all in the model and dataset?

BTW, is there a way to use ElasticNet with some categorical data (like home ownership: rent, own, mortgage, etc), somehow encoded? For neural net it would be one-of-N, like "rent" is "0,0,1", own is "0,1,0", etc, but how about ElasticNet?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Elastic Net could be used to do variable selection and prediction.

In the case of prediction, the advantage of elastic net is that it doesn't allow really high coefficients, contrary to the most classic cases of linear regression.

The first case, variable selection, is what you face when the coefficient is equal to 0. Elastic net balance between a lasso and ridge model, with lasso nullifying coefficients and ridge lowering them. Consequently, the parameter lambda that you choose can make some of your coefficients be equated to 0. If you are happy with the selected variables and the associated coefficients, you could run a classic model with only the variables kept through lasso. And enjoy the easy interpretation of a linear model.

The second case, prediction, is a prediction with the constrained coefficients. It is better to get rid of the null variable before to train your model, for an easier readibility.

For your categorical data, your could add the modalities as flag. Not sure though how python handle them.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Does it mean that if i remove variable with 0 coefficient from the data set, then run model again and find that some other variable now got 0 coefficient in my new model, I can remove associated variable too, and get even smaller data set? Where to stop in this case? $\endgroup$
    – lugger1
    Commented Feb 12, 2016 at 16:44
  • $\begingroup$ Try it, but if the variables had been selected by the algorithm because they were useful to predict the outcome. The coefficients of the model without the 0 variable should not be really far from the coefficients of the model with the 0 variable. The thing, here, is that the 0 variable is useless, so it's better to drop it. $\endgroup$
    – YCR
    Commented Feb 12, 2016 at 16:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.