3
$\begingroup$

I have been using R's glmnet for training elastic net models for both regression and classification problems. In glmnet, you have to set the parameter family as family="binomial" for binary classification tasks and family="gaussian" for regression tasks. This is very much in line with the theory as the cost function for regression and that for classification differs a lot (the cost function of regression is primarily dependent on the difference between predicted and actual quantitative value whereas the cost function for classification from the context of logistic regression is based on sigmoid function and log values).

In scikit-learn, the corresponding function for building Elastic Net model is ElasticNetCV and there is no mention of selecting a loss function or something which is intuitively similar to the usage of glmnet for classification problems. It looks like scikit-learn's ElasticNetCV can only be used for regression tasks in the strict sense. But I have seen many people using the same for classification tasks as well. Is this a valid practice?

$\endgroup$
4
  • 1
    $\begingroup$ You can, and you would be estimating a linear probability model (en.wikipedia.org/wiki/Linear_probability_model), which has some issues. On the subject of a clean pure-python glmnet, I am working on it: github.com/madrury/py-glm $\endgroup$ – Matthew Drury Sep 25 '17 at 20:36
  • $\begingroup$ Yes, I agree. But the model will be highly biased if there are outliers. So I do not think it is a good practice to use ElasticNetCV to do classification tasks. $\endgroup$ – j1897 Sep 25 '17 at 20:41
  • 2
    $\begingroup$ Agreed, it's generally not a great idea. $\endgroup$ – Matthew Drury Sep 25 '17 at 20:42
  • $\begingroup$ It's unfortunate, but sometimes sklearn coincides well with what you want to do, and sometimes it doesn't. When it doesn't coincide well, you're left with the choice of either hammering it into place or starting from scratch. $\endgroup$ – Sycorax Sep 25 '17 at 21:10
5
$\begingroup$

You can use the elasticnet penalty in sklearn's Logistic Regression classifier:

from sklearn.linear_model import LogisticRegression

lr = LogisticRegression(penalty = 'elasticnet', solver = 'saga', l1_ratio = 0.5)

LogisticRegressionCV will take these parameters as well

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.