0
$\begingroup$

I am doing an hyperparameter tuning through GridSearchCV for a binary classification.

model=LogisticRegression(max_iter=10000, class_weight='balanced', dual=False, fit_intercept=True,
                   intercept_scaling=1, l1_ratio=None, n_jobs=None, penalty='l1', random_state=42, 
                   solver='liblinear', tol=0.0001, verbose=0, warm_start=False)
LR_penalty=model_selection.GridSearchCV(estimator=model, param_grid= hyperparameters,
                                        cv=kfold, scoring='f1').fit(X=df_features, y=df_targets)

where kfold is obtained through TimeSeriesSplit.

My doubt is on the scoring. I know that roc_auc is not good when we have imbalanced dataset because it tends to be high due to large FP, rather than large TP (True positive). So one solution would be to use the f1 score. On the other side, the imbalance can be managed by the option class_weight='balanced'.

Now my question is: if I use both these options at same time as above (correction of imbalance and f1 scoring), am I doing something redundunt? In other words, should I use either f1 score with an imbalanced or roc_auc with a balanced? or both of them at same time is fine?

Many thanks. Luigi

Improve this question
$\endgroup$
2
  • $\begingroup$ 1) Why do you think ROC AUC is problematic for imbalanced data? 2) As usual, optimize strictly proper scoring rules like log loss and Brier score. $\endgroup$
    – Dave
    Commented Jan 4, 2021 at 15:56
  • $\begingroup$ thanks for answering: 1- stats.stackexchange.com/questions/502890/… (see last comment of the response), 2- why you define log loss and Brier as strictly proper scoring? are they available as scoring metrics in skitlearn? $\endgroup$
    – Luigi87
    Commented Jan 4, 2021 at 16:07

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.