3
$\begingroup$

I would like to use the F1-score metric for crossvalidation using sklearn.model_selection.GridSearchCV. My problem is a multiclass classification problem. I would like to use the option average='micro' in the F1-score.

See also: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html#sklearn.metrics.f1_score

I already checked the following post: https://stackoverflow.com/questions/34221712/grid-search-with-f1-as-scoring-function-several-pages-of-error-message

If I try exactly what is standing in this post, but I always get this error:

TypeError: f1_score() missing 2 required positional arguments: 'y_true' and 'y_pred'

My question is basically only about syntax: How can I use the f1_score with average='micro' in GridSearchCV?

I would be very grateful for any answer.

EDIT: Here is an executable example:

import numpy as np
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.pipeline import Pipeline
from sklearn.metrics import f1_score, make_scorer
from sklearn.preprocessing import RobustScaler
from sklearn.svm import SVC


data = load_breast_cancer()
X = data['data']
y = data['target']

#X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33)

scaler = RobustScaler()
estimator = SVC()

pipeline_steps = [('scaler', scaler), ('estimator', estimator)]
pipeline_steps = Pipeline(steps=pipeline_steps)

grid = [{'estimator__C': [0.1, 0.5, 1.5, 2, 2.5, 3]}]

gridsearch = GridSearchCV(estimator=pipeline_steps,
                       param_grid=grid,
                       n_jobs=-1,
                       cv=5,
                       scoring=make_scorer(f1_score(average='micro')))

# now perform full fit on whole pipeline
gridsearch.fit(X, y)
print("Best parameters from gridsearch: {}".format(gridsearch.best_params_))
print("CV score=%0.3f" % gridsearch.best_score_)
cv_results = gridsearch.cv_results_
#print(cv_results)
$\endgroup$
1
$\begingroup$

Ok, I found it out:

If you use scoring='f1_micro' according to https://scikit-learn.org/stable/modules/model_evaluation.html, you get exactly what I want.

| cite | improve this answer | |
$\endgroup$
0
$\begingroup$

You can follow the example that is provided here, simply pass average='micro' to make_scorer. https://scikit-learn.org/stable/modules/generated/sklearn.metrics.make_scorer.html

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thx for your help. When I do this, I get the following error: scoring=make_scorer(f1_score), average='micro') TypeError: __init__() got an unexpected keyword argument 'average' --> I added a full code example at my post $\endgroup$ – jordin1987 Nov 21 '19 at 11:07
  • $\begingroup$ this is the correct way make_scorer(f1_score, average='micro'), also you need to check just in case your sklearn is latest stable version $\endgroup$ – Yohanes Alfredo Nov 21 '19 at 11:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.