4
$\begingroup$

I would like to use the F1-score metric for crossvalidation using sklearn.model_selection.GridSearchCV. My problem is a multiclass classification problem. I would like to use the option average='micro' in the F1-score.

See also: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html#sklearn.metrics.f1_score

I already checked the following post: https://stackoverflow.com/questions/34221712/grid-search-with-f1-as-scoring-function-several-pages-of-error-message

If I try exactly what is standing in this post, but I always get this error:

TypeError: f1_score() missing 2 required positional arguments: 'y_true' and 'y_pred'

My question is basically only about syntax: How can I use the f1_score with average='micro' in GridSearchCV?

I would be very grateful for any answer.

EDIT: Here is an executable example:

import numpy as np
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.pipeline import Pipeline
from sklearn.metrics import f1_score, make_scorer
from sklearn.preprocessing import RobustScaler
from sklearn.svm import SVC


data = load_breast_cancer()
X = data['data']
y = data['target']

#X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33)

scaler = RobustScaler()
estimator = SVC()

pipeline_steps = [('scaler', scaler), ('estimator', estimator)]
pipeline_steps = Pipeline(steps=pipeline_steps)

grid = [{'estimator__C': [0.1, 0.5, 1.5, 2, 2.5, 3]}]

gridsearch = GridSearchCV(estimator=pipeline_steps,
                       param_grid=grid,
                       n_jobs=-1,
                       cv=5,
                       scoring=make_scorer(f1_score(average='micro')))

# now perform full fit on whole pipeline
gridsearch.fit(X, y)
print("Best parameters from gridsearch: {}".format(gridsearch.best_params_))
print("CV score=%0.3f" % gridsearch.best_score_)
cv_results = gridsearch.cv_results_
#print(cv_results)
$\endgroup$
1
$\begingroup$

Ok, I found it out:

If you use scoring='f1_micro' according to https://scikit-learn.org/stable/modules/model_evaluation.html, you get exactly what I want.

$\endgroup$
0
$\begingroup$

You can follow the example that is provided here, simply pass average='micro' to make_scorer. https://scikit-learn.org/stable/modules/generated/sklearn.metrics.make_scorer.html

$\endgroup$
2
  • $\begingroup$ Thx for your help. When I do this, I get the following error: scoring=make_scorer(f1_score), average='micro') TypeError: __init__() got an unexpected keyword argument 'average' --> I added a full code example at my post $\endgroup$ – jordin1987 Nov 21 '19 at 11:07
  • $\begingroup$ this is the correct way make_scorer(f1_score, average='micro'), also you need to check just in case your sklearn is latest stable version $\endgroup$ – Yohanes Alfredo Nov 21 '19 at 11:16
0
$\begingroup$

gridsearch = GridSearchCV(estimator=pipeline_steps, param_grid=grid, n_jobs=-1, cv=5, scoring='f1_micro')

You can check following link and use all scoring in classification columns.

link : https://scikit-learn.org/stable/modules/model_evaluation.html

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.