1
$\begingroup$

I split the data 80/20 as follows:

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

Then, I cross validate a decision tree as follows:

from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import cross_validate

tree_clf1 = DecisionTreeClassifier(criterion='entropy')

metrics = ['accuracy', 'precision', 'recall', 'f1']

tree_clf1_scores = cross_validate(tree_clf1, X_train, y_train, scoring=metrics, cv=5, return_train_score=True)

I understand that the model is trained using 4 folds and tested using the remaining fold to get the test_score but I wonder how the train_score is calculated when I specify return_train_score=True?

I don't find any info from the docs: https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.cross_validate.html

$\endgroup$

1 Answer 1

2
$\begingroup$

Since you pass cv=5, the function cross_validate performs k-fold cross-validation, that is, the data (X_train, y_train) is split into five (equal-sized) subsets and five models are trained, where each model uses a different subset for testing and the remaining four for training. For each of those five models, the train scores are calculated in the same manner as the test scores in the sense that the same functions for computing the metrics are used, but for the 80% of the data it was trained on as opposed to for the other 20%.

I hope that helps.

$\endgroup$
2
  • $\begingroup$ essentially, for the train score, the model is trained and tested using the same dataset, correct? $\endgroup$
    – dami.max
    Commented Apr 14, 2022 at 18:47
  • $\begingroup$ yes - that is my understanding at least $\endgroup$ Commented Apr 14, 2022 at 19:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.