How is the train_score from sklearn.model_selection.cross_validate calculated?

Question

I split the data 80/20 as follows:

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

Then, I cross validate a decision tree as follows:

from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import cross_validate

tree_clf1 = DecisionTreeClassifier(criterion='entropy')

metrics = ['accuracy', 'precision', 'recall', 'f1']

tree_clf1_scores = cross_validate(tree_clf1, X_train, y_train, scoring=metrics, cv=5, return_train_score=True)

I understand that the model is trained using 4 folds and tested using the remaining fold to get the test_score but I wonder how the train_score is calculated when I specify return_train_score=True?

I don't find any info from the docs: https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.cross_validate.html

Answer 1

Since you pass cv=5, the function cross_validate performs k-fold cross-validation, that is, the data (X_train, y_train) is split into five (equal-sized) subsets and five models are trained, where each model uses a different subset for testing and the remaining four for training. For each of those five models, the train scores are calculated in the same manner as the test scores in the sense that the same functions for computing the metrics are used, but for the 80% of the data it was trained on as opposed to for the other 20%.

I hope that helps.