The following code, which uses the function
sklearn.model_selection.cross_validate and the scikit learn compatible
from sklearn.model_selection import cross_validate from xgboost import XGBClassifier from sklearn.model_selection import StratifiedShuffleSplit cv_clf = XGBClassifier(gpu_id=0, tree_method='gpu_hist', n_estimators=300, max_depth=9) cv = sklearn.model_selection.cross_validate(estimator=cv_clf, X=X, y=labels, scoring='accuracy', verbose=100, n_jobs=4, cv=10)
produces results that are wildly different from the code below, even though they're pretty much supposed to do the same thing:
sss = StratifiedShuffleSplit(n_splits=10, test_size=0.1, random_state=42) for train_indices, test_indices in sss.split(np.zeros(len(labels)), labels): X_train = dataset.data[train_indices] y_train = labels[train_indices] X_test = dataset.data[test_indices] y_test = labels[test_indices] age_train = age.values[train_indices] age_test = age.values[test_indices] X_train = np.concatenate([X_train, age_train], axis=1) X_test = np.concatenate([X_test, age_test], axis=1) cv_clf = XGBClassifier(gpu_id=0, tree_method='gpu_hist', n_estimators=300, n_jobs=4, max_depth=9) cv_clf.fit(X_train, y_train) preds = cv_clf.predict(X_test) accuracy = accuracy_score(preds, y_test) print(accuracy)
X in the first code listing is the same as
np.concatenate([dataset.data, age], axis=1)
In the first case, the accuracy is very low (40-70), in the second case, it's 88-91.