For me is not clear the way to implement LOOCV in Python, I have the next Python scripts:
1) ACC = 76.92 %
pr = np.zeros(len(y_valence)) loo = LeaveOneOut() loo.get_n_splits(flt_a) for train_index, test_index in loo.split(flt_a): X_train, X_test = flt_a[train_index], flt_a[test_index] y_train, y_test = y_valence[train_index], y_valence[test_index] #pr[test_idx] = clf.predict(flt_a[test_idx]) clf.fit(X_train, y_train) clf.score(X_train, y_train)
2) ACC = 62.5 %
loo = LeaveOneOut() mdm = MDM() # Use scikit-learn Pipeline with cross_val_score function scores = cross_val_score(mdm, cov_data_train, y_valence, cv=loo) # Printing the results class_balance = np.mean(y_valence == y_valence) class_balance = max(class_balance, 1. - class_balance) print("MDM Classification accuracy: %f / Chance level: %f" % (np.mean(scores), class_balance))
In both Python scripts I got diferent perfomances, I know that the behaviour between both scripts (Cros_val_score and the other one), it is a little bit diferent but I want to be sure if are correct or not.
On the other hand, according to their experience for classification: Is most common to use the form of code 1 or code 2 (Cross-Validation)?