0
$\begingroup$

For me is not clear the way to implement LOOCV in Python, I have the next Python scripts:

1) ACC = 76.92 %

pr = np.zeros(len(y_valence))
loo = LeaveOneOut()
loo.get_n_splits(flt_a)
for train_index, test_index in loo.split(flt_a):
    X_train, X_test = flt_a[train_index], flt_a[test_index]
    y_train, y_test = y_valence[train_index], y_valence[test_index]
    #pr[test_idx] = clf.predict(flt_a[test_idx])

clf.fit(X_train, y_train)
clf.score(X_train, y_train)

2) ACC = 62.5 %

loo = LeaveOneOut()

mdm = MDM()

# Use scikit-learn Pipeline with cross_val_score function
scores = cross_val_score(mdm, cov_data_train, y_valence, cv=loo)

# Printing the results
class_balance = np.mean(y_valence == y_valence[3])
class_balance = max(class_balance, 1. - class_balance)
print("MDM Classification accuracy: %f / Chance level: %f" % (np.mean(scores),
                                                        class_balance))

In both Python scripts I got diferent perfomances, I know that the behaviour between both scripts (Cros_val_score and the other one), it is a little bit diferent but I want to be sure if are correct or not.

On the other hand, according to their experience for classification: Is most common to use the form of code 1 or code 2 (Cross-Validation)?

Thanks :).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.