I'm implementing Naive Bayes and a Decision Tree on the same data, and I need to cross validate with Kfold.

Do I have the right sequence of events? Overall, does it make sense or am I misunderstanding something? Some of what I've read is contradictory.

  1. Train_test_split (75/25 or whatever)
  2. Fit the model with X, y Training data
  3. Predict the model with X Test data
  4. cross_val_score using K(5)fold validation using Training data (and not the whole dataset) to see how good the model can fit.
  5. Confusion matrix on y_test, y_pred (y_pred is output from #3)
  6. Classification report on y_test, y_pred (y_pred is output from #3)
  7. Display the model with training data and display it again with test data to visualize the two results.
  • $\begingroup$ This seems to short for an answer, but since there is not much else to say: Yes, this seems about right. Perhaps there would be more to say if you could elaborate what you have read that was contradictory. $\endgroup$ – Frans Rodenburg Oct 7 '17 at 7:10
  • $\begingroup$ I've seen people apply kfold validation to the whole dataset, instead of just the training portion. $\endgroup$ – Josh Bond Oct 7 '17 at 7:22
  • $\begingroup$ That would be wrong, because you then don't have an independent dataset anymore for reporting the prediction accuracy $\endgroup$ – Frans Rodenburg Oct 7 '17 at 7:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.