I am making a machine learning model with supervised classification. I want to input my data, cross validate my model and then test it on a test set. Finally, I will use this to transform/make predictions for another dataset.
When it comes to creating the model I've been using cross validation on the training set (I have another set to test with once the cross validation is over), but I wasn't using it to test different models (I only input one model, decision tree classifier) or fine tune any paramaters. I am using random stratified 5 fold cross validation on the training set to get a confusion matrix/contingency table for each fold which I then average. Is this acceptable use of cross validation, or should I try fine tuning hyperparameters? Thanks
