is it okay to include the prediction data from k-fold cross validation for machine learning training?

Question

Currently, I started to study machine learning to write an academic paper.

Lets say I have 1000 data, and I split to 70:30 for training:testing.

While training the machine learning (assume binary classification model is KNN), I added the 10-fold cross validation to validate the training accuracy as well as to avoid the over-fitting problem.

With this methods, I found the optimized hyper-parameter for machine learning classifier which shows the highest validation accuracy.

And then, I utilized the test sets, which was not used in training, to get testing accuracy.

My question is....

Is it okay to include the prediction data from cross-validation for overall accuracy?

For example,

in training, Cross-validation:

Tr_label1: true, Tr_prediction1:false

Tr_label2: true, Tr_prediction2:true

Training validation accuracy: 50%

in testing:

Tst_label1: true, Tst_prediction1:true Tst_label2: true, Tst_prediction2:true

test accuracy: 100%

over all accuracy:

3/4= 75%

Answer 1

No, it's not ok because you used your validation set(s) in hyper parameter optimization, e.g. probably for choosing the right K or distance metric for KNN algorirhm. In your toy example, validation set has lower success while test set has higher. It can happen, but the converse situation (without much gap of course if there is not overfitting) is more common since you've tuned your algorithm according to validation set(s). Unrelated to this discussion, accuracy may not be a good metric for evaluation.