I am trying to get my head straight on terminology which appears confusing. I know there are three 'splits' of data used in Machine learning models.:
- Training Data - Train the model
- Validation Data - Cross validation for model selection
- Testing Data - Test the generalisation error.
Now, as far as I am aware, the validation data is not always used as one can use k-fold cross-validation, reducing the need to further reduce ones dataset. The results of which are known as the validation accuracy. Then once the best model is selected, the model is tested on a 33% split from the initial data set (which has not been used to train). The results of this would be the testing accuracy?
Is this the right way around? or is vice versa? I am finding conflicting terminology used online! I am trying to find some explanations why my validation error is larger than my testing error, but before I find a solution, i would like to get my terminology correct.