I'm designing a CNN model for a data mining competition in which we are provided with N sample of training data. We do not know the test size, but presumably it is from the same distribution as training set is from.
I obtained my best result while using 85% of data for training and 15% for validation to prevent the CNN from overfitting.
However different split sizes and data shuffling lead to different accuracies.
So, I wasn't sure if this type of training is reliable especially when we do not know the test size?