I have a deep neural architecture. I am using a following apporoach. Lets say I have 100 datapoints(including train, test and validate)

1) Split the dataset into 3 non overplapping sets

60 : train samples 20 : Validation samples 20 : test samples

What I am doing is train on 60, and validate on 20, and test on 20 (call this as test1Accuracy). Then again do this 60-20-20 non overlapping split and then get test2Accuracy. Repeat this 5 times.

Final Accuracy = average over test1Accuracy ..... test5Accuracy.

2) Split the dataset into 3 non overplapping sets

80 : (Combined train and validation. Note, that I will have five iterations, in each 60 sample becomes train and 20 become test : I am refering to idea of cross validation)

20 : test samples (This will be used at the last, only once after the model is trained by cross validation)

The main difference is that in first approach, in every iteration is a full iteration of train-validate-test stage, and the test accuracies are averaged. While in second, using 5 train-validate iterations, a final testing is done only once on the learned weight. Second one is a general cross validaiton algorithm. Is my first approach also a valid approach or should I go with second one. Thanks in advance :)

  • $\begingroup$ This is a good question, commenting to stay informed. Are you dealing with a time series with 100 instances or a non-ordered bag of 100 observations? $\endgroup$ – ERT Aug 1 '18 at 16:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.