I have trained my neural networks on full dataset and found out that the best architecture contains 27 hidden neurons. Is it appropriate for me to perform k-fold validation solely for the network with 27 hidden neurons? Or i must apply k-fold validation during my training process?
closed as unclear what you're asking by Michael Chernick, SmallChess, kjetil b halvorsen, Peter Flom♦ Apr 17 '18 at 13:12
Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
I think you may be misunderstanding the incentive to use K-fold CV, I will explain my understanding of it.
K-Fold CV is usually employed to improve the validity of your test data set. In cases where you may have a limited size data-set it's important to maximize the amount of data you train your network with; the more data your network trains with the more variation it has experience dealing with. This, however, limits the size of the portion of your data-set put aside for testing. The issue with this is that if your test data isn't representative of you entire data-set then your neural networks performance with it also isn't really representative either. The performance you get from a small test set is now down to 'luck of the draw'. That's an issue.
Enter K-Fold CV. The idea here is to split your small data-set up into K chunks and run your neural network K times using a different chunk for testing each time. The other K-1 chunks are used for training. Now for that particular network architecture you have K different sets of results which, combined, have been tested across the entire data-set. These results can be averaged and errors can be calculated to produce an accurate range for the performance of your network. So in essence K-Fold CV helps to relinquish the difficulties created by a undersized data-set.
Hopefully you have understood me, we're speaking the same language, and we can tackle the question you've asked.
Firstly, when you say you've trained your networks on the 'full data-set' does this mean you didn't put aside a fraction of that data-set to be tested on? If that were so I'm afraid you'll have to either find some more data that matches your data-set or you'll have to retrain your networks from scratch with a fraction of the data left to test on.
Secondly, with this in mind, how big is your data-set? Is it necessary to perform K-Fold CV as it's a costly, but sometimes valuable, process. If you think you have a smaller data-set then the answer would be a YES to the last question you've asked. You WILL need to perform K-Fold cross validation during the training process.
Lastly, you may also want to look into reserving what is called a validation set of data. This answer can give you some insight into why that may be of interest to you.