I'm looking for some advice on a general approach to optimise the training of a neural network. My primary concern is to avoid over-fitting to the training data and maintain as much generality as possible.
I'm using the Resilient Backproppgation method and as such need to optimise:
- number of epochs
- learning rate
- network topology (number of neurons in hidden layer)
I had in my mind that I could perform the following process:
Load LabelledData.csv
Foreach epoch
{
// Randomly split labelled data into 90% training and 10% test
// Backprop the training data
// Calculate error against test data
}
plot(epoch, error)
// Iterate number of neurons, epochs and learning rate then repeat
The drawback with this method is that over time, the network is trained with all the data since a portion is never removed absolutely before training. If I was to remove the test set prior to beginning training, how could I perform a, say, k-fold cross validation, to ensure that the test set was representative of the training data?
Thank you for your help!