# Optimising neural network to prevent overfitting

I'm looking for some advice on a general approach to optimise the training of a neural network. My primary concern is to avoid over-fitting to the training data and maintain as much generality as possible.

I'm using the Resilient Backproppgation method and as such need to optimise:

• number of epochs
• learning rate
• network topology (number of neurons in hidden layer)

I had in my mind that I could perform the following process:

Load LabelledData.csv

Foreach epoch

{

// Randomly split labelled data into 90% training and 10% test

// Backprop the training data

// Calculate error against test data

}

plot(epoch, error)

// Iterate number of neurons, epochs and learning rate then repeat


The drawback with this method is that over time, the network is trained with all the data since a portion is never removed absolutely before training. If I was to remove the test set prior to beginning training, how could I perform a, say, k-fold cross validation, to ensure that the test set was representative of the training data?