What is the better estimate of how well your NN is doing?

Plotting how weights change with the number of iterations or Plotting how the error changes with the number of iterations ?!


I like error in - error out approach. The basic idea that you split your data set into train and validation. You use the train data for the learning process and validation just to test the prediction error for the data which network didn't see before. On each epoch you compute your training error and after that you just run throw the network validation data set part (no learning procedure at this part!) and compute the error for the validation data set. At this case error in would be the error for training data and the error out for the validation one. After some number of iteration you will plot all your errors on the one plot. Example you can find on figure below:


On figure you can see two separated lines for both errors. From this plot you can see your learning progress from the epochs. Also you can track an overfitting. On this plot there is no overfitting, because green line goes monotonically down. If it increases on each epoch from some point it will be overfitting and you will easily find it on the plot.

Also some times can be useful to make a logarithmic scale for the x-axis when number of training epochs is very big.

  • $\begingroup$ Excellent. So is there a need to plot the changes in the weights vs number of iterations ? $\endgroup$ – Mustafa Sep 4 '15 at 11:17
  • $\begingroup$ If you want visualize weights the good way to do it is Hinton diagram. But it not very useful if you have a huge number of weights. $\endgroup$ – itdxer Sep 4 '15 at 13:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.