What is the better estimate of how well your NN is doing?
Plotting how weights change with the number of iterations or Plotting how the error changes with the number of iterations ?!
I like error in - error out approach. The basic idea that you split your data set into train and validation. You use the train data for the learning process and validation just to test the prediction error for the data which network didn't see before. On each epoch you compute your training error and after that you just run throw the network validation data set part (no learning procedure at this part!) and compute the error for the validation data set. At this case error in would be the error for training data and the error out for the validation one. After some number of iteration you will plot all your errors on the one plot. Example you can find on figure below:
On figure you can see two separated lines for both errors. From this plot you can see your learning progress from the epochs. Also you can track an overfitting. On this plot there is no overfitting, because green line goes monotonically down. If it increases on each epoch from some point it will be overfitting and you will easily find it on the plot.
Also some times can be useful to make a logarithmic scale for the x-axis when number of training epochs is very big.