I'm an academic researcher, working with Convolutional Neural Networks, particularly for image classification. In academic publications, a typical metric for evaluating the performance of a recognition pipeline is the classification accuracy. What I am wondering, is exactly at what point during the training stage this number is taken.
For example, in my experiments, I train the network with back propagation, and reduce the learning rate over time. For this, I observe the testing accuracy, and reduce the rate by a certain amount whenever this accuracy is no longer increasing. However, what I notice is that once the system has converged, and I continue to train with minibatches, the overall testing accuracy still varies after each minibatch by around 1%, even though the average testing error over all minibatches is constant.
So, my questions are:
When reporting the testing accuracy in an academic publication, is it acceptable to simply take the highest accuracy over all these minibatches? Or should something more representative be used, such as an average over all minibatches?
Sometimes, the testing accuracy actually begins to fall as further training is carried out, due to overfitting. Is it acceptable to stop the training at this point, and report this peak testing accuracy, even though the testing dataset is distinct from the training dataset (i.e. the validation set is not a subset of the training dataset)?