I'm using Weka multilayer-perceptron classifier to do classifications. I want to know after exactly how many epochs the neural network converges(weights don't update any more). I'm using its java API, but I cannot figure out a way to get the weight variables and test whether they're still changing or not in a loop. Can anybody help? Thanks.
Regretfully, I am not familiar with WEKA. Still, here is some ideas that might help to you to look for that you need.
There is no bound on the number of steps required for a network to converge.
I would stress two points,
- Neural networks are not guaranteed to converge to a global optima, but to a local optima. They solve non-convex problems, which suffer from that problem. Still, most times they converge to a meaningful state, i.e. they deliver correct responses on test data. This means that you need to check whether the optimization algorithm has converged to a local minimum, and whether that minimum is a good one. This leads to,
- There are two ways to test for convergence: either the weights or the value of the objective do not change significantly (up to some numerical threshold), or the error rate does not decrease further. They are not exclusive but, rather, complementary (as pointed out previously). For example, it could be the case that you are going through a plateau, i.e. a flat region of the objective where gradients are small (not significant change in weights) but you are still far away from the local optima (high error rate). At convergence to a meaningful state both criteria should be met.
You may check for different optimization methods that might be offered by WEKA, for example conjugate gradients, or stochastic gradient with momentum,...