Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
In an article by Hofmann pdf, he proposes:
initialize $β$ to one, run until convergence, then rescale $β$ by a factor $η<1$, run again until convergence, and iterate this until changing $β$ no longer improves the result.
I used the TEM algorithm on a dataset for my project, but the problem is that the algorithm ends up traped in several local minima and doesn't give me full convergence. My question is: what is the optimal condition under which I can assume that the algorithm converged?
The first and still popular method for ensuring convergence relies on line searches, which optimize a function along one dimension. A second and increasingly popular method for ensuring convergence uses trust regions. Both line searches and trust regions are used in modern methods of non-differentiable optimization. Usually a global optimizer is much slower than advanced local optimizers, so often an efficient global optimizer can be constructed by starting the local optimizer from different starting points.
If you are converging to a local minima instead of the global minima, there may be a problem with the values used for $β$ or $η$. You may try initializing $β$ to a different value, or using a larger scaling factor $η$ (closer to one).
