What is meant by the term "convergence" in Restricted Boltzmann Machine? I have come across the term "convergence" in training RBM. Can someone give a brief definition / explanation of it?
 A: Depending on which neural net (NN) you are using and what you are doing, NN tries to minimize some cost function $f(W)$ where $W$ are weights.  For example, for autoencoders the cost function involves reconstruction error. If you are doing logistic regression, the cost function would be related to classification error.  NN tries to find a $W$ that minimizes the cost.
Generally in optimization two quantities are monitored for convergence: $f(W)$ and $W$. If you observe either your cost $f$ or your parameter $W$ does not vary much from one iteration ($iter-1$) to the next ($iter$), the convergence has happened and the loop stops:
$|f(W_{iter}) - f(W_{iter-1})| < \epsilon$
OR
$||W_{iter} - W_{iter-1}|| < \epsilon$
A: That mean that NN in finite iterations (epochs) can approximate local/global minimum of function as close as you need. In math term you can think as:
$\mid W_{real\ minimum} - W_{approximated\ minimum}\mid \leq \epsilon$
Where $\epsilon$ is some small value which allows you to adjust the similarity of two values.
