According to Hinton's slides (slide 34) the following relationship holds for Boltzmann Machines in thermal equilibrium:
$$ p(v,h) = \frac{e^{-E(v,h)}}{\sum_{u,g}{e^{-E(u,g)}}} $$
To me this is not obvious at all. Why does the energy and the probability have this very specific kind of relationship? What is the proof for this?
Recognizing this relation is what ultimately allowed the explanation of classical thermodynamics on the basis of individual molecules. The adaptation of this idea to machine learning is relatively recent.
Concepts like temperature, heat, and energy had proved to be very useful by the middle of the 19th century. More than a hundred years earlier, Daniel Bernoulli, nephew of the Bernoulli most beloved by statisticians, had postulated that gases consist of large numbers of individual particles that we call molecules. But the connection between those larger-scale concepts and molecular-scale processes was missing until Maxwell, of Maxwell's equations fame, explained properties of diffusion by suggesting in 1859 that molecules had a particular distribution of different velocities.
Boltzmann showed that you could explain the distribution of molecular velocities with a model in which the molecules bounce off each other. Say that the total energy (the sum of all the individual molecules' energies) and the number of molecules are constant. As the molecules bounce off each other exchanging energy elastically in thermal equilibrium, they explore all the possible energy states (distributions of energy among the molecules) of the system as a whole.
If all distinguishable distributions of energy among the molecules (at constant total system energy) are equally probable, then an exponential relation between the energy of a substate of the system and its probability necessarily follows. For example, if 2 molecules exchange energy in a collision, the joint distribution of their post-collision probabilities as a function of their individual energies is the product of their individual probabilities. Yet the sum of energy of the 2 molecules after the collision is still the same as before. If you require a distribution in which the probability of a sum of 2 arguments equals the product of the individual probabilities, only an exponential relation will do. The equation in the question (the partition function) normalizes to unit probability when summed over all states.
One accessible presentation develops this more fully, starting from 6 molecules sharing a total energy of 8 units, and then generalizing. See the Wikipedia page on the Maxwell-Boltzmann distribution and links from it for more detail.
