# Detailed mathematical derivation of the energy function and joint probability of restricted Boltzmann machine

I am wondering how the joint distribution of restricted Boltzmann machine (RBM) is mathematically derived. Consider a RBM with input layer $$\bf x$$ (with bias $$\mathbf b$$), hidden layer $$\bf h$$ (with bias $$\bf c$$) and connection matrix $$\bf W$$, most of the tutorials that I've seen first directly define the energy function:

$$E(\mathbf{x},\mathbf{h})=-\mathbf{h}^{T}\mathbf{Wx}-\mathbf{b}^{T}\mathbf{x}-\mathbf{c}^{T}\mathbf{h}$$

and then directly tells me that the joint probability is:

$$p(\mathbf{x},\mathbf{h}) = \frac{e^{-E(\mathbf{x},\mathbf{h})}}{\sum_{x,h} e^{-E(\mathbf{x},\mathbf{h})}}$$

and this makes me hard to understand why they are defined in this way. I know it somehow relates to some conclusions in statistical mechanics, but I have never taken a course in this field.

I hope to know if there is a clear and detailed mathematical proof of the above formulas. Specifically, from just the following conditional probabilities:

$$p(\mathbf{h}|\mathbf{x})=\mathrm{sigmoid}(\mathbf{Wx}+\mathbf{c})$$ $$p(\mathbf{x}|\mathbf{h})=\mathrm{sigmoid}(\mathbf{W}^{T}\mathbf{h}+\mathbf{b})$$

is there a way to derive both the energy function and joint probability shown above?

• Lookup Boltzmann distribution, that will answer 75% of your question Oct 25, 2021 at 12:35
• @Aksakal Already checked the Wikipedia page of Boltzmann distribution, but I am still confused about how the formulas of joint probability and the two conditional probabilities are related to each other. Any other recommendations? Oct 25, 2021 at 12:46
• if you understand how logistic regression works in this regard then this paper will help to understand multinomial case: jstor.org/stable/2336391 Oct 25, 2021 at 12:52