# Why Energy in Restricted Boltzmann Machine?

In Restricted Boltzmann Machine (RBM), we define the energy function $$E(\mathbf{v}, \mathbf{h}; \, \mathbf{W}, \mathbf{a}, \mathbf{b})$$.

• $$\mathbf{v}$$ is visible unit
• $$\mathbf{h}$$ is hidden unit
• $$\mathbf{W}$$ is the connection matrix between $$\mathbf{v}$$ and $$\mathbf{h}$$
• $$\mathbf{a}, \mathbf{b}$$ are the bias vector for visible and hidden units

And the aim is to learn the parameter $$\mathbf{W}$$, $$\mathbf{a}$$ and $$\mathbf{b}$$. from the training sample.

My question: What is the intuition behind this setup? In particular, the concept of energy function $$E$$ is not found in other machine learning methods (e.g. ANN, CNN ...etc).

I know that RBM is related to statistical mechanics in Physics (e.g. Ising Model and Inverse Ising Problem), but I don't really understand why such concept is useful in machine learning.

Thanks.

• Which papers have you read? Please add the citations to your question. – Neil G Jan 21 at 1:33
• I have edited the question with a paper added. – K_inverse Jan 21 at 1:46

## 1 Answer

Artificial neural networks can be written as energy-based models. See LeCun, Yann, et al. "A tutorial on energy-based learning." Predicting structured data 1.0 (2006).

The energy-based model framework is just a convenient, intuitive way of thinking about models.

• On p.11 of the paper (Sec. 2.2: Examples of Loss Functions), it stats that the loss function can be chosen simply as the energy function -- Eqn. (6). Does it mean that ANN can be regarded as EBM? – K_inverse Jan 22 at 2:20
• @K_inverse no... – Neil G Jan 22 at 2:22