In Restricted Boltzmann Machine (RBM), we define the energy function $E(\mathbf{v}, \mathbf{h}; \, \mathbf{W}, \mathbf{a}, \mathbf{b})$.

  • $\mathbf{v}$ is visible unit
  • $\mathbf{h}$ is hidden unit
  • $\mathbf{W}$ is the connection matrix between $\mathbf{v}$ and $\mathbf{h}$
  • $\mathbf{a}, \mathbf{b}$ are the bias vector for visible and hidden units

And the aim is to learn the parameter $\mathbf{W}$, $\mathbf{a}$ and $\mathbf{b}$. from the training sample.

My question: What is the intuition behind this setup? In particular, the concept of energy function $E$ is not found in other machine learning methods (e.g. ANN, CNN ...etc).

I know that RBM is related to statistical mechanics in Physics (e.g. Ising Model and Inverse Ising Problem), but I don't really understand why such concept is useful in machine learning.

Related paper: Restricted Boltzmann Machines for Collaborative Filtering


  • $\begingroup$ Which papers have you read? Please add the citations to your question. $\endgroup$ – Neil G Jan 21 at 1:33
  • 1
    $\begingroup$ I have edited the question with a paper added. $\endgroup$ – K_inverse Jan 21 at 1:46

Artificial neural networks can be written as energy-based models. See LeCun, Yann, et al. "A tutorial on energy-based learning." Predicting structured data 1.0 (2006).

The energy-based model framework is just a convenient, intuitive way of thinking about models.

  • $\begingroup$ On p.11 of the paper (Sec. 2.2: Examples of Loss Functions), it stats that the loss function can be chosen simply as the energy function -- Eqn. (6). Does it mean that ANN can be regarded as EBM? $\endgroup$ – K_inverse Jan 22 at 2:20
  • $\begingroup$ @K_inverse no... $\endgroup$ – Neil G Jan 22 at 2:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.