2
$\begingroup$

In Restricted Boltzmann Machine (RBM), we define the energy function $E(\mathbf{v}, \mathbf{h}; \, \mathbf{W}, \mathbf{a}, \mathbf{b})$.

  • $\mathbf{v}$ is visible unit
  • $\mathbf{h}$ is hidden unit
  • $\mathbf{W}$ is the connection matrix between $\mathbf{v}$ and $\mathbf{h}$
  • $\mathbf{a}, \mathbf{b}$ are the bias vector for visible and hidden units

And the aim is to learn the parameter $\mathbf{W}$, $\mathbf{a}$ and $\mathbf{b}$. from the training sample.

My question: What is the intuition behind this setup? In particular, the concept of energy function $E$ is not found in other machine learning methods (e.g. ANN, CNN ...etc).

I know that RBM is related to statistical mechanics in Physics (e.g. Ising Model and Inverse Ising Problem), but I don't really understand why such concept is useful in machine learning.

Related paper: Restricted Boltzmann Machines for Collaborative Filtering

Thanks.

Improve this question
$\endgroup$
3
  • $\begingroup$ Which papers have you read? Please add the citations to your question. $\endgroup$
    – Neil G
    Jan 21, 2019 at 1:33
  • 1
    $\begingroup$ I have edited the question with a paper added. $\endgroup$
    – K_inverse
    Jan 21, 2019 at 1:46
  • $\begingroup$ Originates from Sherrington–Kirkpatrick model $\endgroup$
    – patagonicus
    Feb 10, 2022 at 16:58

1 Answer 1

Reset to default
1
$\begingroup$

Artificial neural networks can be written as energy-based models. See LeCun, Yann, et al. "A tutorial on energy-based learning." Predicting structured data 1.0 (2006).

The energy-based model framework is just a convenient, intuitive way of thinking about models.

Improve this answer
$\endgroup$
2
  • $\begingroup$ On p.11 of the paper (Sec. 2.2: Examples of Loss Functions), it stats that the loss function can be chosen simply as the energy function -- Eqn. (6). Does it mean that ANN can be regarded as EBM? $\endgroup$
    – K_inverse
    Jan 22, 2019 at 2:20
  • $\begingroup$ @K_inverse no... $\endgroup$
    – Neil G
    Jan 22, 2019 at 2:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.