2
$\begingroup$

In Restricted Boltzmann Machine (RBM), we define the energy function $E(\mathbf{v}, \mathbf{h}; \, \mathbf{W}, \mathbf{a}, \mathbf{b})$.

  • $\mathbf{v}$ is visible unit
  • $\mathbf{h}$ is hidden unit
  • $\mathbf{W}$ is the connection matrix between $\mathbf{v}$ and $\mathbf{h}$
  • $\mathbf{a}, \mathbf{b}$ are the bias vector for visible and hidden units

And the aim is to learn the parameter $\mathbf{W}$, $\mathbf{a}$ and $\mathbf{b}$. from the training sample.

My question: What is the intuition behind this setup? In particular, the concept of energy function $E$ is not found in other machine learning methods (e.g. ANN, CNN ...etc).

I know that RBM is related to statistical mechanics in Physics (e.g. Ising Model and Inverse Ising Problem), but I don't really understand why such concept is useful in machine learning.

Related paper: Restricted Boltzmann Machines for Collaborative Filtering

Thanks.

$\endgroup$
3
  • $\begingroup$ Which papers have you read? Please add the citations to your question. $\endgroup$
    – Neil G
    Commented Jan 21, 2019 at 1:33
  • 1
    $\begingroup$ I have edited the question with a paper added. $\endgroup$
    – K_inverse
    Commented Jan 21, 2019 at 1:46
  • $\begingroup$ Originates from Sherrington–Kirkpatrick model $\endgroup$ Commented Feb 10, 2022 at 16:58

1 Answer 1

1
$\begingroup$

Artificial neural networks can be written as energy-based models. See LeCun, Yann, et al. "A tutorial on energy-based learning." Predicting structured data 1.0 (2006).

The energy-based model framework is just a convenient, intuitive way of thinking about models.

$\endgroup$
2
  • $\begingroup$ On p.11 of the paper (Sec. 2.2: Examples of Loss Functions), it stats that the loss function can be chosen simply as the energy function -- Eqn. (6). Does it mean that ANN can be regarded as EBM? $\endgroup$
    – K_inverse
    Commented Jan 22, 2019 at 2:20
  • $\begingroup$ @K_inverse no... $\endgroup$
    – Neil G
    Commented Jan 22, 2019 at 2:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.