# Energy function of Restricted Boltzmann Machine (RBM)

The energy function for RBM (Restricted Boltzmann Machine) is defined as $$E(v,h) = -\sum_{i,j} w_{ij} \, v_i \, h_j -\sum_i a_i \, v_i - \sum_i b_i \, h_i$$ with the joint distribution $$\tag{1} p(v, h) = \frac 1 Z \exp(-E(v,h))$$

I am thinking why this form of potential function (or energy function) can satisfy the Markov properties of RBM graph?

I've read some relevant posts (e.g., 1 and 2) but they are answered with high level arguments and none of them have convinced me mathematically.

To answer this question, I have read through the proof of Hammersley-Clifford Theorem and I've noticed the potential function constructed in the proof is of the following form: $$\phi_a(x)=\sum_{b: b\subseteq a}(-1)^{|a\backslash b|} H_b(x) = \sum_{b: b\subseteq a}(-1)^{|a\backslash b|} \log f(x_b, *)$$ ($$\phi_a(x)=0$$ if $$a$$ is not a clique)

and the theorem proves the joint probability can be factorized into: $$\tag{2} f(x)=\exp \sum_{a:a\subseteq V} \phi_a(x)$$ ($$V$$ are all vertices in the graph)

and this distribution will respect the Markov properties (by (F)=>(G)=>(L)=>(P)=>(F), see the proofs in Chapt16 of Jordan's book draft).

So without considering the normalization (i.e., $$\frac1Z$$), we can simply compare RHS of equation (1) and (2) and I think they have to be the same form (otherwise no theorem says (P)=>(F)). What I do not get is that they look very similar but the signs do not match.

To be more specific, as for RBM, I let $$\left\{ \begin{array}{rl} f(v_i,h_j,*) &= \exp(w_{ij}\, v_i \, h_j) > 0\\ f(v_i,*) &= \exp(a_i\, v_i) > 0\\ f(h_j,*) &= \exp(b_j\, h_j) > 0\\ f(\emptyset,*) &= c > 0\\ \end{array} \right.$$ Plugging into equation (2) I can get \begin{align} f(v,h) &= \exp(\sum_{i,j}\phi_{v_i,h_j}(x) + \sum_i \phi_{v_i}(x) + \sum_i \phi_{h_i}(x) + C) \\ &\propto \exp(\sum_{i,j}\phi_{v_i,h_j}(x) + \sum_i \phi_{v_i}(x) + \sum_i \phi_{h_i}(x)) \\ &= \exp(\sum_{i,j}\phi_{v_i,h_j}(x) + \sum_i (-1)^0 a_i\, v_i + \sum_i (-1)^0 b_j\, h_j) \\ &= \exp(\sum_{i,j}\phi_{v_i,h_j}(x) + \sum_i a_i\, v_i + \sum_i b_j\, h_j) \\ &= \exp(\sum_{i,j} ( (-1)^{|2-2|} w_{ij}\, v_i \, h_j + (-1)^{|2-1|} a_i\, v_i + (-1)^{|2-1|} b_j\, h_j ) + \\ & \sum_i a_i\, v_i + \sum_i b_j\, h_j) \\ &= \exp(\sum_{i,j} ( w_{ij}\, v_i \, h_j - a_i\, v_i - b_j\, h_j) + \sum_i a_i\, v_i + \sum_i b_j\, h_j) \\ \end{align} since each vertex generally has more than 1 degree in RBM graph, the last equation is equal to $$f(v,h) = \exp(\sum_{i,j} w_{ij}\, v_i \, h_j - L \sum_i a_i\, v_i - M \sum_i b_j\, h_j)$$ where $$L, M > 0$$.

I compare it to equation (1), and it seems the signs and coefficients do not match. Why is it?

Aha, I think by defining these functions instead $$\left\{ \begin{array}{rl} f(v_i,h_j,*) &= \exp(w_{ij}\, v_i \, h_j) > 0\\ f(v_i,*) &= \exp(-a'_i\, v_i) > 0\\ f(h_j,*) &= \exp(-b'_j\, h_j) > 0\\ f(\emptyset,*) &= c > 0\\ \end{array} \right.$$
I can get to the expected form: \begin{align} f(v,h) &= \exp(\sum_{i,j} w_{ij}\, v_i \, h_j + L\sum_i a'_i\, v_i + M \sum_i b'_j\, h_j) \\ &= \exp(\sum_{i,j} w_{ij}\, v_i \, h_j + \sum_i a_i\, v_i + \sum_i b_j\, h_j) \end{align}