# In the Restricted Boltzmann Machine's free energy function, how can it be simplified to only be a function of v?

I realize this is a copy of this question: How to compute the free energy of a RBM given its energy? however I am unable to comment on the best answer to ask that person to elaborate.

$$F(v) = -b^Tv - log\sum_{h} \prod_{j=1}^{M} e^{c_jh_j + v^TW_jh_j}$$

In this tutorial http://deeplearning.net/tutorial/rbm.html it simplifies to:

$$F(v) = -b^Tv - \sum_{j=1}^{M}log\sum_{h_j}e^{c_jh_j + v^TW_jh_j}$$

And we end up with:

$$F(v) = -b^Tv - \sum_{j=1}^{M}log(1 + e^{c_j + v^TW_j})$$

In the first step, what are the prerequisites I need to know to understand why the product can be moved outside the sum?

In the second step, where does the 1 come from? Where does the $h_j$ go?

Answering my own question since I figured it out.

First, it helps if you write down the full h.

In simpler terms and only displaying the relevant part:

$$\sum_{h=\{0, 1\}^M} \prod_{j=1}^M e^{q_jh_j}$$

Suppose we have M=2, h takes in the values {(0, 0), (0, 1), (1, 0), (1, 1)} so then this would be:

$$e^{q_0 0}e^{q_1 0} + e^{q_0 0}e^{q_1 1} + e^{q_0 1}e^{q_1 0} + e^{q_0 1}e^{q_1 1}$$

Which can be factored into:

$$(e^{q_0 0} + e^{q_0 1})(e^{q_1 0} + e^{q_1 1})$$

And this works in general, i.e.:

$$\sum_{h=\{1,..,K\}^M} \prod_{j=1}^M f_j(h_j) = \prod_{j=1}^M \sum_{h_j=\{1,..,K\}} f_j(h_j)$$

So in the original equation we would get:

$$F(v) = -b^Tv - log \prod_{j=1}^M \sum_{h_j=\{0,1\}} e^{(c_j + v^TW_j)h_j}$$

$$F(v) = -b^Tv - \sum_{j=1}^M log \sum_{h_j=\{0,1\}} e^{(c_j + v^TW_j)h_j}$$

$$F(v) = -b^Tv - \sum_{j=1}^M log (e^{(c_j + v^TW_j)0} + e^{(c_j + v^TW_j)1})$$

$$F(v) = -b^Tv - \sum_{j=1}^M log (1 + e^{c_j + v^TW_j})$$