# Derivation of the gradient of log likelihood of the Restricted Boltzmann Machine using free energy method

I am reading "Machine Learning: A Probabilistic Perspective". While I understand the other proof in 27.7.2.1: Deriving the gradient using $p(\boldsymbol{h}, \boldsymbol{v}|\boldsymbol{\theta})$, I don't quite understand the next section 27.7.2.2 Deriving the gradient using $p(\boldsymbol{v}|\boldsymbol{\theta})$.

In section 27.7.2.2, equation 27.114 ~ 27.117

\begin{align} & F(\boldsymbol{v}) \\ = & \sum_\boldsymbol{h} E(\boldsymbol{v}, \boldsymbol{h}) \\ = & \sum_\boldsymbol{h} \exp\Bigg(\sum_{r=1}^{R}\sum_{k=1}^{K}v_r h_k W_{rk}\Bigg) \\ = & \quad ... \\ = & \prod_{k=1}^{K}\Bigg(1+\exp\bigg(\sum_{r=1}^{R}v_r W_{rk}\bigg)\Bigg) \\ \end{align}

and equation 27.122

$$\frac{\partial}{\partial w_{rk}} F(\boldsymbol{v}) = -\mathbb{E}[v_r h_k| \boldsymbol{v}, \boldsymbol{\theta}]$$

I have little idea why the gradient of the free energy $F(\boldsymbol{v})$ turns out to be like that.

Also why does the $E(\boldsymbol{v}, \boldsymbol{h})$ expands into a $\exp(\cdot)$ thing while none of the previous pages of that do?

• What does the indexing over $\bf h$ mean? Nov 21, 2016 at 15:13
• From section 27.7 you can tell. $\boldsymbol{h}$ are hidden nodes; $\boldsymbol{v}$ are visible nodes... Nov 22, 2016 at 13:34

I think you might find your answer in one of the two following videos. The first video shows the derivation of the free energy. The second video shows the derivation of the gradient. I believe both videos assume binary units.

Restricted Boltzmann machine - free energy https://youtu.be/e0Ts_7Y6hZU?list=PL6Xpj9I5qXYEcOhn7TqghAJ6NAPrNmUBH

Restricted Boltzmann machine - contrastive divergence (parameter update) https://youtu.be/wMb7cads0go?list=PL6Xpj9I5qXYEcOhn7TqghAJ6NAPrNmUBH

EDIT: To answer your second question about where does the $exp()$ term come from, after conferring with some colleagues, I believe [2] actually contains a typo and Eqn. 27.114 should read:

$$F(v)=\sum_{h}\exp(E(v,h))=\sum_{h}\exp(\sum_{r=1}^{R}\sum_{k=1}^{K}v_{r}h_{k}W_{rk})$$

The exp() term doesn't just appear, it seems to have accidentally been left out. That correction makes the derivation consistent with [1] Eqn. 18

$$FreeEnergy(x)=-log\sum_{h}\exp(-Energy(x,h))$$

The $-log()$ term shown above is accounted for in [2] in Eqn. 27.118. Also [1] uses $x$ instead of $v$.

Note sure what you mean in your first question. The FreeEnergy term is actually inspired by the Ising model from physics, which models statistical mechanics of neuron dynamics (the extent of my knowledge of where the FreeEnergy definition comes from ends there). To minimize the cross-entropy of an RBM requires taking the gradient of the FreeEnergy to determine how to update the system parameters. By minimizing the system energy we increase the probability to observe a given visible layer and it's associated hidden layer (i.e. learn the distribution of the data). Hope that in some part steers you in the right direction?

[1] Bengio, Yoshua. "Learning deep architectures for AI." Foundations and trends® in Machine Learning 2.1 (2009): 1-127.

[2] Kevin P. Murphy. 2012. Machine Learning: A Probabilistic Perspective. The MIT Press.

• Can you summarise the arguments so this remains valuable even if the videos are withdrawn? Nov 21, 2016 at 15:03
• I actually have watched those videos but the textbook looks deeper into the proof. I still hope someone can answer after half a year... Nov 22, 2016 at 13:38
• Apart from the missing $\exp(.)$, it looks like it should have the $-\log(.)$, as you have pointed out, to make sense when plugging it to the immediate previous paragraph's $p(\boldsymbol{v}|\boldsymbol{\theta}) = \frac{1}{Z(\boldsymbol{\theta})}\exp(-F(\boldsymbol{v};\boldsymbol{\theta}))$; I don't think equation 27.118 accounted for the $\log(.)$ there. For the expansion of $E(\boldsymbol{v}, \boldsymbol{h})$, the paragraph after equation 27.100 makes it more confusing. The youtube video [1] has explained this quite well in this regard. Mar 1, 2017 at 13:31
• Some pages' explanation of the book is quite confusing as it doesn't use consistent notations e.g. confusing $(\ldots;\boldsymbol{\theta}), (\ldots|\boldsymbol{\theta}), (\dots)$. For equation 27.122, I think it should be clear after clearing up the confusing parts. Mar 1, 2017 at 13:32
• It looks like this errata isn't documented here cs.ubc.ca/%7Emurphyk/MLbook/errata.html as well Mar 1, 2017 at 13:35