# How to derive the gradient formula for the Maximum Likelihood in RBM?

I am learning RBM (restricted Boltzmann machine) for deep learning. The log-likelihood of RBM is given as :

$$\ln(L(\theta|v))=\ln(p(v|\theta))=\ln\frac{1}{Z}\sum_h e^{-E(v,h)}=\ln\sum_h e^{E(v,h)}-\ln\sum_{v,h}e^{-E(v,h)}$$

and its gradient w.r.t. the parameter is:

$$\frac{\partial L(\theta|v)}{\partial\theta}=-\sum_h p(h|v)\frac{\partial E(v,h)}{\partial\theta}+\sum_{v,h}p(v,h)\frac{E(v,h)}{\partial\theta}$$

I don't understand how is the gradient derived from the log-likelihood.

Although coldmoon's answer is perfectly valid, I would like to propose a slightly different approach:

First, let's simply restrict our model to be of the form $P_{model}(X) = \frac{f(x; \Theta)}{Z(\Theta)}$ with $f$ a positive parametric function and $Z$ a partition function (=normalizing coefficient).

### Main approach (where the likelihood comes from)

The objective is to learn the underlying data distribution. In practice, we minimize the distance between the model and data distributions according to the Kullback-Leibler divergence.

Let $\chi$ be the space in which samples lie (eg: ${\{0,1 \}}^{h \times w}$ for binary images), ${\bf X} \in \chi^N$ a set of N training samples:

$$D_{KL}(P_{data}, P_{model}) = \int_{x \in \chi} P_{data}(x) log\left(\frac{P_{data}(x)}{P_{model}(x)}\right) dx$$

Since the underlying distribution ruling the dataset is unknown, $P_{data}(x)$ is only known through the samples and cannot be summed over $\chi$. However, this expression is also the expectation of $\log\left(\frac{P_{data}(x)}{P_{model}(x)}\right)$, so we approximate it by an empirical expectation over ${\bf X}$:

\begin{align} D_{KL} (P_{data}, P_{model}) &= \frac{1}{N} \sum_{x \in {\bf X}} \log\left(\frac{P_{data}(x)}{P_{model}(x)}\right) \\ &= \sum_{x \in {\bf X}} \frac{1}{N} \log (\frac{1}{N}) - \sum_{x \in {\bf X}} \frac{1}{N} \log(P_{model} (x)) \quad (\text{iid samples} \rightarrow P_{data}(x) = \frac{1}{N}) \\ &= -\log(N) - \frac{1}{N} \sum_{x \in {\bf X}} \log\left(\frac{f(x, \Theta)}{Z(f,\Theta)}\right) \\ &= -\log(N) + \log(Z(f, \Theta)) - \frac{1}{N} \sum_{x \in {\bf X}} \log(f(x; \Theta)) \end{align}

### Gradient for the fitting process

We want to minimize:

$$D_{KL} (P_{data}, P_{model}) = -\log(N) + \log(Z(f, \Theta)) - \frac{1}{N} \sum_{x \in {\bf X}} \log(f(x; \Theta))$$

The derivative of the sum with respect to the model parameters identifies as an empirical expectation over the training dataset:

$$\frac{1}{N} \sum_{x \in {\bf X}} \frac{\partial \log(f(x; \Theta))}{\partial \Theta} = \left< \frac{\partial \log(f(x; \Theta))}{\partial \Theta} \right>_{x \in {\bf X} }$$

Rewriting the first term's derivative brings another expectation term:

\begin{align} \frac{\partial \log(Z(f; \Theta))}{\partial \Theta} & = \frac{1}{Z(f, \Theta)} \frac{\partial Z(f, \Theta)}{\partial \Theta} \\ & = \frac{1}{Z(f, \Theta)} \frac{\partial }{\partial \Theta} \int_{x \in \chi} f(x,\Theta) dx \\ & = \int_{x \in \chi} \frac{1}{Z(f, \Theta)} \frac{\partial f(x,\Theta)}{\partial \Theta} dx \\ \end{align}

Since $\frac{\partial f}{\partial \Theta} = f \times \frac{\partial \log(f)}{\partial\Theta}$, it comes that:

$$\frac{\partial \log(Z(X; \Theta))}{\partial \Theta} = \int_{x \in \chi} p(x; \Theta) \frac{\partial \log(f(x;\Theta))}{\partial \Theta} dx$$

which is a formal expectation on the underlying model distribution usually written as:

$$\left< \frac{\partial \log(f(x; \Theta))}{\partial \Theta} \right>_{x \sim p(x;\Theta)}$$

$$\Delta\Theta = \left< \frac{\partial \log(f(x; \Theta))}{\partial \Theta} \right>_{x \sim p(x;\Theta)} - \left< \frac{\partial \log(f(x; \Theta))}{\partial \Theta} \right>_{x \in {\bf X} }$$

# Application to RBM with binary units

Since the objective is to fit the distribution of the visible units from the RBM, the model distribution $f$ is given by:

$$p({\bf v}) = \sum_{{\bf \tilde{h}} \in \chi_h} p({\bf v},{\bf \tilde{h}}) = \frac{\sum_{{\bf \tilde{h}} \in \chi_h} f({\bf v},{\bf \tilde{h}},W)} {Z(f, \Theta)}$$

By definition, $f({\bf v}, {\bf h}, W) = exp(\sum_{(i,j) \in V} v_i w_{ij} h_j)$. Using the general expression of the gradient above, we now need to compute:

\begin{align} \frac{\partial \log \left( \sum_{{\bf \tilde{h}} \in \chi_h} f({\bf v},{\bf \tilde{h}},W) \right)} {\partial w_{ij}} &= \frac{1}{ \sum_{{\bf \tilde{h}} \in \chi_h} f({\bf v},{\bf \tilde{h}},W) } \sum_{{\bf \tilde{h}} \in \chi_h} \frac{\partial f({\bf v},{\bf \tilde{h}}, W)} {\partial w_{ij}} \end{align}

for the sake of readability, I now abbreviate the sum indexes:

\begin{align} &= \frac{1}{ Z \times p({\bf v}) } \sum_{{\bf \tilde{h}}} \frac{\partial}{\partial w_{ij}} exp(\sum_{(i,j) \in V} v_i w_{ij} \tilde{h}_j)\\ &= \frac{1}{ Z \times p({\bf v}) } \sum_{{\bf \tilde{h}}} f({\bf v},{\bf \tilde{h}},W) v_i \tilde{h}_j = \frac{v_i}{p({\bf v})} \sum_{{\bf \tilde{h}}} p({\bf v}, {\bf \tilde{h}}) \tilde{h}_j\\ &= \frac{v_i}{p({\bf v})} \sum_{{\bf \tilde{h}}} \tilde{h}_j p({\bf v}, \tilde{h}_j) p({\bf \tilde{h}}^{-j} | {\bf v}, \tilde{h}_j) \end{align}

Using a proper decomposition of the sum, it is possible to factor and cancel the states of hidden units other than $h_j$, which I designate by ${\bf \tilde{h}}^{-j}$:

\begin{align} &= \frac{v_i}{p({\bf v})} \sum_{\tilde{h_j}} \tilde{h}_jp({\bf v}, h_j) \sum_{{\bf \tilde{h}}^{-j}} p({\bf \tilde{h}}^{-j} | {\bf v}, \tilde{h}_j) \\ &= \frac{v_i}{p({\bf v})} \sum_{\tilde{h_j}} \tilde{h}_jp(\tilde{h}_j| {\bf v}) p({\bf v}) = \sum_{\tilde{h_j}} v_i \tilde{h}_j p(\tilde{h}_j| {\bf v}) \\ &= v_i E_{{\bf v}}[h_j] \end{align}

Consequently, the model expectation is given by (hang on :-) ):

\begin{align} \left< v_i E_{{\bf v}}\left[h_j\right]) \right>_{{\bf v} \sim p({\bf v})} &= \sum_{{\bf v}} p({\bf v}) \sum_{\tilde{h_j}} v_i \tilde{h}_j p(\tilde{h}_j| {\bf v}) \\ &= \sum_{{\bf v}} \sum_{\tilde{h_j}} v_i \tilde{h}_j p(\tilde{h}_j, {\bf v}) \\ &= \sum_{v_i} \sum_{\tilde{h_j}} v_i \tilde{h}_j \sum_{{\bf v}^{-i}}p(\tilde{h}_j, v_i, {\bf v^{-i}}) \\ &= \left< v_i h_j \right>_{v_i, h_j \sim p({\bf v}, {\bf h})} \end{align}

And finally, the whole gradient comes as:

$$\Delta W_{ij} = \left< v_i h_j \right>_{v_i, h_j \sim p({\bf v}, {\bf h})} - \left< v_i E_{{\bf v}}\left[h_j\right] \right>_{{\bf v} \in X} \qquad \forall (i,j) \in V$$

• might you possibly be willing to help me through this via chat? I would love to understand it. – P i Sep 24 '16 at 21:24
• You can have a look at the first section of my master thesis which contains more explanations. I can also edit my answer above if you have more specific questions, and you can contact me at nicolas.granger <at> telecom-sudparis.eu – pixelou Sep 26 '16 at 20:41

S represents all of samples you collect. Suppose that we consider only one sample and the red "v" represents the sample: I have the same problem. Refer to this book page 567 and problem 11.8

Simon Haykin. 1998. Neural Networks: A Comprehensive Foundation (2nd ed.). Prentice Hall PTR, Upper Saddle River, NJ, USA.

We're looking for long answers that provide some explanation and context. Don't just give a one-line answer; explain why your answer is right, ideally with citations. Answers that don't include explanations may be removed.

• This does not really answer the question. If you have a different question, you can ask it by clicking Ask Question. You can also add a bounty to draw more attention to this question once you have enough reputation. – gung - Reinstate Monica Jan 19 '15 at 22:21
• @gung This answer leads me to suspect that iBM does know a good answer, though. Rather than deleting it, it would be preferable if it could be expanded to include a brief synopsis of what the solution is. – whuber Jan 19 '15 at 22:32