2
$\begingroup$

I am following along Ian Goodfellow's new Deep Learning book and, reading the last chapter, I am confused about equations 20.7-20.9.

We have a joint distribution function, $P(v,h)$, and we are interested in finding the conditional distribution function, $P(h|v)$. If, from the definition of the Restricted Boltzmann Machine,

$$ P(v, h) = \frac{1}{Z} \exp(b^Tv +c^Th + v^TWh) $$ where $Z$ is the normalizing constant,

$$ Z = \sum_v \sum_h \exp(b^Tv +c^Th + v^TWh). $$

$P(h|v)$ is then (copied from the chapter):

\begin{align} P(h|v) &= \frac{P(h,v)}{P(v)} \tag{20.7}\\[5pt] &= \frac{1}{P(v)}\frac{1}{Z}\exp(b^Tv +c^Th + v^TWh) \tag{20.8}\\[5pt] &= \frac{1}{Z'}\exp(c^Th + v^TWh) \tag{20.9} \end{align}

That last step is where I am confused. What is $Z'$ -- there is nothing about that in the text? Is it some other constant that serves as a new normalizing constant or is it actually a derivative with respect to either $v$ or $h$? Could someone fill in some missing steps?

$\endgroup$
2
$\begingroup$

P(h/v) implies v is given/known. So 1/Z' should lump together all the constant terms i.e. exp(b'v)/(P(v) * Z)

$\endgroup$
1
$\begingroup$

In my opinion, what happens here is treating P(h|v) as a "function of h, knowing v". Everything what has not a h in itself is treated as a constant part and put in a new normalizing constant - Z'.

Intuitively - computations of a partition constant - Z or Z' - are intractable. That's why you use e.g. a Monte Carlo method to approximate it. That's why - in my opinion - author wasn't concentrated on full scrutiny during writing this set of equations.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.