Following is a Sigmoid Belief Networks where we can only observe the bottom observable layer $v.$ Usually we use Wake-Sleep to trainning the bottom-to-up Recognition Weight $r_{ji}$ and up-to-bottom Generative Weight $w_{ij}.$ The conditional probability of all the nodes except for the top layer root nodes $(h^{(2)}_1,h^{(2)}_2)$ are assumed to be the Bernoulli distribution with Logistic probability function. There is no assumption of the probability function on $(h^{(2)}_1,h^{(2)}_2).$

As a generative model, how do we generate the samples of roots $(h^{(2)}_1,h^{(2)}_2)?$ enter image description here


1 Answer 1


From this tutorial:

To do this, for each unit of the top layer, the sigmoid function is simply applied to the additive constant in (1), as there are no subsequent layers providing influence.

This means only a bias is associated with the top random variables to parametrize the logistic function:

$p(h^{(2)}_1=1)=\frac{1}{1 + \exp(-b^{(2)}_1)}$ and $p(h^{(2)}_2=1)=\frac{1}{1 + \exp(-b^{(2)}_2)}$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.