In Neural Networks the bias term of the hidden units can be considered a threshold for the node to fire. This is how neurons basically work in the brain as well. In RBMs, also a bias for the visible layer exists. How can this bias be interpreted and what purpose does it serve?

  • $\begingroup$ Actually the bias in a NN hidden layer can also be interpreted as some sort of normalizer. For example when in an image one pixel is always white (or black), the bias can take account of that. On the other hand, in an RBM the visible layer bias can take account of a latent variable, which is always on as well. For example in a distribution p(v, h), of movies v (visible) to genres h (hidden), a very un-specific genre of which almost all movies are, should have a low overall impact on the reconstruction. $\endgroup$ – Chris Mar 11 '17 at 13:13
  • 1
    $\begingroup$ Possible duplicate of What do the bias units represent in a restricted boltzmann machine? $\endgroup$ – Sycorax Aug 20 '18 at 0:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.