1
$\begingroup$

For the classical RBM: $P(\tilde{h}|\tilde{x})=\sigma\left(\tilde{b}+W\tilde{x}\right) a$ and $P(\tilde{x}|\tilde{h})=\sigma\left(\tilde{c}+W^{T}\tilde{h}\right)$

for for hidden layer $\tilde{h}$ and visible layer $\tilde{x}$ , and weight matrix $W$ and bias vectors $\tilde{b}$ and $\tilde{c}$

Is the correct (/preferred) name for this:

  • a Binary-Binary RBM,
  • a Bernoulli-Bernoulli RBM,
  • a Binary RBM,
  • a Bernoulli RBM

I see Binary RBM most often, in papers -- if it is specified at all, which it usually isn't, this type of RBM is the default.

Bernoulli-Bernoulli RBM makes the most sense to me, as the elements in the visible and in the hidden layers are assumed to be Bernoulli distributed. Which means the take Binary values.

Bernoulli-Bernoulli also works better if we have Gaussian-Bernoulli RBMs also being talked about, as this speaks of the distrobutions of each layer. However I read papers which say Gaussian-Binary RBM, which to me is not right as Binary is not a distribution, but Gaussian is.

What is preferred?

$\endgroup$
1
$\begingroup$

I don't think it matters, since it is mostly clear from context. The hiddens are nearly always binary. Bernoulli and binary mean the same thing. If another distribution is mentioned, e.g. as in "Gaussian RBM" this refers to an RBM with Gaussian visibles.

$\endgroup$
  • $\begingroup$ While both are correct, surely one is to be preferred? Or is it audience dependent? (I might have to edit my audience into the question.) $\endgroup$ – Lyndon White Apr 25 '14 at 15:13
  • $\begingroup$ As already said--I don't think it matters. $\endgroup$ – bayerj Apr 28 '14 at 8:09
1
$\begingroup$

I second the answer from bayerj, but since I cannot comment yet, I add my own answer here:

If you are picky, you may say that the units are chosen as binary random variables, and that their pdf conditioned to the neighbourhood is the Bernoulli distribution.

Btw. it is probably safer to always specify which type of variable is used for both visible and hidden since other types exists (Gaussian for the input, ReLU for both).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.