I am new in the field of RBMs, DBMs and I cannot understand some things. I came across the idea of feature selection using RBMs (or Deep Belief Networks). Although the Hidden nodes which make new features as a result from the processing of Visible Nodes give 0/1 as output. How do I translate the 0/1 from the Hidden Units? Which part to RBM should I take to find the best features?

  • $\begingroup$ Can you please provide a reference on a publication that performs this feature selection procedure for RBMs? Usually feature selection relates to input data and not at hidden nodes. $\endgroup$ – usεr11852 Jun 19 '19 at 9:43
  • $\begingroup$ Can you explain what do you mean feature selection relates to input data and not at hidden notes? $\endgroup$ – nikolaosmparoutis Jun 19 '19 at 9:52
  • $\begingroup$ Feature selection is relative to what is used as input in a model but it is unclear what you mean here in terms of a RBM. Is there a relevant reference to what you refer at? $\endgroup$ – usεr11852 Jun 19 '19 at 11:44

In your trained RBM model, the hidden units can indeed be seen as extracted features automatically learned from the data at the visible units. However, I think most of the time you would have a hard time trying to interpret what the learned features are, but you can try by visualizing the weights for example.

In general, RBM are used to perform feature selection in order to use a classifier on the selected features without much effort for interpreting the features (scikit-learn example). And this is a great, since you do not need to handcraft the relevant features! Looking at the weights has often the role of a sanity check to track any buggy behavior.

  • $\begingroup$ This verifies my assumption for feature selection. The hidden nodes marked with 0/1 i suppose 1 are the activated hidden nodes given the a batch input and zero are the off. Although, can a RBM suggest an input given a previous input? $\endgroup$ – nikolaosmparoutis Jun 20 '19 at 9:40
  • 1
    $\begingroup$ Yes, RBM are generative models. You can use them for image reconstruction or more generally to get what the model "believes" facing a given visible vector (hence the name deep belief networks). See for example this paper or this video by Hinton $\endgroup$ – TheCG Jun 20 '19 at 11:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.