Typically one uses a RBM in an unsupervised fashion. But it is stated that we can do otherwise. As the title says how does one train a supervised RBM?

My idea is the following: by clamping feature and class information to the visible units we let the RBM associate the features with the labels. In the end the RBM should be able to reconstruct the class label simply by seeing the features.

Concretely I mean the following:

Assume you have two features f1, f2 and a binary class label c1 - we clamp f1,f2,c1 for every train example to the visibles and proceed with our training (CD1 or whatever).

Is this a valid procedure or not? If not please tell me why not and where I can find more information about my problem!

Thank you very much

  • $\begingroup$ Which papers have you read? $\endgroup$ – Neil G Mar 8 '15 at 23:54
  • $\begingroup$ I googled for it, opened a few papers, searched for the key phrases but wasn't happy with the results. So can't really say which papers I read (I really skimmed over them) $\endgroup$ – Mike Dooley Mar 8 '15 at 23:56
  • $\begingroup$ I'm in the middle of something, will get back to you in a few minutes… $\endgroup$ – Neil G Mar 8 '15 at 23:58

See Hinton, G. E. (2007). To Recognize Shapes, First Learn to Generate images. Progress in Brain Research, 165, 535–547.

Look at figure 4. They do something like your idea, but you should look at the architecture.

  • $\begingroup$ thank you for your contribution. the paper looks promising but it will take me some time to digest. $\endgroup$ – Mike Dooley Mar 9 '15 at 10:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.