I want to know whether a Deep Belief Network (or DBN) is a supervised learning algorithm or an unsupervised learning algorithm?

After lot of research into DBN working I am confused at this very question. Some of the papers clearly mention DBN as unsupervised and uses supervised learning at at one of its phases -> fine tune. Some other sites clearly specifies DBN as unsupervised and uses labeled MNIST Datasets for illustrating examples.

There are some papers stress about the performance improvement when the training is unsupervised and fine tune is supervised. To top it all in a DBN code, at fine tune stage labels are used to find difference for weight updating.

So what I understand is DBN is a mixture of supervised and unsupervised learning. Why is it is then everywhere mentioned as unsupervised?


Deep belief networks are generative models and can be used in either an unsupervised or a supervised setting.

From Wikipedia:

When trained on a set of examples without supervision, a DBN can learn to probabilistically reconstruct its inputs. The layers then act as feature detectors. After this learning step, a DBN can be further trained with supervision to perform classification.

If you have seen it mentioned as an unsupervised learning algorithm, I would assume that those applications stop after the first step mentioned in the quotation above and do not continue on to train it further under supervision. In that case it seems perfectly accurate to refer to it as an unsupervised method.

  • $\begingroup$ So an algorithm that is fully unsupervised and another one that contains supervised learning in one its phases both are apt to be termed as unsupervised ?? $\endgroup$
    – Devi
    Dec 29 '17 at 3:43
  • $\begingroup$ I'm just saying if you don't do the last phase, then it is unsupervised. It doesn't matter that it can then go on to be supervised, if you use it in an unsupervised way then what you have done is unsupervised learning. This is really just an argument over semantics, just think of this as two algorithms, the first is an unsupervised learning algorithm, the second is supervised and just happens to do everything the first algorithm does as its initial steps. The confusion is that they have the same name, and so to distinguish them people refer to the first as unsupervised. $\endgroup$
    – Dan
    Dec 29 '17 at 12:33
  • $\begingroup$ Ok. So I wonder if DBN could be used for unlabelled dataset ? $\endgroup$
    – Devi
    Dec 30 '17 at 8:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.