0
$\begingroup$

I am trying to understand neural networks and by reading different articles I always find conflicting information. I wanted to understand which neural networks can be used as supervised/unsupervised. One of the many articles I have read is this one and an answer is the following:

"CNN is not supervised or unsupervised, it’s just a neural network that, for example, can extract features from images by dividing it, pooling and stacking small areas of the image. If you want to classify images you need to add dense (or fully connected) layers and for classification, the training is supervised. But, if you want to cluster images based on similarities of a group of images, you will extract the features use the CNN and then use an unsupervised method like k-means."

Following this logic, can all neural network layers be used as either a supervised or an unsupervised model?

Improve this question
$\endgroup$
5
  • $\begingroup$ The phrasing "used as supervised/unsupervised models" is hard to understand. Do you mean "Can all neural networks be used as both a supervised and an unsupervised model?" Or something else? Please edit to clarify. $\endgroup$
    – Sycorax ♦
    Apr 4, 2022 at 12:16
  • $\begingroup$ Thanks for making the edit. Your question asks "can all neural networks be used as either a supervised or an unsupervised model?" and then you write "SOM is only an unsupervised Neural Network." Since you already know of one example of a neural network that is only unsupervised, then we know that it's not true that all neural networks can be used as a supervised or an unsupervised model. What is there that remains to be answered? $\endgroup$
    – Sycorax ♦
    Apr 4, 2022 at 12:43
  • $\begingroup$ When you say "all neural networks" are you referring to all models (a specific configuration of layers & weights) or all layer types (CNN, RNN, FFN), or something else? $\endgroup$
    – Sycorax ♦
    Apr 4, 2022 at 13:02
  • $\begingroup$ I am referring to CNN, RNN, and others. The question published in quora talks about CNN for example. $\endgroup$
    – Inuraghe
    Apr 4, 2022 at 13:14
  • $\begingroup$ Thanks for clarifying. I've revised your question to reflect that you're asking about layers instead of all neural network models. For what it's worth, I wouldn't use Quora as my primary reference for information about neural networks; instead, I recommend starting with high-quality textbooks such as Goodfellow's Deep Learning $\endgroup$
    – Sycorax ♦
    Apr 4, 2022 at 13:20

1 Answer 1

Reset to default
3
$\begingroup$

Supervision has to do with the objective that the network is trained on - not with the network itself. An example of a "supervised" objective is to get a network to classify images into categories that humans have already labeled. You're telling the network exactly how it should behave for every input in the training set. This image should get the label "dog", that image should get the label "platypus" and so forth. This is a way of training that requires a lot of human input or "supervision".

Unsupervised training methods use an objective that requires (almost) no human input. For instance, an autoencoder is trained to encode images in such a way that the input image can be accurately reconstructed from its encoded representation. The objective then is to minimize (e.g.) the mean squared-error loss between the reconstructed images and the actual pixel values. This requires no input from humans (as far as telling the network what output it should give for a particular input).

Can any network be trained with or without supervision? In principle, yes. However, there are usually certain parts of the network that are specific to the output that network is expected to give, and not every output is suited to both supervised and unsupervised objectives. For instance, a network for image classification probably has an output layer that spits out probabilities of different classes. That output is linked to the objectives that such a network can be trained on. E.g. you cannot train this exact network on an image-reconstruction objective, without altering the architecture in such a way that the network produces images as output.

In other words, architectures typically reflect the task(s) that a network is designed for, and not every task can (easily) be cast into both supervised and unsupervised forms. So in that sense you could describe a particular architecture as "supervised", if it was designed for a supervised task or training objective. But if you're talking about a whole class of models, like all CNNs, you can't (typically) say that they are one or the other, because within any model class you (typically) have room to design a specific architecture to allow either supervised or unsupervised learning. And in any case, it is better to keep these concepts separate in your mind, since architectures and training objectives may be correlated but are very different things.

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ So does this mean that by changing the neural network architecture it is possible to do both supervised and unsupervised? CNN or RNN can be used for everything, right? $\endgroup$
    – Inuraghe
    Apr 4, 2022 at 12:50
  • $\begingroup$ Something like that. New entrants to the field, impo, underestimate the level of human engagement required to establish and maintain capably performing machine learners. $\endgroup$
    – EngrStudent
    Apr 4, 2022 at 12:59
  • $\begingroup$ It seems that this answer suggests "All neural networks can be used in either a supervised or an unsupervised fashion." And this answer uses auto-encoders as an example of an unsupervised neural network. How can an autoencoder be used in a supervised fashion? $\endgroup$
    – Sycorax ♦
    Apr 4, 2022 at 13:03
  • $\begingroup$ An autoencoder is really an architecture plus a task. The architecture itself is just a function (parameterized by a neural network) mapping a tensor to another tensor. That same architecture could be used on a different task where the output tensor is evaluated by a supervised metric. $\endgroup$
    – Ruben van Bergen
    Apr 4, 2022 at 14:15
  • $\begingroup$ It seems that you're saying that the term "supervised" or "unsupervised" describes the metric use to train the model, not the model itself. This is a valid perspective, but it doesn't really come through clearly from how the answer is written. Perhaps you could edit to clarify? $\endgroup$
    – Sycorax ♦
    Apr 4, 2022 at 17:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.