1
$\begingroup$

I am learning neural nets, esp. focusing on RNN for my research problem. This question has nothing exactly to do with my research.

With my understanding of RNN, I can think of it as an example of semi-supervised learning rather than supervised learning. It is because we do not have an exact data set (unsupervised, since no actual labels), but we use the shifted value of the input as the data set (makeshift labels). Hence this makes RNN a semi-supervised learning algorithm (at least for time series).

Am I correct in this understanding?

$\endgroup$
0
$\begingroup$

An RNN (or any neural network for that matter) is basically just a big function of the inputs and parameters. There are supervised models which use RNNs, unsupervised models which use RNNs, and semi-supervised models which use RNNs. How something is supervised is generally independent of the architecture used.

The most "classic" use of RNNs is in language modeling, where we model $p(x) = \prod_i p(x_{i} | x_{j<i})$, and each conditional factor is computed by the RNN. This is an unsupervised model.

$\endgroup$
0
$\begingroup$

Semi-supervised refers to using a combination of a labeled dataset (usually quite small) and a (usually much larger) unlabeled dataset.

When the labels are automatically derived from the data itself, this is these days called self-supervised learning or self-supervision. Examples in time series include predicting the next time step, or filling in a gap in the sequence.

This website by Lillian Weng is a good reference for some of the many possible strategies. https://lilianweng.github.io/lil-log/2019/11/10/self-supervised-learning.html

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.