20
$\begingroup$

RNN can be used for prediction, or sequence to sequence mapping. But how can RNN be used for classification? I mean, we give a whole sequence one label.

$\endgroup$
1
  • $\begingroup$ I would try to predict the next term using the previous. I would then prepend that network to a radial basis function (of 3*classlabelcount or so gaussian pdf neurons) whose inputs are the prediction errors of the series and whose outputs are the class labels. I would have a decent number of neurons in the input layer - enough to account for how many elements of the sequence are required to classify. This is just my personal approach. $\endgroup$ Dec 17 '14 at 3:38
14
$\begingroup$

One can use RNN to map multiple input to a single input (label), as this give figure (source) illustrates:

enter image description here

Each rectangle is a vector and arrows represent functions (e.g. matrix multiply). Input vectors are in red, output vectors are in blue and green vectors hold the RNN's state (more on this soon). From left to right: (1) Vanilla mode of processing without RNN, from fixed-sized input to fixed-sized output (e.g. image classification). (2) Sequence output (e.g. image captioning takes an image and outputs a sentence of words). (3) Sequence input (e.g. sentiment analysis where a given sentence is classified as expressing positive or negative sentiment). (4) Sequence input and sequence output (e.g. Machine Translation: an RNN reads a sentence in English and then outputs a sentence in French). (5) Synced sequence input and output (e.g. video classification where we wish to label each frame of the video). Notice that in every case are no pre-specified constraints on the lengths sequences because the recurrent transformation (green) is fixed and can be applied as many times as we like.

$\endgroup$
4
$\begingroup$

In case of simple RNN, feed entire sequence to your network and then output class label at the last sequence element (see this paper and references there for early example of this approach). In training phase we can backpropogate error in time from last sequence element to the start of the sequence. In general this is no different from RNN sequence labeling problem, where we need to assign labels only to some elements of the sequence (or all other elements are labeled as OTHER).

$\endgroup$
1
  • $\begingroup$ Care must be exercised for longer sequences. Information from the start of the sequence can get overwhelmed by the information down the sequence. $\endgroup$ Nov 13 '15 at 6:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.