How can recurrent neural networks be used for sequence classification? RNN can be used for prediction, or sequence to sequence mapping. But how can RNN be used for classification? I mean, we give a whole sequence one label.
 A: In case of simple RNN, feed entire sequence to your network and then output class label at the last sequence element (see this paper and references there for early example of this approach). In training phase we can backpropogate error in time from last sequence element to the start of the sequence. In general this is no different from RNN sequence labeling problem, where we need to assign labels only to some elements of the sequence (or all other elements are labeled as OTHER).   
A: One can use RNN to map multiple input to a single input (label), as this give figure (source) illustrates:


Each rectangle is a vector and arrows represent functions (e.g. matrix multiply). Input vectors are in red, output vectors are in blue and green vectors hold the RNN's state (more on this soon). From left to right: (1) Vanilla mode of processing without RNN, from fixed-sized input to fixed-sized output (e.g. image classification). (2) Sequence output (e.g. image captioning takes an image and outputs a sentence of words). (3) Sequence input (e.g. sentiment analysis where a given sentence is classified as expressing positive or negative sentiment). (4) Sequence input and sequence output (e.g. Machine Translation: an RNN reads a sentence in English and then outputs a sentence in French). (5) Synced sequence input and output (e.g. video classification where we wish to label each frame of the video). Notice that in every case are no pre-specified constraints on the lengths sequences because the recurrent transformation (green) is fixed and can be applied as many times as we like.

