2
$\begingroup$

What exact kind of architecture of neural networks do I need for a sequence binary/multiclass classification? The sequences can be of different length and are to be discriminated by a certain occurrence of smaller subsequences in it.

It would be great if you can provide Lasagne/Keras layers setup and their parameters.

Thanks.

Improve this question
$\endgroup$
4
  • $\begingroup$ "smaller subsequences in it." Are these subsequences of the same length? $\endgroup$
    – Franck Dernoncourt
    Commented Dec 17, 2016 at 16:43
  • $\begingroup$ subsequences may be of any length too, every big sequence is labeled with target variable $\endgroup$
    – Sengiley
    Commented Dec 17, 2016 at 16:47
  • $\begingroup$ position of a symbol or a subsequence do not discriminate classes, certain symbols tend to occur in the beginning or at the end of the sequence for all classes, but their positions and permutations within smaller subseuqences matter $\endgroup$
    – Sengiley
    Commented Dec 17, 2016 at 16:54
  • $\begingroup$ You can use either RNN or CNN. For example, there is a nice article about using LSTMs for sequence classification in Keras here. $\endgroup$
    – Yuri
    Commented Jul 17, 2017 at 20:47

2 Answers 2

Reset to default
3
$\begingroup$

I recommend to use a combination of CNN layers and a RNN layer (e.g. long short-term layer LSTM or gated recurrent units).

It depends on your sequence classification problem, I tried to solve a typical application (e.g. in bioinformatics) and failed.

Then I used this network architecture - which I found by very experienced data scientist - and suddenly it worked very well.

Network architecture for sequence classification using CNN and RNN

Improve this answer
$\endgroup$
4
  • $\begingroup$ Welcome to the site, @Wolfgang123. Note that your username, identicon, & a link to your user page are automatically added to every post you make, so there is no need to sign your posts. In fact, we prefer you don't. $\endgroup$
    – gung - Reinstate Monica
    Commented Jan 17, 2019 at 16:22
  • $\begingroup$ thank you for this info, in the future, I will not sign any post. $\endgroup$
    – Wolfgang123
    Commented Jan 17, 2019 at 16:25
  • 1
    $\begingroup$ Could you share who you got this architecture from? Sharing sources would be an ideal way to improve your answer. $\endgroup$
    – Alfredo Hernández
    Commented Jan 20, 2019 at 0:35
  • $\begingroup$ @AlfredoHernández The architecture is explained in slightly more detail here: community.wolfram.com/groups/-/m/t/1135708 $\endgroup$
    – RNs_Ghost
    Commented Mar 13, 2019 at 11:25
1
$\begingroup$

For sequence classification, I would recommend an RNN like LSTM with an Attention layer added. Adding Attention significantly improves the output because now you are paying attention to all hidden states of the RNN layer and not just the last one. Each hidden state is assigned a attention weight and has a 'say' in determining the final label.

A simple sequence classification implementation is explained here: https://stackoverflow.com/questions/63060083/create-an-lstm-layer-with-attention-in-keras-for-multi-label-text-classification/64853996#64853996

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.