2
$\begingroup$

What exact kind of architecture of neural networks do I need for a sequence binary/multiclass classification? The sequences can be of different length and are to be discriminated by a certain occurrence of smaller subsequences in it.

It would be great if you can provide Lasagne/Keras layers setup and their parameters.

Thanks.

$\endgroup$
4
  • $\begingroup$ "smaller subsequences in it." Are these subsequences of the same length? $\endgroup$ Dec 17, 2016 at 16:43
  • $\begingroup$ subsequences may be of any length too, every big sequence is labeled with target variable $\endgroup$
    – Sengiley
    Dec 17, 2016 at 16:47
  • $\begingroup$ position of a symbol or a subsequence do not discriminate classes, certain symbols tend to occur in the beginning or at the end of the sequence for all classes, but their positions and permutations within smaller subseuqences matter $\endgroup$
    – Sengiley
    Dec 17, 2016 at 16:54
  • $\begingroup$ You can use either RNN or CNN. For example, there is a nice article about using LSTMs for sequence classification in Keras here. $\endgroup$
    – Yuri
    Jul 17, 2017 at 20:47

2 Answers 2

3
$\begingroup$

I recommend to use a combination of CNN layers and a RNN layer (e.g. long short-term layer LSTM or gated recurrent units).

It depends on your sequence classification problem, I tried to solve a typical application (e.g. in bioinformatics) and failed.

Then I used this network architecture - which I found by very experienced data scientist - and suddenly it worked very well.

Network architecture for sequence classification using CNN and RNN

$\endgroup$
4
  • $\begingroup$ Welcome to the site, @Wolfgang123. Note that your username, identicon, & a link to your user page are automatically added to every post you make, so there is no need to sign your posts. In fact, we prefer you don't. $\endgroup$ Jan 17, 2019 at 16:22
  • $\begingroup$ thank you for this info, in the future, I will not sign any post. $\endgroup$ Jan 17, 2019 at 16:25
  • 1
    $\begingroup$ Could you share who you got this architecture from? Sharing sources would be an ideal way to improve your answer. $\endgroup$ Jan 20, 2019 at 0:35
  • $\begingroup$ @AlfredoHernández The architecture is explained in slightly more detail here: community.wolfram.com/groups/-/m/t/1135708 $\endgroup$
    – RNs_Ghost
    Mar 13, 2019 at 11:25
1
$\begingroup$

For sequence classification, I would recommend an RNN like LSTM with an Attention layer added. Adding Attention significantly improves the output because now you are paying attention to all hidden states of the RNN layer and not just the last one. Each hidden state is assigned a attention weight and has a 'say' in determining the final label.

A simple sequence classification implementation is explained here: https://stackoverflow.com/questions/63060083/create-an-lstm-layer-with-attention-in-keras-for-multi-label-text-classification/64853996#64853996

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.