# How can I approach this Neural Network problem?

Suppose, I have the following data-set:

... ...
... ...
AABBB  7.027  5.338  5.335  8.122  5.537  6.408
ABBBA  5.338  5.335  5.659  5.537  5.241  7.043
BBBAA  5.335  5.659  6.954  5.241  8.470  8.474
BBAAA  5.659  6.954  5.954  8.470  9.266  9.334
BAACA  6.954  5.954  6.117  9.266  9.243 12.200
AABAA  5.954  6.117  6.180  9.243  8.688 11.842
ACAAA  6.117  6.180  5.393  8.688  5.073  7.722
ABAAC  6.180  5.393  6.795  5.073  8.719  7.854
BAACC  5.393  6.795  5.796  8.719  9.196  9.705
... ...
... ...


Apparently, the feature values represent a string pattern comprising of only three letters A, B, and C.

I have to design a neural network that would be able to detect these patterns and spit out a binary representation of these strings where the letters should be encoded in 3-bit binary(one-hot encoding).

How should I approach this problem to solve it?

• This seems like five softmax outputs of three classes each.
– Sycorax
Jul 13, 2021 at 3:05
• Do you have to predict a sequence that is built on A, B, C? (if yes, then make a research for sequence prediction...) Jul 15, 2021 at 10:18
• @malocho, YES, I need to predict a sequence. However, the sequences are made of classes. This means, this is a multi-output problem. Jul 15, 2021 at 11:02
• then I would say that it is multi-labeling problem, here take a look on this paper ke.tu-darmstadt.de/publications/papers/…, there you can see related work and so on Jul 15, 2021 at 11:42
• What is the input of the model what is the expected output? Is the first column the labels (y) and the rest is x?
– hans
Jul 19, 2021 at 13:39

You should output a tensor of a shape (5,3) treated with softmax along second axis.

The label (aka target variable) may look for ABACC like this:

[[1 0 0]
[0 1 0]
[1 0 0]
[0 0 1]
[0 0 1]]



The output of the model will be numbers from 0 to 1, summing up to 1 for each row. So you have to use argmax (again with axis=1) to get answer that you can then interpret as letters:

[[0.9 0.05 0.05] # -> A
[0.2 0.8 0.0]    # -> B
[0.4 0.5 0.1]    # -> B
[0.0 0.01 0.99]  # -> C
[0.05 0.05 0.9]] # -> C



Which you will interpret as ABBCC.

BTW, before going with NN, try a simpler model, e.g. random forests to predict those letters.

• Can this problem be solved using the Sequential model? Jul 27, 2021 at 12:59
• Sure, if the input and the output are of the same shape - a sequential model will be fine.
– hans
Jul 28, 2021 at 9:26
• stackoverflow.com/questions/68521906/… Jul 28, 2021 at 9:39

I would use some feed-forward layers to encode the input features into a representation of fixed length. Then you could use either a GRU or LSTM to decode the sequence step by step. At each step you make a prediction using a softmax over 3 output nodes. The LSTM allows you to take into consideration the earlier letters. This is only necessary if later letters depend on the choice of early letters.

If that is not the case it might be easier to just identify the number of unique patters and dummy code each of the complete sequence. So rather predicting the sequence step by step you could predict the complete sequnce at ones. This way you would need many more output nodes