Dealing with the order of features (sequences)?

Question

Assume we have following sequence database that is subsequently converted with one-hot encoding:

  1 2 3 4
0 A B C D
1 B A D NA
2 A D C NA

One-hot encoded:

A B C D
1 1 1 1
1 1 0 1
1 0 1 1

Actually, the real data has cases like co-occuring items:

  1    2    3   4
0 A,B  C        D
1 B    A,D      NA
2 A    D    C   NA

Problem:

When converting the sequential data through one-hot encoding, one key information is lost: The order (sequence) of items in the dataframe. Given that I like to make predictions based on the sequence of actions (A,B,C,D), I am puzzled how to solve this problem?

Or: Is an LSTM able to deal with this data?

Answer 1

From what I understand, you are looking for a way to evaluate the pairwise (dis)similarity between your sequences that would account for the order of the events. You can do that using the Optimal Matching OME distance for event sequences (See Ritschard et al. (2013) ). OME is implemented in the TraMineRextras R package.

Here is how you would compute the distances between your three example event sequences.

library(TraMineRextras)
tse <- seqecreate(id=c(0,0,0,0,1,1,1,2,2,2),
                  timestamp=c(1,1,2,4,1,2,2,1,2,3),
                  event = c("A","B","C","D","B","A","D","A","D","C"))

## Below are the event sequences with 
## numbers indicating time elapsed between successive transactions
tse
#[1] 1-(A,B)-1-(C)-2-(D)    1-(B)-1-(A,D)       1-(A)-1-(D)-1-(C)  

## Compute distances between sequences with a unique insertion/deletion cost
## of 1 for all events and a .1 cost for moving an event one unit of time
idcost <- rep(1, 4)
dd <- seqedist(tse, idcost=idcost, vparam=.1)

## Resulting pairwise distances
dd
#      [,1]     [,2]      [,3]
# [1,] 0.0000000 0.627451 0.6138614
# [2,] 0.6274510 0.000000 0.8000000
# [3,] 0.6138614 0.800000 0.0000000

Of course, you can try to play with the idcost and vparam parameters.

I don't know what an LSTM is. However, I can imagine that you should be able to then input the pairwise distances to an LSTM.