2
$\begingroup$

I try to use a LSTM model for a problem of next basket recommendation. I would like to apply the same approach as this article in Python using Keras : A Dynamic Recurrent Model for Next Basket Recommendation
So basically (tl;dr), I want to predict the items that the users will purchase in their next order based on their previous orders. Also, for each user I only need to predict items that have already been purchased by the user.

In my case, I have users whom are buying items at different timestamps. this is what my X_train looks like :

    user ID       timestamp     sequence of items    
    user1            1          array(1, 20)
    user1            2            ...       

    user2            1            ...
    user2            2            ...
    user2            3            ...

    user3            1            ...
    user3            2            ...

The sequence of items represent an array of shape (1,20). These vectors are the mean representation of each items (generated with word2vec) purchased during each sequence.

Then I design my label y_train that corresponds to the next items bought by each user at a given timestamp :

    user ID       label    
    user1         np.array(1, 6000)
    user2         ...
    user3         ... 

The labels are 1*6000 vectors [1 0 1 0 0 0 .. 1 ] : 1 indicates that the user purchased the item i, otherwise 0.

So I would like to use a LSTM model to train the past sequences of each user to predict the next purchases. Below, I define a LSTM model. I don't return the sequence because I have one label by user.

  model_rnn = Sequential()
  model_rnn.add(LSTM(20, return_sequences=False, input_shape=(None, 20)))
  model_rnn.add(Dropout(0.2))
  model_rnn.add(Dense(nb_classes)) 
  model_rnn.add(Activation("sigmoid"))                

  model_rnn.compile(loss='binary_crossentropy', optimizer="Adagrad")
  n_index = X.index.values
  n_sample = int(len(X.index.values)*0.7)
  user_index = np.random.choice(n_index, n_sample, replace=False)
  n_epochs = 10      
  for _ in range(n_epochs):
       for index in user_index:
          X_train = X.ix[index, "sequence_items"]
          X_train.reshape(1, X_train.shape[0], X_train.shape[1])
          y_train = y[index, :].toarray()
          model_rnn.fit(X_train, y_train, batch_size=1, epochs=1, shuffle=1)

As you can see, I train my LSTM with batch_size = 1 because the timestamp is different between the users. I fit the model on 70% of the users and I test the model on the rest.

My results are very poor, the top-n items recommended by the model for each user are very similar. For example, for a specific user, the model recommend items that never appear in its old sequences while normaly, it must predict items that have already been purchased.

Obviously, my approach is wrong. Maybe the design and training data aren't adapted for my goal. Have you any idea or advice to fit the data, to reach my goal ? My goal being to predict items that the user have already purchased in the past

Note : When I fit a LSTM model with only one user, I get good results (I predict the next order based on the last user orders). But this approach forces me to train N LSTM model (with N the number of users) and this approach doesn't take into consideration the patterns of purchases of each users.

Thank you,

$\endgroup$
0
$\begingroup$

Two suggestions to improve the model.

  1. Use average of word2vec to represent the item sequence may lose too much information. It would be worth trying to add an encoding layer to first get the encoded vector of each history item, then concatenate them and put into a dense layer and use that as the representation of history item sequence (you can choose a dimension say 200, 20 is too small).

  2. Enlarge the size of the LSTM. It would be better to first make sure the LSTM can overfit on the input data. From what you have described, the model may even not have learnt well even on the training set.

$\endgroup$
0
$\begingroup$

I am quite new to Neural Networks and I am also dealing with some similar problems on the same topic however because I am working based on same author's papers on Next Basket Recommendation i would suggest you to read their previous papers that you can find in this link and this link.

The DREAM model is the third approach they have applied on the same datasets for the same purpose and they got the best results, however you might get some more information where the idea came from first in the referenced papers.

The first suggestion from blacksoil should be the right one to help you but i am also struggling on the implementation of this encoding layer.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.