# How to train LSTM model on multiple time series data?

How to train LSTM model on multiple time series data?

Use case: I have weekly sales of 20,000 agents for last 5 years. Need to forecast upcoming weekly sales for each agent.

Do I need to follow a batch processing technique - take one agent at a time, train LSTM model then forecast? Any better ways?

• did you figure this out? I am looking at a similar problem. Nov 20, 2017 at 20:12
• @vishnuviswanath we are working on developing a neural net(RNN) which can be a one model for all agents. Nov 28, 2017 at 6:52
• thanks. how are you training the model for multiple agents? are you training one agent per batch. Nov 28, 2017 at 15:36
• we are still in construction stage. kaggle.com/c/web-traffic-time-series-forecasting/discussion/… can help. Nov 29, 2017 at 9:41

Make the identity of the agent one of the features, and train on all data. Probably train on a mini-batch of eg 128 agents at a time: run through the time-series from start to finish for those 128 agents, then select a new mini-batch of agents. For each mini-batch, run a slice of say 50 timesteps, then backprop. Keep the end states from that slice, and run the next 50 timesteps, starting from those end states. Rinse and repeat till you get to the end of the timesteps, for that mini-batch of ~128 agents.

By putting the identity of each agent as one of the features, you make it possible for the network to:

• learn from all of your data, thereby maximizing the utilization of your data, and
• learn the unique characteristics of each agent, so that it won't just average all of the agents together
• when you predict the future for a specific agent, make sure to use their corresponding agent id features, and the network will adjust the predictions accordingly.

Edit: Alpo Jose wrote:

ok, do I need to use one-hot encoding to make identity of agent?

Ooo, that's true. There are 20,000 of them. That's kind of a lot. I think that what you might want to do is 'embed' them. Have a lookup layer, that takes in an agent id (expressed as an integer, an index), and outputs a high-dimensional vector, like something like a 50-300 length vector, probably a vector whose length approximately matches the hidden size of your LSTM.

Mathematically, a lookup table, also called an 'embedding layer', is equivalent to making the agent ids into one-hot vectors, then passing through a linear (fully-connected) layer. However, the requirements in memory are much reduced for the embedding layer.

In terms of what the embedding layer will learn, as you train it, the embedding layer will form some kind of latent representation of each agent. The latent representation won't likely be readable/interpretable in any way, but will allow the model to learn things like 'ok this agent, 1524, is relatively effective, but not on weekends; whereas 1526 is great every day; etc ....'. The latent dimensions of the embedding vector might actually mean stuff, but no-one ever tries to figure out what they mean (I think that would be hard/impossible). However, the high-dimensional per-agent embeddings allow the model to learn something about the behavior of each agent, and model this in the time-series predictions.

• ok, do I need to use one hot encoding to make identity of agent ? Jul 26, 2018 at 5:38
• @AljoJose updated the answer to take into account this question Jul 26, 2018 at 8:45
• I understand now, Will give a try. thank you Hugh. Jul 26, 2018 at 11:03
• I have a similar problem, but unlike the agents here, I have multiple time series as instances of the same process, but each of variable length. So I do not need the agent( process, in my case) as a feature. Any suggestions how to handle that? Apr 11, 2019 at 11:03
• I have 10 agents and 100 time series for each agent (i.e. 1000 time series in total), and I'm using this keras neural network: model = Sequential(); model.add(Dense(100, activation='relu', input_dim=X_train.shape[1])); model.add(Dense(1)); model.compile(loss='mse', optimizer='adam'); model.fit(X_train.values, Y_train, epochs=20) where X_train.shape = (1000, 30) and Y_train.shape = (1000,), i.e. each time series has 30 timesteps. In this case how can we use the agent id as a feature? Mar 7 at 18:10