7
$\begingroup$

How to train LSTM model on multiple time series data?

Use case: I have weekly sales of 20,000 agents for last 5 years. Need to forecast upcoming weekly sales for each agent.

Do I need to follow a batch processing technique - take one agent at a time, train LSTM model then forecast? Any better ways?

$\endgroup$
  • $\begingroup$ did you figure this out? I am looking at a similar problem. $\endgroup$ – vishnu viswanath Nov 20 '17 at 20:12
  • $\begingroup$ @vishnuviswanath we are working on developing a neural net(RNN) which can be a one model for all agents. $\endgroup$ – Aljo Jose Nov 28 '17 at 6:52
  • $\begingroup$ thanks. how are you training the model for multiple agents? are you training one agent per batch. $\endgroup$ – vishnu viswanath Nov 28 '17 at 15:36
  • $\begingroup$ we are still in construction stage. kaggle.com/c/web-traffic-time-series-forecasting/discussion/… can help. $\endgroup$ – Aljo Jose Nov 29 '17 at 9:41
7
$\begingroup$

Make the identity of the agent one of the features, and train on all data. Probably train on a mini-batch of eg 128 agents at at time: run through the time-series from start to finish for those 128 agents, then select a new mini-batch of agents. For each mini-batch, run a slice of say 50 timesteps, then backprop. Keep the end states from that slice, and run the next 50 timesteps, starting from those end states. Rinse and repeat till you get to the end of the timesteps, for that mini-batch of ~128 agents.

By putting the identity of each agent as one of the features, you make it possible for the network to:

  • learn from all of your data, thereby maximizing the utilization of your data, and
  • learn the unique characteristics of each agent, so that it wont just average all of the agents together
  • when you predict the future for a specific agent, make sure to use their corresponding agent id features, and the network will adjust the predictions accordingly.

Edit: Alpo Jose wrote:

ok, do I need to use one hot encoding to make identity of agent ?

Ooo, that's true. There are 20,000 of them. Thats kind of a lot. I think that what you might want to do is 'embed' them. Have a lookup layer, that takes in an agent id (expressed as an integer, an index), and outputs a high-dimensional vector, like something like a 50-300 length vector, probably a vector whose length approximately matches the hidden size of your LSTM.

Mathematically, a lookup table, also called an 'embedding layer', is equivalent to making the agent ids into one-hot vectors, then passing through a linear (fully-connected) layer. However, the requirements in memory are much reduced for the embedding layer.

In terms of what the embedding layer will learn, as you train it, the embedding layer will form some kind of latent representation of each agent. The latent representation won't likely be readable/interpretable in any way, but will allow the model to learn things like 'ok this agent, 1524, is relatively effective, but not on weekends; where as 1526 is great every day; etc ....'. The latent dimensions of the embedding vector might actually mean stuff, but no-one ever tries to figure out what they mean (I think that would be hard/impossible). However, the high-dimensional per-agent embeddings allow the model to learn something about the behavior of each agent, and model this in the time-series predictions.

$\endgroup$
  • $\begingroup$ ok, do I need to use one hot encoding to make identity of agent ? $\endgroup$ – Aljo Jose Jul 26 '18 at 5:38
  • $\begingroup$ @AljoJose updated the answer to take into account this question $\endgroup$ – Hugh Perkins Jul 26 '18 at 8:45
  • $\begingroup$ I understand now, Will give a try. thank you Hugh. $\endgroup$ – Aljo Jose Jul 26 '18 at 11:03
  • $\begingroup$ I have a similar problem, but unlike the agents here, I have multiple time series as instances of the same process, but each of variable length. So I do not need the agent( process, in my case) as a feature. Any suggestions how to handle that? $\endgroup$ – Anakin Apr 11 at 11:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.